microsoft presented Phi Silica -the first small language model (SLM) for optimization of artificial intelligence performance on personal computers. The product will help to solve the problem of high computational costs, due to which central and graphic processors are often overloaded, reducing the overall efficiency of the system.
Phi Silica, having only 3.3 billion parameters, is the most compact model in the PHI family from Microsoft. Despite its small size, it demonstrates good indicators – the delay in generating the first token is 650 tokens per second, and the energy consumption of only 1.5 watts. This allows you to free the resources of the CPU and GPU for other tasks, providing the smooth operation of other applications.
Phi Silica is designed specifically for neural processors (Neural Processing Units (NPU) of new personal computers Copilot+. The generation of the model tokens uses the KV cache on the NPU and works on the CPU, producing about 27 tokens per second. Developers will access the API Phi Silica via Windows App SDK, which will help them create innovative applications with AI in the Windows ecosystem.
The Phi Silica functionality will include the recognition of optical symbolic input (Optical Character Recognition, OCR), studio effects, live subtitles and API to track user activity. In the future, it is planned to add vectors, API to summarize the answers (Retrieval-Augmented Generation, RAG) and automatic reference of the text.
Phi Silica is part of the Phi-3 series along with Phi-3-Mini models (3.8 billion parameters), Phi-3-Small (7 billion parameters), Phi-3-Medium (14 billion parameters) and recently announced PHI-3-Vision (4.2 billion parameters). However, Phi Silica will be the first advanced small language model that will be delivered with Windows, which will mark an important step in expanding the access of end users to advanced artificial intelligence capabilities.
The first Copilot+ computers with PHI Silica support will go on sale in mid -June 2024. They will be equipped with Snapdragon X Elite and Plus processors from Qualcomm based on ARM architecture. Later in the third quarter of 2024, Copilot+ laptops are expected with Intel Lunar Lake processors.