Small Language Model

June 24, 2011 - Microsoft today announced the availability of Mu, a new device-side mini-language model built into Windows 11. The model is designed to provide users with a smarter and more natural language interaction experience, further optimizing the search function in system settings. Last year, Microsoft unveiled Copilot + PCs with dedicated Neural Processing Units (NPUs) with more than 40 TOPS of computational power.These PCs were equipped with Phi - Silica, an on-device language model for Microsoft's first-party apps as well as...

Zyphra recently officially launched Zamba2-7B, a small language model with unprecedented performance and a parameter count of 7B. The model is claimed to outperform current competitors, including Mistral-7B, Google's Gemma-7B, and Meta's Llama3-8B, in terms of quality and speed. Zamba2-7B is designed The goal is to satisfy environments that require robust language processing power but are hardware-constrained, such as on-device processing or the use of consumer-grade GPUs.By increasing efficiency without sacrificing...

The recent release of the new Zamba2-2.7B language model from Zyphra is significant in the history of small language modeling. The new model achieves significant improvements in performance and efficiency with a training dataset of approximately 3 trillion tokens, making it comparable in performance to Zamba1-7B and other leading 7B models. And most surprisingly, Zamba2-2.7B has significantly lower resource requirements for inference, making it an efficient solution for mobile device applications. Zamba2-2.7B ...

Microsoft Research announced Tuesday local time its Phi-2 Small Language Model (SML), a text-to-text AI program that is "small enough to run on a laptop or mobile device," according to a post on the X platform. With 2.7 billion parameters, Phi-2 rivals the performance of other, larger models, including Meta's Llama 2-7B (with 7 billion parameters) and Mistral-7B (another 7 billion parameter model). Microsoft Research also noted in its blog post on the Phi-2 release that...