June 24th.MicrosoftThe company today announced the launch of Mu, a new device sidebar built into Windows 11.Small Language Model. The model aims to provide users with a more intelligent and natural language interaction experience, further optimizing the search function in the system settings.
Last year, Microsoft had launched a new product with a dedicated neural processing unit (NPUThe NPUs are equipped with Phi - Silica, a native SLM (Small Language Model) used to provide linguistic intelligence for Microsoft's first-party applications as well as third-party developer applications on the device.)
Now, Microsoft is introducing Mu, a new native mini-language model, in Windows 11. Microsoft's goal was to create an intelligent AI-based agent that would be able to understand a user's natural language query and seamlessly integrate it into the existing search box in the system's Settings app to provide a smoother user experience. To that end, Microsoft is leveraging the Mu model to power the new agent feature in the Settings app, which was recently made available to Windows Preview Experience Program members in the Development Channel and is limited to users with Copilot + PCs.
According to 1AI, similar to Phi - Silica, Mu is designed to run efficiently on NPUs, capable of running locally at over 100 tokens per second. In an official blog post, Microsoft describes the design and training process of the Mu language model in detail. Below is a summary of the Mu language model:
- Mu is an encoder-decoder language model with 330 million parameters. The encoder-decoder architecture reduces first token latency by approximately 47% and improves decoding speed by a factor of 4.7 compared to a decoder model of the same size. mu employs weight sharing in some components to reduce the total number of parameters. Microsoft trained Mu on the Azure machine learning platform using NVIDIA A100 GPUs. Although Mu is one-tenth the size of a Phi - 3.5 - mini, its performance is nearly equivalent to a similarly fine-tuned Phi - 3.5 - mini. In addition, the model is more suitable for handling multi-word queries. For short or partial word inputs, the System Settings app still displays lexical and semantic search results in the search box.
For those interested in experiencing this new AI agent feature, you can download Windows 11 Build 26120.3964 (KB5058496) or later to experience this new feature in the Windows Settings app.