Microsoft has recently launched a new artificial intelligence model called “Mu,” which is embedded in Windows 11 to power an AI agent within the Settings app. This model represents a significant advancement in on-device AI technology for operating systems. Here are the key details about Mu:
Overview of Mu:
- Mu is a small language model (SLM) with 330 million parameters, designed specifically for efficient, local operation on Windows 11 devices equipped with a Neural Processing Unit (NPU), such as Microsoft’s Copilot+ PCs/
- It uses an encoder-decoder transformer architecture, which improves efficiency by separating input token encoding from output token decoding. This design reduces latency and memory overhead, resulting in about 47% lower first-token latency and 4.7 times faster decoding speed compared to similarly sized decoder-only models.
- Mu runs entirely on the device (offline), ensuring user privacy and low-latency responses, with the ability to process over 100 tokens per second. It delivers response times under 500 milliseconds, enabling seamless natural language interaction with Windows system settings.
Functionality and Use Case:
- The primary use of Mu is to power an AI assistant integrated into the Windows 11 Settings app, allowing users to control hundreds of system settings through natural language commands. For example, users can say “turn on dark mode” or “increase screen brightness,” and Mu will directly execute these commands without manual navigation through menus.
- The AI agent is integrated into the existing search box in Settings, providing a smooth and intuitive user experience. It understands complex queries and maps them accurately to system functions, substantially simplifying system configuration for users.
- Mu is particularly optimized for multi-word queries and more complex input-output relationships, while shorter or partial-word inputs still rely on traditional lexical and semantic search results within the Settings app.
Development and Training:
- Mu was trained on NVIDIA A100 GPUs using Azure Machine Learning, starting with pre-training on hundreds of billions of high-quality educational tokens to learn language syntax, semantics, and world knowledge.
- The model was further refined through distillation from Microsoft’s larger Phi models, enabling Mu to achieve comparable performance to a Phi-3.5-mini model despite being only one-tenth its size.
- Extensive fine-tuning was performed with over 3.6 million training samples covering hundreds of Windows settings, using advanced techniques like synthetic data labeling, prompt tuning, noise injection, and smart sampling to meet precision and latency targets.
Strategic Importance:
- Mu exemplifies Microsoft’s push toward privacy-preserving, local AI processing, reducing reliance on cloud connectivity and enhancing user data security by keeping all processing on-device.
- This approach also improves responsiveness and usability, making Windows 11 more accessible and user-friendly, especially for those who may find traditional settings menus complex or cumbersome.
- Mu builds on Microsoft’s earlier on-device AI efforts, such as the Phi Silica model, and signals a broader strategy to embed efficient AI capabilities directly into hardware-equipped PCs, particularly those with dedicated NPUs.
Availability:
- The AI-powered Settings agent powered by Mu is currently available for testing to Windows Insiders in the Dev Channel on Copilot+ PCs running Windows 11 Build 26120.3964 (KB5058496) or later.
Microsoft’s Mu is a cutting-edge, compact AI language model embedded in Windows 11 that enables natural language control of system settings with high efficiency, privacy, and responsiveness. It marks a significant step forward in integrating intelligent, local AI agents into mainstream operating systems.