Overview
On October 12 at the OCP Global Summit in San Jose, Microsoft unveiled Maia 300, its third-generation in-house AI chip, positioning it as the “primary AI inference system” for its products and services. Production deployments begin in 2027, with the Maia family planned to scale to multi-gigawatt build-outs. Scott Guthrie, EVP of Cloud and AI, said Maia 300 is expected to deliver 40% more tokens per megawatt than leading GPU-based systems, and a full 96-accelerator rack can reach up to 7x the token throughput of its Maia 200 predecessor.
Compared with the prior generation, Maia 300 uses TSMC’s N2P process, integrates eight stacks of HBM4 with 18 TB/s peak bandwidth within a 950W envelope, and adopts a multi-chiplet design separating compute, memory-controller and I/O functions. Its predecessor Maia 200, released this January, already powers Microsoft Copilot, GitHub Copilot and the company’s own models, deployed across US, European and Australian data centers.
Background & Interpretation
1. Technical Background and Evolution
Maia 300 is built for inference—running trained models to answer prompts, process information and execute tasks. As AI moves beyond chat toward reasoning, planning and multi-step “agentic” workflows, inference is the fastest-growing workload. The chip is optimized for long-context reasoning, latency-sensitive dialogue and sustained agentic tasks, marking a shift in compute competition from “training supremacy” to “inference efficiency.”
2. Core Drivers and Mechanisms
The core logic is an efficiency contest measured by tokens per megawatt. As AI expansion hits power constraints, more efficient inference means serving more queries per unit of electricity. Microsoft says operating experience on Maia 200 directly shaped Maia 300, aiming to cut dependence on Nvidia GPUs and improve Azure AI unit margins.
3. Market Structure and Value-Chain Impact
In-house AI silicon is now a trend: Google’s TPU, Amazon’s Trainium, Meta’s custom chips and OpenAI’s Broadcom-built accelerator have all landed. For Nvidia, growing hyperscaler design capability is a potential challenge, but training and heterogeneous workloads still rely on GPU platforms. Evercore estimates Maia will be only about 10% of Microsoft’s AI chip spend by early 2027—substitution is gradual.
Implications & Outlook
Near term, Maia 300’s 2027 scale-up limits any immediate hit to Nvidia demand. Medium term, inference efficiency (tokens per watt/megawatt) becomes the central battleground for AI infrastructure, amplifying power scarcity. Watch whether independent benchmarks validate the 40% claim.
Takeaways for Industry Participants
- Cloud providers and large enterprises should optimize on inference efficiency, not peak FLOPS.
- AI application vendors can lower unit costs of long-context, multi-step agent workloads via in-house silicon ecosystems.
- The semiconductor chain should watch rising value share of HBM4, chiplets and advanced packaging in inference chips.
This column compiles industry information and shares technical perspectives; it does not constitute investment advice.