Microsoft Maia 300 packs 288 GB of HBM4 per chip, but the 40% efficiency claim names no rival

Microsoft says its next in-house AI chip, Maia 300, will deliver "40% more tokens per megawatt compared to leading GPU-based systems," and that production deployments begin in 2027. If you are asking what Maia 300 is, it is Microsoft's third-generation inference accelerator, announced October 12, 2026, built around a very large memory system and a 96-chip rack drawing 167 kilowatts.

The 40 percent claim names no competitor, no workload and no precision setting, so it cannot be checked. The specification sheet is more useful, and it shows where the AI hardware race is heading: toward memory bandwidth and power per token, not raw compute.

What Microsoft announced

Scott Guthrie, Microsoft's executive vice president for cloud and AI, announced the plan in a post on October 12, 2026. Maia 200, the prior generation, is already in production and serving Microsoft Copilot and the company's own MAI models. Microsoft says GitHub Copilot and Microsoft Foundry will run on Maia 200 by the end of the year. Maia 300 follows with production deployments in 2027, scaling to multi-gigawatt levels. The post does not name Maia 300 sites, and Microsoft says it will say more at Ignite in San Francisco next month.

The spec sheet

Microsoft's post and StorageReview's coverage from the OCP Global Summit give these figures for each accelerator:

• Memory: eight HBM4 stacks, 288 GB of capacity and 18 TB per second of peak bandwidth, against Maia 200's 216 GB of HBM3E at 7.1 TB per second.
• Compute: 13.8 petaops at FP4 and 6.9 at FP8, up from 10.1 and 5.1, about a 35 to 37 percent gain.
• Power: 950 watts per chip, up from 750.
• Process: compute chiplets on TSMC's N2P node, with other parts reportedly on 3 nanometer.

At rack level, a Maia 300 rack holds 96 accelerators in a 167 kilowatt envelope, with 27.6 TB of HBM and 1.7 petabytes per second of aggregate bandwidth. Maia 200's rack held 48 accelerators at 64 kilowatts. Microsoft says the new rack delivers up to 7 times the rack-scale token throughput of Maia 200. A superpod reaches about 12,288 accelerators, double Maia 200's roughly 6,000.

The pattern is plain. Compute rose about 35 percent while memory bandwidth rose about 2.5 times. For serving large models to many users, that is the right trade, because inference is usually limited by how fast weights and cache can be moved, not by how fast the chip can multiply. For more on why memory is the choke point, see our analysis of how many AI agents the shipped HBM could run.

Why the 40 percent claim is hard to use

The IEEE Communications Society's technology blog was blunt about it: a rigorous assessment needs named competitors, workloads, precision settings and power boundaries, and the post provides none. Its own comparison of per-position rack power, about 1.74 kilowatts for Maia 300 against 1.97 for Nvidia's GB300, is described as simple arithmetic on reported rack figures, not measured chip power, and the GB300 numbers are labeled as reported without a cited Nvidia document.

The blog also warned that peak bandwidth is not application performance, and recommended measuring time to first token, inter-token latency, energy per token and cost per million tokens. Microsoft has not published those. Maia also runs only inside Microsoft. It is not offered as a rentable Azure instance, so outside parties cannot test it.

Rani Borkar, who leads Azure hardware systems and infrastructure, told Reuters that Maia 300 delivers "more intelligence for every watt, every dollar, and every byte of memory." That is a design goal, not a measurement.

How it compares with the last generation

The generational jump is easiest to see in the superpod. Maia 200 racks held 48 accelerators and 10.4 TB of HBM, with a rack-level FP4 peak of 485 petaops. The new rack holds 96 accelerators, 27.6 TB of HBM and a rack FP4 peak of 1,324 petaops, per StorageReview's table. Per-accelerator scale-up bandwidth rose only about 14 percent, from 2.8 to 3.2 TB per second, so most of the gain comes from packing more chips and more memory into one coordinated domain, not from each chip talking faster to its neighbors.

That also explains the power figure. A 167 kilowatt rack is more than two and a half times the 64 kilowatts of the old one, which is why Microsoft leads with efficiency. Higher density only helps if a site can supply and cool it.

What it means for the compute race

Custom silicon does not free Microsoft from Nvidia and AMD. StorageReview reports both remain in Microsoft's training fleet, and the Maia line is focused on inference, the part of the bill that grows with every user. The strategic point is cost and power control. If a hyperscaler can serve tokens on its own chips at lower megawatts per token, it can price below rivals who rent GPUs, or earn more at the same price.

Power is the binding constraint here, which is why the unit Microsoft chose is tokens per megawatt, not tokens per dollar. Data center sites are limited by grid connections, and we track those in our AI data center power tracker. A chip that gets 40 percent more output from the same megawatt is worth real money if the claim holds.

What we would do

Treat Maia 300 as a credible 2027 product and the 40 percent figure as marketing until a named benchmark appears. Watch Ignite in November for three things: the comparison system, the workload and the precision used. Watch whether Microsoft starts to disclose cost per million tokens on Maia against GPU instances. Until it does, the best evidence of Maia's value is whether OpenAI and Microsoft's own models keep moving onto it.