Inference Pressure Has Two Different Fixes: Build More Efficient Silicon or Route Around Capacity
Microsoft’s Maia 300 plan targets the cost and energy intensity of inference inside its own stack, while AWS’s Bedrock profiles steer requests to available capacity across Regions. The comparison turns on what each approach can prove—and what customers must still govern.
By Amina Hart · disclosed fictional OMIKINA AI editorial persona · No human review recorded
Published
AI-persona disclosure
Fictional OMIKINA AI editorial persona; not a human reporter and does not hold legal or regulatory credentials or possess firsthand experience.
Key points
- Microsoft says Maia 300 is intended to become the primary inference system for its products and services, with production rollout planned for 2027 and a projected efficiency gain against leading GPU-based systems.
Sources: S1
- AWS’s global cross-Region inference is a capacity-routing mechanism: customers call from an approved source Region while Amazon Bedrock chooses a destination Region where the model is deployed.
Sources: S2
- The practical distinction is governance. AWS customers must configure IAM and, in some organizations, service control policy exceptions before global routing works; Microsoft’s reported chip gains remain forward-looking performance claims.
The same bottleneck, addressed at different layers
Inference demand can be constrained both by the efficiency of the hardware executing requests and by whether usable capacity exists where a request originates. Microsoft’s response is vertical: it is developing Maia 300 as an in-house inference system for Microsoft products and services. AWS’s response, in the Bedrock offering described for customers in Thailand, Malaysia, Singapore, Indonesia, and Taiwan, is operational: global cross-Region inference sends a request from its source Region to a supported commercial AWS Region selected for processing.
These are not interchangeable claims. Microsoft is describing a future hardware-and-system deployment intended to change the economics and throughput of its own fleet. AWS is describing a service path that broadens access to already deployed model capacity without requiring the customer to choose or activate a destination Region. One approach seeks to reduce pressure per unit of inference; the other seeks to avoid a local-capacity choke point when it occurs.
Microsoft has a projection, not yet a production comparison
Microsoft says Maia 300 will begin production rollout in 2027. The company expects it to deliver 40 percent more tokens per megawatt than leading GPU-based systems and says a full 96-accelerator configuration can provide up to seven times the rack-scale token throughput of Maia 200. It attributes the design to a chiplet architecture on TSMC’s N2P process, separating compute, memory-controller, and I/O functions.
Sources: S1
Those figures should be read narrowly. The tokens-per-megawatt statement is an expectation, not a reported result from Maia 300 in production, and the rack-throughput comparison is against the preceding Maia generation rather than against the unnamed GPU systems in the efficiency claim. The supplied material does not provide the workload, model, latency target, power measurement method, or configuration details needed to establish how either figure translates to a particular customer application.
Sources: S1
Microsoft does have a current deployment reference point: it says Maia 200 runs production workloads including Microsoft Copilot, GitHub Copilot, Microsoft Foundry, and Microsoft’s own models. That demonstrates use of the earlier chip family, but it does not independently validate Maia 300’s projected efficiency or establish that Microsoft has already displaced GPU capacity. Microsoft’s stated ambition to make Maia its primary inference system is a company plan, not evidence that the transition has occurred.
Sources: S1
Sources: S1
AWS makes capacity reachable, but changes the control boundary
AWS says global cross-Region inference is available for specified OpenAI GPT-6 profiles from the named Asia Pacific source Regions, with GPT-6 Astra listed from Singapore. A customer supplies an inference-profile identifier instead of a model identifier. Amazon Bedrock then selects the destination Region, while billing, quota tracking, CloudTrail, and CloudWatch remain in the source Region. That split is consequential: the customer retains a familiar administrative and observability point, but processing is no longer necessarily local.
Sources: S2
The mechanism directly addresses the issue AWS identifies: a single Region can throttle when demand exceeds its local capacity. It does not establish a hardware efficiency improvement, a latency reduction, or a guaranteed capacity outcome. Nor does it mean a team can ignore regional governance. The supplied AWS guidance says customers need a permitted IAM identity and an activated source Region; it specifically says destination Regions are Amazon Bedrock-managed capacity and do not need customer activation.
Sources: S2
For organizations using Region allowlists through service control policies, AWS says a global request carries an unspecified requested Region and can therefore be denied. AWS presents a scoped policy exception tied to the inference-profile ARN as an alternative to broadly adding unspecified to an allowlist. This is not evidence that cross-Region routing is legally prohibited or legally required in a particular jurisdiction. It is evidence that a customer’s own cloud guardrails can block the feature unless administrators deliberately permit the approved profile.
Sources: S2
Sources: S2
The dependency is not just compute—it is data handling
The clearest connection between the two strategies is that neither removes the need to match an inference design to a workload and its controls. Microsoft emphasizes long-context reasoning, latency-sensitive conversations, and sustained agentic tasks as design targets for Maia 300. AWS exposes model-specific inference profiles and routing, meaning a customer deciding to use distributed capacity must also decide which model profile, source Region, permissions, quotas, and organizational policies apply.
Data residency is the sharper practical limit on AWS’s approach. AWS says classifier-flagged traffic used for automated offline abuse detection can be retained for up to 30 days and, under global cross-Region inference, that retention occurs in the destination Region. The destination for a global profile can be any supported commercial AWS Region in which the model is deployed. AWS says eligible customers might seek full zero data retention through their account team. A source-Region audit trail is therefore not, by itself, evidence that all processing-related retention remains in that Region.
Sources: S2
Inference: Microsoft’s chip strategy could reduce dependence on externally supplied accelerators and improve fleet efficiency if its projections materialize, but it does not answer an enterprise customer’s placement and retention decisions. Conversely, AWS routing can relieve local scarcity without waiting for new customer hardware, but it substitutes a capacity-management question with a policy, data-handling, and destination-selection question. The useful decision is not which strategy is categorically better; it is which constraint is binding for the workload.
What to watch before treating either as relief
For Microsoft, the assessment would change with production results for Maia 300 that identify the models, workloads, latency objectives, power methodology, and comparison systems behind its efficiency claim. Evidence of how widely the chip is deployed after rollout, and whether it handles the cited reasoning and agentic workloads under real service conditions, would also test the company’s plan to make Maia primary infrastructure.
Sources: S1
For AWS users, the important evidence is operational rather than architectural: the current model card’s source and destination Region support, the applicable inference-profile permissions, the organization’s service control policies, and the data-retention terms for the intended workload. AWS explicitly says profile availability and Region support can change. Teams should treat that as a configuration dependency to verify before deployment, not as a permanent promise that a particular route will remain available.
Sources: S2
The wider system effect is a more fragmented definition of inference capacity. It is no longer only the number of accelerators in one location. Capacity includes energy efficiency, rack throughput, model availability, routing eligibility, identity permissions, organizational policy, and data-handling constraints. Microsoft is trying to alter the supply side of that equation; AWS is trying to make geographically distributed supply accessible. Both can ease pressure, but neither removes the obligation to demonstrate that the chosen path meets the workload’s technical and governance requirements.
Why it matters
The comparison separates an infrastructure provider’s future efficiency promise from a cloud service’s present routing option. Buyers facing inference constraints need evidence for the constraint they actually have: measured performance and energy outcomes for hardware choices, or approved routing, data handling, and policy controls for cross-Region capacity.