AI Compute Surges as Token Tiering Drives Cost Control

Last Modified

2026-11-12

Update Frequency

Aperiodically

Format

PDF

AI chip volumes keep rising in 2027, more than doubling compute; FP4, MoE and model tiering cut costs, but post-2028 oversupply looms.

Key Highlights

Chip Shipments: Strong GPU and ASIC demand in 2027 lifts volumes, with compute supply set to more than double.
Compute Mix: CSP in-house ASICs gain share from incumbents; FP8 emerges as a training option.
Cost Optimization: FP4 and MoE lower per-token inference costs, while flagship and efficient models differ widely in output and price.
Outlook Risks: Agentic AI strains memory; after 2028, demand lagging supply may slow growth.

Demand for GPUs and ASICs from End Clients (e.g. CSPs) to Remain Robust in 2027 and Result in Growth in Global AI Chips
End Clients (e.g. CSPs) to Adopt Various Different Tiers of AI LLMs to Improve Usage Efficiency of Tokens under Persistently Shortages of Upstream Materials
AI Model Training Still Focuses on BF16/FP16; FP8 Gradually Emerges as an Option for Training High-Efficiency Models
- BF16/FP16 Still Mainly Used For Training In 2026, But FP8 Becoming New Option
Demand for AI Inference Drives Expeditious Growth of FP4 Compute; MoE Architecture Serves as Key in Lowering Compute Cost
- AI Model Developers Increasingly Adopt FP4 and MoE to Reduce Computational Costs
Output Token Volume of Efficient LLMs Far Exceeds That of Flagship Models; Task-Based Routing Across AI Model Tiers to Enhance Economic Efficiency Is Expected to Become the Next Paradigm
- Classifying Requests to Different Levels of AI Models Grows Critical for Enterprises in Saving Costs
Enterprise AI Application Cost Containment Emerges as Key Focus; Post-2028 Compute Supply-Demand Imbalances and Decelerating Growth Risks Warrant Further Attention