Not Just GPUs: Agentic AI Brings CPUs Back

CPU's Role in Agentic AI

Historically, CPUs have served as the "brains" of data center servers—handling complex logic, sequential processing, and classical machine learning workloads such as linear regression and decision trees.

The AI boom changed that dynamic. AI models now require massive parallel matrix multiplication, which GPUs—with their embarrassingly parallel architectures—are built to handle at scale. CPUs were relegated to compressing and routing memory data to GPUs. As a result, today's AI data centers operate at CPU-to-GPU ratios of roughly 1:4 to 1:8.

Figure 1. CPU vs. GPU key comparison table covering computing principle, core count, memory type, AI application, and vendors.

2026 Agentic AI Wave: CPU Shortage and GPU Ratio Structural Changes

NVIDIA's Vera CPU Rack and Arm's AGI CPU signal a structural shift. Analyzes the CPU shortage and GPU-to-CPU ratio changes driven by the agentic AI wave.

Explore the Report

The rise of AI agents is changing that balance. Unlike static LLMs, Agentic AI is designed to interact dynamically with its environment—planning tasks, calling tools, making decisions, and taking actions on behalf of users. The coordination layer that manages all of this—scheduling sub-tasks, routing tool calls, passing data between sub-agents, and evaluating whether the original request has been fulfilled—falls squarely on the CPU. This is what makes Orchestration a CPU-intensive workload.

Figure 2. Diagram showing CPUs orchestrating multiple AI agents in a cloud environment, with agents interacting with an AI data center where CPUs and accelerators divide tasks.

Source: Arm

Agentic RL adds further demand. When AI agents are trained through Reinforcement Learning (RL), each action the agent takes must be evaluated—a process that places additional load on the CPU.

A November 2025 paper, A CPU-Centric Perspective on Agentic AI, noted that CPUs remain crucial for tool processing tasks such as Python interpretation, web crawling, lexical summarization, and database searches—scenarios that remain prevalent in Agentic AI. AI Agents may face three primary bottlenecks regarding CPUs:

1. Latency: Tool processing on CPUs can account for up to 90.6% of total latency.
2. Throughput: Bottlenecks can stem from CPU factors (number of cores, coherence and synchronization) or GPU factors (main memory capacity and bandwidth).
3. Energy: CPU dynamic energy consumption can reach 44% of the total dynamic energy at large batch sizes.

To address these, the traditional CPU:GPU ratio must change. Arm estimates that while traditional AI data centers require about 30 million CPU cores per GW, demand will surge to 120 million CPU cores per GW in the AI Agent era—a fourfold increase. The future CPU-to-GPU ratio is expected to shift to between 1:1 and 1:2, significantly boosting market demand for CPUs.

Figure 3. CPU to GPU ratio shift from Traditional LLM to Agentic AI applications.

For a closer look at how the inference economy is reshaping AI chip architectures, see Inference Economy Arrives: AI Chip Rules Are Being Rewritten.

CPU demand has surged across both AI workloads and general-purpose servers within data centers. Intel and AMD responded by raising prices across select CPU product lines toward the end of 1Q26.

Trends to Watch 2026: Agentic AI and Real-time Inference

Identifies pivotal 2026 trends from agentic AI deployment to real-time inference architecture and their implications for hardware and supply chains.

Master the Market Trends

2026 CPU Market Landscape

This demand shift is reshaping the competitive landscape. Beyond traditional server CPU vendors Intel and AMD, a new wave of non-traditional players—GPU maker Nvidia, IP licensor Arm, and major cloud service providers (CSPs) including AWS, Google, and Microsoft—are now entering the server CPU market.

Traditional x86 Vendors: Intel and AMD

Intel's Xeon processors long commanded over 95% of the data center CPU market. That dominance began to erode in 2021, when yield issues with the Intel 7 process delayed the Xeon Sapphire Rapids launch by nearly two years, creating an opening for AMD's EPYC Milan—built on TSMC N7 with the Zen3 architecture—to make meaningful inroads.

Intel's Xeon 6+ (Clearwater Forest) launched in 2026 on the Intel 18A process, while its next-generation Xeon, Diamond Rapids, will use the enhanced Intel 18A-P process.

• Xeon 6+ (Clearwater Forest): Darkmont architecture, 288/288 cores/threads, TDP ~450W, featuring Foveros Direct Hybrid Bonding for the first time.
• Xeon 7 (Diamond Rapids): Panther Cove-X architecture, up to 256/256 cores/threads, TDP up to 650W.

Meanwhile, AMD continues its partnership with TSMC. Its EPYC Venice, launched in 2026, adopts the Zen 6 architecture, TSMC N2 process, and advanced packaging (CoWoS-L, SoIC), featuring 256/512 cores/threads. AMD is expected to continue gaining market share from Intel in 2026.

Arm-Based Entrants: Nvidia, Arm, Ampere, and the CSPs

In March 2026, Nvidia announced it would begin selling the Vera CPU as a standalone product, responding to customer demand for more flexible CPU:GPU configurations. Early partners include Alibaba, ByteDance, Cloudflare, CoreWeave, Crusoe, Lambda, Nebius, Nscale, Oracle, Together.AI, and Vultr.

Vera uses Nvidia's custom Olympus architecture, along with TSMC N3 process and CoWoS-R packaging technology, and features 88 cores/176 threads with Spatial Multithreading. While its core count is lower than Intel or AMD flagships, it offers a 1.8 TB/s NVLink-C2C interconnect for memory sharing with Nvidia GPUs.

Figure 4. Architecture diagram of the NVIDIA Vera CPU featuring 88 Olympus cores with 176 threads, NVLink-C2C at 1,800GB/s, and 1.5TB LPDDR5X memory with 1.2TB/s bandwidth.

Source: Nvidia

2026 NVIDIA AI Outlook: From GPU to LPU Racks & Inference

NVIDIA expands its AI factory via integrated GPU, CPU, and LPU racks for training and inference, securing its dominant market position.

Gain the Full Intelligence

Nvidia also introduced the Vera CPU Rack, built on its MGX architecture. A single rack features 32 × 1U compute trays, each housing 4 Vera CPU Nodes, totaling 256 CPUs (22,528/45,056 cores/threads and 400 TB of memory). These racks are interconnected via BlueField-4 DPUs—which integrate Grace CPUs and ConnectX-9—ensuring high integration within the Nvidia ecosystem. For the broader scale-out interconnect landscape, see InfiniBand vs Ethernet: Broadcom and NVIDIA Scale-Out Tech War.

Figure 5. Nvidia server rack product lineup showing HGX Rubin NVL8, Vera CPU Rack liquid cooled, Vera Air-Cooled CPU Rack, and Vera Rubin NVL72, unified under one architecture and toolchain.

Source: Nvidia

Also in March 2026, Arm took an unprecedented step into the CPU product market with the Arm AGI CPU, ending 35 years of pure licensing. Built on TSMC N3 with the Arm Neoverse V3 architecture, it offers 136/136 cores/threads. Launch partners include Meta, Cerebras, Cloudflare, F5, OpenAI, Positron, Rebellions, SAP, and SK Telecom.

Figure 6. Arm AGI CPU architecture highlighting up to 136 Neoverse V3 cores, 300W TDP, 3nm efficiency, DDR5-8800 memory support, PCIe Gen6, and sub-100ns memory latency.

Source: Arm

Arm has introduced standalone CPU racks in two configurations: an air-cooled version integrating 60 AGI CPUs (8,160/8,160 cores/threads, ~180 TB memory), and a liquid-cooled version supporting 336 CPUs (45,696/45,696 cores/threads, 1 PB memory).

Figure 7. Arm AGI CPU rack configurations: air-cooled 36kW rack with 8,160 CPU cores and 180TB memory; liquid-cooled 200kW rack with 45,696 CPU cores and 1PB memory.

Source: Arm

Arm AGI CPU: Reshaping AI Data Center Architecture

Arm's AGI CPU marks a strategic shift from IP licensor to platform provider, targeting AI data centers and accelerating gains against x86.

See the Shift

Following the release of the 192-core AmpereOne and AmpereOne M, SoftBank-backed Ampere is expected to launch the AmpereOne MX in 2026, which will feature an increased core and thread count of 256/256.

Among the major U.S. CSPs, AWS was the earliest mover, launching its first in-house Graviton CPU in 2016 using the TSMC N16 process. In December 2025, it released Graviton5 on TSMC N3 (192/192 cores/threads). By pairing these custom CPUs with its Trainium 3 AI ASIC, AWS is able to effectively lower the costs of AI computation.

Microsoft became the second CSP to develop in-house CPU, debuting the Cobalt 100 CPU alongside the Maia 100 AI ASIC in November 2023 using TSMC's N5 process. In November 2025, the company introduced the next-generation Cobalt 200, which utilizes the N3 process and provides 132/132 cores/threads.

Google followed suit in April 2024 with the launch of its first custom CPU, the Axion C4A, based on the TSMC N5 process. For 2026, Google plans to introduce the bare-metal Axion C4A.metal (96/96 cores/threads) and the next-generation Axion N4A (64/64 cores/threads), with the strategic goal of delivering the highest price-to-performance ratio in the market.

Below we summarize the core and thread counts of major CPUs in 2026. It is observable that the Intel Xeon 6+ (Clearwater Forest) will achieve the maximum core count of 288. This is followed by the AMD EPYC Venice, Intel Xeon 7 (Diamond Rapids), and AmpereOne MX, all of which feature 256 cores. However, due to the utilization of Simultaneous Multithreading (SMT) technology, the AMD EPYC Venice can execute 512 threads across its 256 cores, reaching the highest thread count currently available.

Figure 8. Core and thread counts of major CPUs in 2026, including AMD EPYC Venice, Intel Xeon, Nvidia Vera, and Arm AGI CPU.

TrendForce expects the growing trend of major CSPs developing custom CPUs to create additional business opportunities for IC back-end design service providers. Currently, while AWS maintains its design in-house, both Google and Microsoft outsource their CPU back-end design services to Global Unichip Corp. (GUC). The following table summarizes the projected projects to be secured by ASIC design service providers between 2026 and 2028.

For more on how NVIDIA's scale-up ecosystem is evolving and where ASICs fit in, see ASIC Set to Outpace GPU? NVIDIA's Scale-Up and Beyond.

The Rise of AI Agent Applications Will Drive Demand for Core Semiconductor Components from Edge to Cloud

Analyzes how AI agent applications are reshaping core computing hardware configurations and which semiconductor supply chains stand to benefit directly.

Master the Market Trends