A CPU-Centric Perspective on Agentic AI
Until May 2026, CPU manufacturers had been overshadowed in the AI boom. Hyperscalers focused their capital expenditure on GPUs from Nvidia and AMD, while CPUs were treated as mature, low-growth infrastructure. This positioning made sense given how large language models are trained. LLM training is dominated by parallel matrix multiplications, which map directly to GPU architectures with their thousands of small cores executing the same operations across different data. As demand for AI training capacity exploded, GPU market size grew dramatically while CPUs were relegated to a supporting role, coordinating data movement to and from accelerators rather than performing the AI work itself.
Intel illustrated the depth of this shift. After years of ceding share to AMD in both server and consumer markets, Intel struggled through manufacturing setbacks on its leading-edge process nodes. In August 2025, the U.S. government took an equity stake in Intel to support its foundry strategy, with shares trading around $20 at the time. The consensus view was that CPUs had become a mature commodity in an AI landscape dominated by accelerators.
That narrative has reversed sharply. By May 2026, Intel shares had reached approximately $125, the highest level since the dot-com peak in 2000, while AMD had appreciated roughly 350% year-to-date. The market is repricing CPU manufacturers based on a new thesis: the next phase of AI, defined by autonomous agents rather than single-turn chatbots, will be increasingly bottlenecked on CPU rather than GPU resources.
This essay examines why agentic AI is shifting the binding constraint of the AI infrastructure stack, drawing on the recent Georgia Tech and Intel paper "A CPU-Centric Perspective on Agentic AI" to ground the argument in empirical measurement. The paper provides the technical foundation for what is now industry-wide investment thesis. Understanding its findings precisely is essential to evaluating which parts of the market narrative are robust and which depend on extrapolation.
The paper begins by establishing why agentic AI represents a distinct computational paradigm from monolithic large language models. While monolithic LLMs process tasks through single-pass inference, agentic AI frameworks orchestrate multiple components including tool use, memory modules, and iterative reasoning loops. The authors cite empirical evidence that specific agentic frameworks substantially outperform monolithic models of comparable size. The ReAct framework, for example, achieves 27% higher success rates on ALFWorld household tasks and 34% improvement on the WebShop online shopping benchmark.
The paper further argues that small language models are a strong fit for agentic AI, a claim that runs against the common assumption that bigger models always perform better. In agentic settings, tool use externalized much of the work that parametric memory does in monolithic models. SLMs such as GPT-J 6B can therefore match or exceed much larger models such as OPT 66B and GPT-3 175B on knowledge-intensive tasks, because tools and retrieval offload the factual recall that larger parametric memory would otherwise provide.
This shift has hardware implications often overlooked in the AI literature. While AI models themselves run on GPUs, the tools that agentic systems depend on, including Python execution, web search, document retrieval, and code interpreters, cannot run on GPUs. They rely entirely on CPU processing. As agentic systems shift more computation from the LLM to its surrounding tools, the importance of CPUs rises proportionally.
To analyze where bottlenecks emerge in agentic systems, the paper introduces three orthogonal axes for classification. First, the orchestrator axis asks who controls the execution flow: the language model itself, in which case decisions are generated as part of LLM output, or a host program written in Python, in which case decisions are encoded in software logic. Second, the path axis asks whether the workflow follows a predetermined sequence (static) or is constructed adaptively at runtime based on intermediate results (dynamic). Third, the repetitiveness axis asks whether the agent completes its task in a single inference pass or loops through multiple iterations of action and observation. Each axis maps to different hardware behavior, which the paper then measures empirically.
The classification framework matters because it predicts where it predicts where bottlenecks will merge. Systems combining LLM orchestration with dynamic paths and multi-step execution, such as autonomous coding and research agents, are far more demanding than the simpler production deployments common today. As the agentic AI frontier moves toward this more demanding corner, hardware pressure intensifies systemically rather than randomly. To measure this empirically, the authors profile five representative workloads on top-tier hardware combining a 48-core Intel Emerald Rapids CPU with an Nvidia B200 GPU.
The central finding is that tool processing on CPUs dominates the user's wait time. For Haystack performing retrieval-augmented question answering, CPU-bound search consumes 84.5 to 90.6 percent of end-to-end latency, while LLM inference on the GPU takes under 0.5 seconds. For SWE-Agent performing coding tasks, Bash and Python execution accounts for 43.8 to 78.7 percent of latency. Even in the most LLM-heavy workloads measured, tool work meets or exceeds LLM inference. The pattern holds across different workload types, model sizes, and orchestration patterns, which makes it a structural property of agentic computation rather than an artifact of any single choice. This is the empirical finding that anchors the entire wall street thesis about CPU demand growth.
The throughput and energy measurements reinforce the same conclusion through different lenses. Different agentic workloads saturate on different bottlenecks. Haystacks saturates on CPU cache pressure. LangChain and SWE-Agent saturate on CPU core over-subscription. Toolformer saturates on GPU memory. The implication is that data center capacity planning becomes more complex than the GPU-cenric approach that dominated chatbot-era infrastructure. On energy, CPU consumption grows much faster than GPU consumption as workload scale, expanding the CPU's share of total dynamic energy from 20 percent a t small batches to 44 percent at large ones. Since modern AI data centers are constrained by gird power rather than capital, every watt the CPU burns is a watt unavailable for additional GPUs in the same power envelope. CPU efficiency has become a power-budget concern, not just a hardware-cost concern.
The paper does not stop at diagnosis. The authors propose two software optimizations, CGAM and MAWS, that achieve roughly 2 times improvement in median latency through smarter scheduling alone. This result has significant implications for the investment thesis. The CPU bottleneck is partially software-flexible. Better orchestration recovers substantial performance without requiring additional hardware. Production frameworks do not yet implement these optimization, but as they do, the per-task CPU requirement will decline.
This software-versus-hardware question is the central uncertainty in the wall street thesis. Morgan Stanley estimates 32.5 to 60 billion dollars of incremental CPU TAM by 2030. TrendForce projects CPU to GPU ratios shifting from 1:8 toward 1:1. AMD has revised its server CPU forecast from 18 percent CAGR to over 35 percent CAGR. these forecasts implicitly assume that software optimization does not outpace agent adoption growth. If CGAM-style techniques become standard in production frameworks within two years, the implied hardware demand shrinks materially.
The relationship between this paper and the market narrative deserves careful framing. The paper supports the directional thesis robustly. CPU and memory have become mjaor bottlenecks in agentic workloads running in enterprise data centers today. The specific magnitude claims, however, are analysts extrapolations built on the paper's measurements rather than paper findings. The 90.6 percent figure is from one workload on one benchmark, not from agentic AI in general. The TAM forecasts depend on assumptions about deployment curves and software optimization rates that no one can measure precisely.
The conclusion is that the agentic AI infrastructure shift is real and structurally grounded, but its magnitude remains genuinely uncertain. The directional thesis is well supported. CPU and memory will play a larger role than the chatbot era anticipated. The specific timing and scale of resulting demand growth depend on the rate of agentic AI deployment in enterprises and the rate of software optimization in production frameworks.