Separating AI agent execution from GPU inference could improve accelerator utilization, but the resulting network demands may complicate deployment and erode potential cost savings, analysts say.
AMD has introduced Helion, an open blueprint for data center racks that aims to give CPUs a larger role in running AI agents while allowing customers to configure GPU resources around their inference requirements.
Based on Open Compute Project (OCP) specifications, the architecture supports different combinations of computing hardware. AMD cited examples including CPU-only racks for hyperscalers, powered by sixth-generation EPYC processors with up to 32,768 cores, and all-in-one enterprise racks accommodating up to 16 Instinct MI350P GPUs.
AMD has listed networking bandwidth of up to 102 Tbps for Helion, although the company has not specified how that figure is calculated. The specification represents a planned rack capability rather than demonstrated performance in customer environments.
An integrated token router is intended to direct AI agents’ requests to different models, allowing enterprises to use cloud-hosted frontier models alongside open-source models running within the rack.
AMD said the design supports both air and liquid cooling. The company plans to contribute the Helion specification to OCP for further development by the industry.
The chipmaker has published statements of support from infrastructure vendors, including Cisco, Dell Technologies, HPE, and Lenovo. Rack configurations developed with ecosystem partners are expected to become available in November 2026.
Greater emphasis on CPUs
The growing use of AI agents could increase demand for data center CPUs, according to Lian Jye Su, chief analyst at Omdia. While GPUs handle model inference, CPUs support tasks such as code execution and workflow orchestration.
“The AI industry is already expecting a significant adjustment of the XPU to CPU ratio, going from 16:1 to 4:1 and eventually 1:1,” Su said, using XPU as an umbrella term for accelerators including GPUs and custom AI silicon.
AMD’s approach reflects an existing division of work in enterprise AI rather than a fundamental change in how AI applications are designed, according to Kanishka Chauhan, senior principal analyst at Gartner. CPUs already handle much of the orchestration surrounding GPU-based model inference, he said.
Charlie Dai, principal analyst at Forrester, said separating agent execution from inference would allow data center operators to scale CPU and GPU resources independently, potentially improving accelerator utilization.
“The tradeoff is greater east-west network traffic, making low-latency networking, data locality, and composable rack architectures increasingly important,” Dai said.
Chauhan said repeated exchanges between CPU and GPU systems could add network delays at successive stages of lengthy agent workflows, making inter-rack latency an important consideration.
Integration challenges
Helion could reduce the complexity of designing AI infrastructure, but enterprises would still need to calculate the total cost of ownership for individual agent workloads. Dai said potential savings from better accelerator utilization would need to be weighed against power, cooling, and integration costs.
Chauhan said Helion’s all-in-one configuration could reduce the need for custom integration between CPU servers and separate GPU clusters, potentially lowering deployment costs. Hyperscalers could also deploy CPU-only racks alongside their existing GPU infrastructure, he said.
If separating agent execution from GPU inference requires network upgrades to handle increased traffic between racks, the additional costs could offset some of the savings from more efficient GPU use.
Su identified software maturity, compatibility with AMD’s ROCm ecosystem, and achieving operational simplicity comparable to more tightly integrated platforms as additional deployment challenges.
Alternative to Nvidia
Helion’s modular design could appeal to enterprises looking to integrate AI infrastructure into existing environments.
“An open architecture can meaningfully challenge Nvidia in segments where customers prioritize multi-vendor choice, reduced lock-in, cost flexibility, or integration with existing x86/OCP environments,” Su said.
Nvidia is also expanding its use of open infrastructure standards. This week, the company said it had contributed Vera Rubin rack and compute tray designs to OCP and outlined a Vera CPU rack architecture intended to support agentic AI workloads.
Dai said support from infrastructure vendors such as Cisco, Dell, HPE, and Lenovo would be important in providing enterprises with validated designs and operational support. For wider adoption, AMD would need to demonstrate competitive performance and software maturity while proving that Helion can deliver lower total cost of ownership, he said.
