Enterprise IT leaders face a maze of choices as they modernize infrastructure for AI. In this conversation, Madhu Rangarajan, Corporate VP of Server Products at AMD, lays out a pragmatic roadmap: first reclaim space, power, and cooling by upgrading to modern CPUs; then align compute choices (CPU vs. GPU) to workload scale; and finally embrace open standards across networking and software to avoid lock-in.
Core Takeaways
How Enterprises Should Modernize for AI
- Start by creating headroom: Refreshing older servers with modern EPYC-class CPUs can cut space and power dramatically, freeing capacity for AI infrastructure (CPU and/or GPU) while improving efficiency.
- Match compute to the workload: Classical ML and small-scale LLM/chat workloads often run well on CPUs; large-scale generative AI typically pairs high-performance CPUs with GPUs for maximum throughput.
- Open standards reduce lock-in: AMD champions open ecosystems—ROCm, Ultra Ethernet Consortium, and UALink—to enable interoperable AI supercomputers and integrator flexibility as AI adoption broadens.
Key Quotes
— The First Step: Create Space, Power, and Cooling Headroom
“Before you debate CPU versus GPU, get your data center AI-ready. Many enterprises are running five-year-old servers; moving to the latest generation can deliver the same general-purpose work in far less space and power. That reclaimed capacity becomes your runway for deploying the right mix of CPU and GPU infrastructure.”
“Efficiency isn’t a nice-to-have—it’s the enabler. When you free up 70% in space and power, you gain options: scale out CPUs for traditional analytics and smaller models, or slot in GPUs for large-scale generative AI. Modernization is the foundation that makes all those choices possible.”
— When to Use CPUs vs. GPUs
“Classical machine learning and smaller language model workloads often run great on CPUs—especially with optimized libraries and framework integrations. If you’re serving an internal chatbot intermittently, CPU can be the simplest, most cost-effective starting point.”
“At large scale for LLMs and generative AI, you want CPU + GPU together. And even then, the CPU matters a lot: it handles orchestration, preprocessing, and kernel launch phases. Right-sizing that CPU layer reduces bottlenecks and increases GPU utilization.”
— Why High-Performance CPUs Still Matter in GPU Boxes
“We built our latest high-frequency, many-core CPUs to ‘get out of the way’ faster—spiking to very high clocks during CPU-bound phases so GPUs stay fed. In real deployments, that can translate into double-digit percentage gains on an eight-GPU server.”
“Think of it as performance per dollar: if a GPU server costs into the six figures, a 10–20% uplift from the CPU layer is real money. Tuning the CPU side isn’t optional in AI at scale; it’s central to achieving the ROI you promised the business.”
— Open Ecosystems and the Road Ahead
“AI is bigger than any single company. That’s why we’re investing in open software and interconnect standards—so integrators and customers can assemble the right solution without being boxed in. Openness gives you choice whether you want a fully integrated stack or a mix-and-match architecture.”
“New tech often starts proprietary and opens over time. We’re already seeing AI supercomputers move from vertical stacks to standards-based components. As agentic AI takes off, demand will be ‘insatiable’ across CPU, GPU, and networking—and open ecosystems will help the industry keep pace.”
