AMD Helios Rack Scale AI Infrastructure Explained for Enterprises

AMD officially launched its Helios rack-scale AI infrastructure platform in July 2026, and the specification sheet alone explains why enterprise infrastructure teams are paying attention, especially organizations investing in AI Development Services.  A single Helios rack packs 72 AMD Instinct MI455X GPUs, delivers up to 2.9 exaFLOPS of FP4 compute, and carries 31 terabytes of HBM4 memory across an open, Ethernet-based fabric rather than a proprietary interconnect. That last detail matters more than the raw numbers. For nearly a decade, the highest-performance AI training and inference racks have been built around vendor-specific scale-up networks that lock a buyer into a single accelerator family for the lifetime of a cluster. Helios is the first credible, shipping, rack-scale system that reaches Nvidia-class exaflop performance while running its scale-up traffic over UALink and Ultra Ethernet Consortium standards that any hardware vendor can implement.

This is not a paper launch. Oracle has already committed to deploying 50,000 AMD GPUs on the platform, Anthropic has signed a deal to deploy up to 2 gigawatts of MI450 series GPUs in Helios racks starting in the first half of 2027, and HPE and Supermicro have both announced turnkey Helios systems built on the Open Compute Project Open Rack Wide standard. AMD is also reporting that MI455X delivers up to 34 times higher token throughput and up to 18 times lower token cost than its predecessor MI355X on DeepSeek V4 Flash, a concrete inference economics claim that enterprise buyers can actually model against their own workloads. What existed before Helios was a market where every rack-scale AI system worth deploying at frontier scale ran on Nvidia's NVLink domain, and the only real question for a buyer was how much lock-in they were willing to accept in exchange for performance and software maturity.

What changes with Helios is that buyers now have a second, standards-based path to the same performance tier, backed by real silicon, real hyperscale customers, and a software stack that is being actively hardened rather than promised. This blog breaks down how Helios actually works at the architecture level, how its performance and cost claims hold up against the previous generation of rack-scale systems, what adopting it actually requires from an infrastructure team, and where AMD Helios rack-scale AI infrastructure fits into a realistic 2026 and 2027 deployment roadmap.

What Problem Helios Actually Solves

The core problem Helios addresses is not raw compute scarcity; it is architectural lock-in at the rack level. Frontier model training and high-throughput inference both depend on a scale-up domain, meaning a group of accelerators that can share memory and communicate with extremely low latency as if they were a single giant GPU. Nvidia's answer to this has been NVLink, a proprietary interconnect that ties GPUs and Grace CPUs together inside systems like the GB200 and Vera Rubin NVL72 racks. NVLink is fast and mature, but it is also closed, which means every component in that scale-up domain has to come from Nvidia or a small set of certified partners.

That closed model worked when AI infrastructure was a smaller, more experimental market. It is far less workable now that AI compute has become a multi-hundred-billion-dollar annual capital expenditure category for hyperscalers and sovereign AI programs. Buyers purchasing gigawatts of capacity do not want their entire scale-up fabric, their networking roadmap, and their long-term pricing leverage dictated by a single supplier's release cadence.

Before Helios, AMD's own MI300 and MI355X generations competed reasonably well on raw FLOPS and memory capacity, but they shipped as individual accelerators or smaller pods rather than as a fully integrated, validated rack-scale answer to NVL72-class systems. That gap meant AMD could win individual GPU deals but struggled to win entire data center buildouts because system integrators and hyperscalers had to do the rack-level engineering themselves. Helios closes that gap by shipping the GPU, the CPU, the networking, the cooling, and the software stack as one validated system, which is the same packaging strategy that made Nvidia's DGX and NVL72 systems so easy to procure at scale.

Why Proprietary Scale-Up Fabrics Became a Strategic Risk

Enterprises and cloud providers building multi-year AI infrastructure roadmaps have started treating single-vendor scale-up dependency as a genuine supply chain risk, not just a pricing inconvenience. Export control tightening, HBM memory allocation constraints, and GPU lead times measured in quarters have all made procurement teams nervous about depending entirely on one accelerator family for the capacity they need to guarantee years in advance. A second, open, credible rack-scale platform gives procurement and infrastructure leaders actual negotiating leverage and a realistic fallback plan if allocation on one vendor's roadmap slips.

This is precisely the dynamic that explains why organizations like Oracle and Anthropic are willing to commit tens of thousands of GPUs and multiple gigawatts of capacity to a platform that only reached general availability in mid-2026. They are not simply chasing a better price per token, although the economics matter. They are diversifying the physical layer their AI stack depends on, in the same way large enterprises diversify cloud providers or semiconductor foundries.

Inside the Helios Architecture

At the compute layer, each Helios rack is built from 18 compute trays inside a double-width, 4OU Open Rack Wide enclosure that follows the OCP specification Meta contributed to in 2025. Every compute tray houses four MI455X GPUs paired with a single-socket sixth-generation EPYC Venice CPU built on the Zen 6 architecture, along with up to three Pensando Vulcano 800 AI network interface cards per GPU for scale-out traffic. Six additional switch trays handle the fabric that turns those 18 compute trays into one coherent 72-GPU scale-up domain.

The MI455X itself is a chiplet-based accelerator built on a 2-nanometer process, carrying 432 gigabytes of HBM4 memory per GPU with up to 19.6 terabytes per second of memory bandwidth and roughly 40 petaFLOPS of peak FP4 compute. Aggregated across the full rack, that works out to 31 terabytes of HBM4 memory, up to 2.9 exaFLOPS of FP4 performance, 1.4 exaFLOPS of FP8 performance, and reported figures ranging from 1.4 to 1.7 petabytes per second of aggregate memory bandwidth, depending on the specific configuration and source.

The networking layer is where Helios genuinely departs from the Nvidia playbook. Instead of a proprietary scale-up interconnect, AMD uses UALink over Ethernet, referred to as UALoE, which layers the Ultra Accelerator Link protocol on top of standard Ethernet physical infrastructure. Each MI455X provides 3.6 terabytes per second of bidirectional scale-up bandwidth, and the full rack delivers all-to-all GPU connectivity through a single switch layer using Broadcom Tomahawk-based switching silicon. Scale-out traffic between racks runs over AMD Pensando networking aligned with Ultra Ethernet Consortium standards, with the rack rated for up to 43 terabytes per second of aggregate scale-out bandwidth.

Several architectural components define what makes a Helios rack function as a single system rather than a collection of connected servers.

• The compute tray design integrates GPU, CPU, and networking into a single field-replaceable unit, with quick-disconnect liquid cooling connections that eliminate recabling during maintenance.

• The UALink over Ethernet scale-up fabric exposes all 72 GPUs as one coherent memory and communication domain, matching the functional role NVLink plays in NVIDIA systems.

• A 50-volt DC liquid-cooled busbar and rear blind mate quick disconnect cooling system manages thermal load across a rack that weighs approximately 5,000 pounds fully populated.

• Hardware root of trust and continuous attestation run at every layer, supporting encrypted memory and interconnects for multi-tenant workload isolation.

• The sovereign AI variant pairs Venice X CPUs with MI430X accelerators specifically for organizations required to keep AI workloads within national borders.

Why the Open Standard Choice Is a Real Engineering Bet, Not Just Marketing

Choosing Ethernet as the scale-up transport instead of a dedicated low-latency fabric is a genuine engineering tradeoff, not a purely symbolic one. Dedicated interconnects like NVLink are optimized from the ground up for the tiny message sizes and extreme latency sensitivity of collective operations likeall-reducee and all-gather, which dominate distributed training and large-batch inference. Ethernet historically carried higher tail latency and more jitter under these workloads, which is exactly why UALink and the Ultra Ethernet Consortium exist as separate standards bodies working to close that gap.

AMD's bet is that Ultra Ethernet's congestion control, packet spraying, and reliable transport improvements, combined with UALink's protocol-level optimizations for accelerator-to-accelerator traffic, can close the performance gap to proprietary fabrics closely enough that the openness and multi-vendor flexibility outweigh the residual latency cost. Early published benchmarks suggest this bet is at least partially validated in production representative workloads, though independent, workload-diverse verification outside AMD's own reference numbers is still limited given how recently the platform shipped.

Performance and Economics Against the Previous Generation.

AMD's own published benchmarks provide the clearest starting point for evaluating whether Helios changes the economics of AI compute in a way that matters for buyers, even though these figures come from the vendor and deserve the same scrutiny any vendor benchmark deserves. On DeepSeek V4 Flash, a widely used open-weight model, AMD reports that the MI455X delivers up to 34 times higher token throughput at high interactivity settings and up to 18 times lower token cost compared with the previous-generation MI355X GPU. That is a generational jump driven by the combination of higher memory bandwidth, larger HBM4 capacity that reduces the need for aggressive quantization or model sharding, and inference software optimizations layered into the current ROCm release.

Comparative figures against Nvidia's rival Vera Rubin NVL72 rack are more contested but still informative. On Kimi K2, with a 32-thousand-token input and 8-thousand-token output sequence, AMD's modeled Helios throughput per GPU comes in 15 percent higher at low interactivity settings, 12 percent higher at medium interactivity, and 10 percent higher at high interactivity than modeled Vera Rubin NVL72 performance. Against Nvidia's published NVL72 specifications more broadly, AMD claims 15 percent more peak FP4 performance, 50 percent more HBM capacity, 6 percent more HBM bandwidth, and 50 percent more scale-out bandwidth per rack.

These numbers should be read with an important caveat that any principal engineer evaluating the platform will apply immediately. Vendor-reported comparative benchmarks are optimized for scenarios where the reporting vendor performs well, and real workload performance depends heavily on software maturity, kernel optimization coverage in ROCm versus CUDA, and how well a specific inference or training pipeline maps onto AMD's memory hierarchy and quantization support. The direction of the numbers is credible given the underlying hardware specifications, but the magnitude should be validated against representative production workloads before being used in a capacity planning model.

What the Cost Per Token Numbers Actually Mean for Buyers

Cost per token is the metric that ultimately drives inference infrastructure procurement decisions at scale, more than raw FLOPS or memory bandwidth in isolation. An 18-times reduction in token cost between MI355X and MI455X, if it holds across a broader set of production workloads beyond the DeepSeek V4 Flash benchmark AMD cited, would represent one of the larger single-generation cost improvements the industry has seen in inference serving economics. That kind of improvement changes which use cases are economically viable, particularly for high-volume, latency-sensitive agentic workloads that make repeated small inference calls rather than a small number of large batch requests.

Enterprises evaluating this should model their own workload characteristics against the benchmark conditions AMD used, since interactivity level, sequence length, and batch size all materially change the throughput and cost outcome. A workload dominated by long-context retrieval-augmented generation will behave very differently on this hardware than a short, high-frequency agent tool-calling workload, even on the same rack.

Practical Adoption Guide for Infrastructure Teams

Adopting Helios is not simply a procurement decision; it is a platform migration decision that touches the software stack, the operations team, and the capacity planning model an organization already runs. Teams currently standardized on CUDA and NVIDIA's software ecosystem need to evaluate how much of their inference and training pipeline depends on CUDA-specific kernels, libraries, or tooling that does not have a mature ROCm equivalent yet. AMD's ROCm software stack now offers native support for PyTorch, TensorFlow, and JAX, and the company introduced ROCm.AI at the Helios launch specifically to simplify installation, diagnosis, model serving, and performance tuning, but ecosystem maturity still trails CUDA in coverage depth for less common operators and libraries.

A realistic adoption path generally follows a sequence that limits risk while building internal ROCm expertise.

1. Start with inference workloads on well-supported open-weight models, since AMD's own benchmark data and community reporting are strongest for inference rather than training at this stage of the platform's maturity.

2. Validate ROCm kernel coverage for the specific model architectures, quantization formats, and serving frameworks the team already runs in production, rather than assuming parity with CUDA-based deployments.

3. Pilot on a partial rack or smaller MI400 series deployment before committing to full Helios rack procurement, since OEM partners, including HPE and Supermicro,o are shipping validated systems at multiple scale points.

4. Build monitoring and observability around ROCm-specific tooling early, since operational runbooks writtenfor NVIDIA'sa system management tools generally need to be rewritten rather than lightly adapted.

5. Negotiate procurement terms that account for the open standard positioning, since UALink and Ultra Ethernet compatibility theoretically opens the door to multi-vendor networking components over time, which is a genuine point of leverage in contract negotiation.

6. Plan for a hybrid fleet in most cases rather than a full cutover, since many organizations will run Helios alongside existing Nvidia capacity for at least the next several procurement cycles.

Security and multi-tenancy features are also worth evaluating directly rather than assuming parity with established platforms. Helios ships with hardware root of trust, continuous attestation, and encrypted memory and interconnects designed for multi-tenant environments, which matters significantly for any organization running Helios capacity inside a shared cloud environment rather than a fully dedicated, single-tenant deployment.

Reliability, Serviceability, and Operational Characteristics

Rack-scale serviceability was a deliberate design focus for Helios, and it shows in details that matter more to operations teams than to anyone reading a spec sheet in isolation. The rack's power, cooling, and networking connections are engineered to eliminate recabling during maintenance, using rear blind mate quick disconnect liquid cooling and integrated busbar power distribution so that a failed compute tray can be swapped without disrupting the rest of the rack's operation. At roughly 5,000 pounds for a fully populated rack, physical logistics for installation and maintenance also require data center facilities planning that many organizations have not previously needed for GPU infrastructure at this density.

Liquid cooling is not optional at this power and density level, which means any organization evaluating Helios needs data center facilities capable of supporting direct liquid cooling infrastructure, not just high-density power delivery. This is broadly consistent with what Nvidia's own NVL72 class systems require, so it is not a unique burden Helios introduces, but it does rule out deployment in older air-cooled data center facilities that have not already been retrofitted for liquid-cooled AI racks.

Current Limitations and Open Questions

Software ecosystem maturity remains the most significant honest limitation of the Helios platform at launch. ROCm has closed substantial ground against CUDA over the past several years, but coverage gaps still exist for less common model architectures, custom kernels, and some of the more specialized libraries that CUDA native teams have built up over a decade of ecosystem investment. Organizations with deep custom CUDA code in their training or inference pipelines should expect nontrivial porting work, not a drop-in replacement.

Independent, workload-diverse benchmarking is also still limited relative to the volume of vendor-published figures available at launch. AMD's DeepSeek V4 Flash and Kimi K2 Thinking comparisons are useful directional signals, but they represent specific model architectures and interactivity settings rather than a comprehensive picture of how Helios performs across the full range of production workloads enterprises actually run, including fine-tuning, long context retrieval pipelines, and multimodal serving.

Supply chain execution risk is a further open question worth naming honestly. AMD has publicly denied reports of MI455X delays and states Helios systems remain on target for second-half 2026 availability, but HBM4 memory supply constraints have already been flagged industry-wide as a bottleneck affecting multiple accelerator vendors simultaneously, not just AMD. Buyers should treat announced timelines as targets rather than guarantees given how frequently the broader AI hardware industry has experienced schedule slippage over the past several years.

Adoption Trajectory and Strategic Implications

The near-term trajectory for Helios looks strongest in three categories of buyers. Hyperscalers and neoclouds seeking a credible second source for exaflop-scale compute, exemplified by Oracle's 50,000 GPU commitment and TensorWave's stated deployment plans, represent the clearest early adopter segment because they have both the procurement scale and the internal engineering capacity to manage a platform transition. Frontier AI labs diversifying their compute supply chain, exemplified by Anthropic's 2 gigawatt commitment beginning in the first half of 2027, represent a second category driven as much by supply security as by cost or performance. Sovereign AI programs requiring in-country deployment, served by the Venice X and MI430X sovereign variant, represent a third category where open standards and data residency requirements align naturally with what Helios offers.

Enterprises further from the frontier of AI infrastructure procurement will likely encounter Helios indirectly at first, through cloud providers and OEM partners like HPE and Supermicro offering it as an instance type or managed deployment option rather than through direct rack procurement. This is a more realistic adoption path for most organizations, since it removes the facilities, cooling, and operational burden while still capturing whatever price and performance advantage the underlying hardware delivers. Even for organizations that never deploy AMD hardware directly, the existence of a credible alternative to Nvidia's NVL72 class systems is likely to influence pricing and availability across the broader GPU cloud market over the next several procurement cycles, simply through the negotiating leverage a genuine second option provides.

KriraAI tracks infrastructure shifts like the Helios launch closely because the economics of inference and training compute directly shape what is realistic to build for enterprise clients, particularly for high-throughput agentic workloads where cost per token determines whether a use case is viable at production scale. Understanding the real trade-offs between open-standard and proprietary rack-scale platforms, rather than reacting to vendor marketing claims in isolation, is part of how KriraAI advises clients on infrastructure decisions that will shape their AI cost structure for years, not just quarters.

Conclusion

AMD Helios rack-scale AI infrastructure matters for three concrete reasons that any enterprise infrastructure team should take away from this launch. First, it works by combining 72 MI455X GPUs, Venice CPUs, and open standard UALink over Ethernet networking into a single validated rack that reaches exaflop-scale performance without relying on a proprietary scale up fabric. Second, it matters most for hyperscalers, frontier AI labs, and sovereign AI programs that need genuine supply chain diversification and negotiating leverage against single-vendor dependency, though its influence on pricing and availability will ripple across the broader GPU cloud market even for organizations that never deploy it directly. Third, organizations evaluating it now should treat vendor-published benchmarks as directional rather than final, validate ROCm coverage against their own specific workloads, and plan a phased pilot rather than a full platform cutover given how recently the system reached general availability.

KriraAI stays close to infrastructure shifts like this one precisely because the cost and performance characteristics of the underlying compute layer determine what enterprise AI systems are actually viable to build and operate at production scale, and applying a new platform before its software ecosystem and benchmark evidence are ready wastes client budget rather than accelerating outcomes. The company's approach has always been to track the frontier of AI hardware, models, and techniques carefully and apply what is genuinely production-ready rather than chasing every new release. If your organization is weighing what AMD Helios rack-scale AI infrastructure, or the broader shift toward open-standard AI compute, could mean for your own infrastructure roadmap and cost structure, KriraAI is well positioned to help you evaluate that decision against your actual workloads rather than vendor marketing claims.