Google Unveils A5X Scaling to 960,000 GPUs

Google Cloud and NVIDIA announced the next-generation AI infrastructure offering, `A5X`, a rack-scale system built on `Vera Rubin NVL72` hardware, ConnectX-9 SuperNICs, and Google networking. The codesigned stack claims up to 10x higher token throughput per megawatt and 10x lower inference cost per token compared to the previous generation. A5X scales to 80,000 Rubin GPUs in a single-site cluster and to 960,000 Rubin GPUs across multi-site clusters, and it introduces previews of running `Gemini` on Blackwell and Blackwell Ultra GPUs within Google Distributed Cloud and confidential VMs.

The announcement emphasizes extreme codesign across chips, systems, and software to reach the stated efficiency gains. Key components include:

• •`Vera Rubin NVL72` rack-scale systems providing the GPU building blocks
• •ConnectX-9 SuperNICs paired with Google's next-generation Virgo networking fabric for high-throughput, low-latency interconnect
• •Multi-site orchestration that federates up to 960,000 Rubin GPUs, and single-site clusters up to 80,000 GPUs

The 10x claims target token throughput per megawatt and inference cost per token; those metrics matter for inference economics and for running long-horizon agentic workloads. Running Gemini on Blackwell hardware inside Google Distributed Cloud and confidential VMs gives enterprises a path to deploy frontier models close to sensitive data while retaining hardware-level confidentiality. The stack integrates with agent frameworks and NVIDIA toolchains such as Nemotron and NeMo for model development and optimization.

This is a clear infrastructure play to make hyperscale model training and inference more economical and to enable "AI factories" - production deployments that coordinate agents, digital twins, and robots in industrial settings. The multi-site pooling number, 960,000 GPUs, is notable because it signals both ambition and an engineering bet on wide-area GPU fabric, global orchestration, and power-efficient datacenter interconnect. For cloud users, lower inference cost per token can change the unit economics of large-scale chatbots, real-time agents, and robotics workloads. For competitors, the combination of confidential Blackwell VMs and on-premise Distributed Cloud support raises the bar for private, regulated deployments.

Scaling to near-million-GPU federations is technically difficult. Key operational constraints include cross-site latency, model parallel synchronization, failure modes at scale, and realistic utilization that delivers the claimed cost savings. Details on pricing, region availability, SLAs, scheduler semantics for multi-site jobs, and support for open-source model runtimes were not disclosed in full. The confidential Blackwell capability is strategically important for regulated industries, but auditors and security teams will ask for attestation details and threat models.

Availability dates, pricing and instance types for A5X bare-metal, the software toolchain Google exposes for cross-site orchestration, and benchmarks on real-world agentic workloads. Also watch how competitors respond on multi-site scaling and confidential GPU offerings.

A5X is an infrastructure-first push to make very large training and inference workloads cheaper and to support sensitive, distributed deployments of frontier models. For teams building agentic systems, robotics, or large-scale inference services, the combination of codesigned hardware, high-performance networking, and confidential GPU options materially changes the deployment choices available today.