NVIDIA Reports Vera Rubin NVL72 Agentic Efficiency Gains
NVIDIA reported on August 24 that its Vera Rubin NVL72 delivered up to 30 times higher throughput per megawatt than its GB300 NVL72 on the SemiAnalysis AgentX agentic coding workload. NVIDIA also reported 35 times lower token costs, based on measurements that replay recorded coding-agent trajectories with growing context, tool calls, and sub-agent activity.
NVIDIA reported on August 24 that its Vera Rubin NVL72 system achieved up to 30x higher throughput per megawatt than GB300 NVL72 on agentic AI workloads measured with SemiAnalysis AgentX. The company also reported 35x lower token costs for Vera Rubin NVL72 in the same comparison.
The results are based on NVIDIA's measurements using the AgentX workload, which replays recorded real-world agentic coding sessions. According to NVIDIA, those sessions preserve context growth, tool calls, and sub-agent spawning rather than reducing inference to fixed-length prompt-and-response tests. NVIDIA's technical blog describes AgentX as an open-source benchmark in SemiAnalysis's InferenceX suite, designed to capture production-style coding-agent behavior including long-context prefill, KV-cache reuse, tool-call gaps, and dynamic concurrency.
Why agentic measurement differs
NVIDIA argues that agentic workflows place different demands on inference infrastructure than single-turn chat and summarization. Its blog contrasts typical 1K to 8K token chat or document interactions with agent sessions in which context can accumulate to hundreds of thousands of input tokens across multiple steps.
The distinction matters because agent systems repeatedly process prior context while also generating new tokens, invoking tools, and sometimes coordinating sub-agents. In comparable deployments, benchmark outcomes can therefore depend heavily on request-length distributions, cache reuse, concurrency, and serving-runtime behavior, not only on a GPU's peak compute specifications.
System-level serving claims
NVIDIA's technical blog attributes the reported efficiency gains to a combination of software and hardware components. It identifies mixture-of-experts serving runtimes including SGLang, TensorRT-LLM, and vLLM; DeepGEMM-based kernels; MXFP4 and MXFP8 mixed-precision formats; the session-aware NVIDIA Dynamo serving stack; and NVLink connectivity across 72 GPUs.
These results should be read as vendor-reported, workload-specific measurements rather than a universal ranking across all inference jobs. AgentX's replay-based design is more representative of certain coding-agent sessions than conventional static benchmarks, but organizations evaluating infrastructure still need to test their own models, context lengths, precision choices, batching policies, tool latency, and service-level objectives.
NVIDIA noted that the figures are early Vera Rubin NVL72 results and that continuing software optimization could improve performance on both Vera Rubin NVL72 and GB300 NVL72.
Key Points
- 1NVIDIA reported up to 30x higher throughput per megawatt for Vera Rubin NVL72 on replayed agentic coding trajectories, versus GB300 NVL72.
- 2The AgentX benchmark includes context growth, tool calls, cache reuse, and dynamic concurrency, making it more relevant to coding agents than fixed prompts.
- 3For comparable agent deployments, serving efficiency commonly depends on cache behavior, request variability, precision, runtime software, and interconnect alongside accelerator performance.
Scoring Rationale
The report provides an unusually large vendor-reported efficiency comparison for rack-scale agentic inference, a growing workload category for AI infrastructure teams. Its practical importance is high, although the results are based on NVIDIA's measurements and a specific benchmark rather than independent cross-platform testing.
Sources
Primary source and supporting public references used for this report.
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.
Try 250 free problems
