Networks Struggle to Handle Growing AI Traffic

AI compute capacity has ballooned, but networking is now a systemic bottleneck. Analyst firm Omdia finds many "neocloud" and GPU-as-a-service providers scaled GPU fleets faster than they hardened network stacks, creating real-world performance and sovereignty constraints. Camille Mendler warns that "network infrastructure will make or break neoclouds." Lumen CEO Kate Johnson frames the network as the "nervous system" for AI, and the industry is responding with upgrades and vendor repositioning.

The mismatch shows up across several dimensions: throughput, latency, resilience, and data locality. AI workloads move large datasets and parameter updates between clouds, datacenters, and edge endpoints, so bandwidth alone is not sufficient. Practitioners should evaluate:

• •network topology and peering quality between sites, not just raw bandwidth
• •latency and jitter for synchronous training and inference workloads
• •secure, auditable paths for cross-border data movement and sovereignty controls
• •edge connectivity and last-mile performance for low-latency inference

Providers and customers are using a mix of approaches: colocating datasets with compute, private backbones or direct interconnects, WAN optimization and compression, RDMA/GPUDirect-style transports where supported, and hybrid topologies that minimize cross-site gradients. Origin stories matter: firms that evolved from CDN or hosting may already have peering and backbone assets, while those from crypto mining often lack sophisticated network operations.

This is a structural shift in how AI infrastructure is evaluated. The market previously emphasized GPU count and accelerator flops; the next wave privileges integrated network engineering, data governance, and cross-site orchestration. That rebalances competition: hyperscalers and network-savvy providers gain an advantage, and neoclouds without robust connectivity face pressure to partner, buy, or build. Enterprises need to revise procurement checklists to include network SLAs and data flow architecture.

Expect vendor differentiation around private interconnects, managed backhaul, and sovereign networking features. Procurement teams should require topology diagrams, measured latency/throughput tests, and clear data sovereignty controls as part of any AI compute RFP.