Global networks were not originally designed for the type of traffic AI generates. Right now, infrastructure is primarily designed to meet predictable workloads and legacy demand for video, SaaS, and cloud migration. AI behaves very differently. It strains networks at all levels, whether it’s via applications, training runs, or everyday user interactions, and traffic is highly distributed, bursty, and latency-sensitive.
These patterns ripple across every layer of infrastructure, from application stacks to interconnects, pushing existing architectures toward a tipping point for what they were designed to handle.
What’s driving AI network demand?
Any time an AI model generates outputs or makes decisions, it goes through the process of AI inference by referencing back to its training data and extrapolating its best educated guess. Each response requires a model to process an input, reference patterns learned during training, and calculate the most probable output. This process puts incredible strain on network infrastructure as it happens billions of times per day when users interact with AI through search, productivity tools, coding assistants, enterprise applications, and increasingly autonomous systems.
Adding to the network demand challenge is the fact that generative AI models don’t live on a single machine or device. They operate across a wide variety of GPUs that span multiple data centers, clouds, and edge locations. When a user submits a prompt, the resulting inference can trigger a cascade of network activity as systems retrieve data and coordinate across clouds, tools, and data sources. In many cases, the “brain” of the model is effectively scattered across the network.
The efficiency of AI moving across the network is constrained by timely processing demands. LLMs are products and retaining razor-thin attention spans is top priority. As such, providers put emphasis on fast response times though low-latency channels. This makes sense in theory, but in practice, if everyone is competing for the same slice of the latency pie, there isn’t enough to go around. As such, organizations must prioritize expanding their network.
Where traditional networks falter
As AI traffic continues to proliferate, the weakest areas will become bottlenecks. Without modernization, traditional interconnect and DCI links are particularly vulnerable as they meet at the intersections of AI workloads. Specifically, north-south traffic will be greatly outpaced by east-west traffic, as information flows between data centers and external devices, instead of between internal servers. As a result, internal and inter-data-center links are increasingly under strain.
Some industries are demanding a higher network usage than others. For example, Megaport’s 2025 Cloud Network Report found that financial VXC capacity grew 2x more than than any other industry. As verticals with high AI-demand such as commerce grow their usage, networks will need to rapidly scale alongside them.
WAN is another area where things could falter. Some estimates project up to a 700% increase in traffic by 2034, and we’re already starting to see it play out in real time. In certain high-demand AI corridors, traffic has already grown tenfold in the past 24 months, putting strain on vital channels and slowing down the systems that rely on them.
The reality is that networks are asymmetric by design. Most existing IT infrastructure setups were never intended to handle high traffic in the areas where AI demands it. Downlink connections represent around 85% of internet capacity, with uplink only capturing 15%. AI will heavily skew this ratio, with estimates predicting that uplink could skyrocket up to 35% of traffic share, flipping the traditional model.
If infrastructure isn’t prepared for the flood of AI demand, network capacity could be swept away in the tide of token requests.
Where enterprises can patch the holes
Addressing these network capacity challenges will require a rethink of how networks are designed. The infrastructure that powered the first wave of cloud computing was built around general-purpose connectivity, where the same network handled everything from web traffic to storage access. AI is already forcing a different model to emerge.
Right now, successful enterprises are moving towards a hybrid model where public clouds support traditional workloads, but sensitive workloads are transitioning back to traditional data centers or specialized bare metal providers. This approach diffuses a portion of demand without sacrificing key protections.
Latency is a top concern amongst global IT leaders, and as organizations step in to address this issue, some have found success in moving latency-sensitive operations, such as AI inference, to edge-first architectures. Building infrastructure with service-level intent will be particularly important.
Providers are beginning to build dedicated connectivity fabrics between AI facilities, extending fiber capacity and high-speed optical links to locations where GPU clusters live. The goal is to create networks that treat distributed compute resources as a single logical system rather than isolated data centers.
Keeping up with this demand will also require significant network investment over the coming years. Providers will need to expand the physical reach of their networks while also deploying technologies that extract greater efficiency from existing infrastructure, from higher-capacity optical links to more intelligent traffic management and routing. These investments have to evolve inline with the rapid expansion of AI-focused data centers, ensuring that connectivity scales alongside the compute powering the next generation of AI workloads.
Take action now
The rise of AI agents is already straining traditional network architecture. Agent-to-agent communication often demands more tokens than human-to-AI prompting, and with big players like Nvidia pushing for an agent-forward future, it’s unlikely that existing networks have sufficient capacity for what’s to come.
As enterprises widen their AI usage and integrate agentic solutions, it’s vital that their network infrastructure grows alongside it. Sufficient low-latency connections can make or break efficiency metrics, and we’ve already observed certain channels experiencing over 100% increases in traffic. Don’t let your organization fall behind. Build network architecture that enables AI now.
