Kshitij Mahant is Director of Technical Marketing Engineering at Cisco, where he leads Competitive Intelligence for Enterprise Networking.
For the past three decades, networking and application development operated independently. Applications worked with the network as it was: occasionally congested, reliable but not guaranteed. The network was engineered as a neutral substrate, optimized for packet movement without concern for content.
However, AI workloads disrupt this arrangement. Large-scale training and inference jobs require networks capable of handling massive, bursty flows between thousands of accelerators with microsecond-level fairness. At the same time, AI has become a powerful tool for network management: predicting congestion, correlating telemetry and automating remediation.
This convergence of networking for AI and AI for networking presents significant engineering challenges. However, as a co-design loop, they point toward an intriguing outcome: increasingly intelligent infrastructure that supports AI at the same pace as AI improves its ability to manage infrastructure.
Traditional enterprise and data center traffic is forgiving. Dropped packets in video calls slightly degrade quality, and retransmits in file transfers cost only milliseconds. However, distributed AI training operates differently. Gradient synchronization across GPUs and accelerators is tightly coupled, where every participant waits for the slowest one. A single congested link or poorly load-balanced path slows its own flow and stalls an entire job across thousands of clusters, resulting in significant wasted compute and energy.
Reevaluating ‘Good’ Performance
These new network requirements call for a reevaluation of “good” performance. Metrics that matter are tail latency, flow completion time and fairness across limited massive, synchronized flows. The economic landscape also transforms. When a training cluster incurs costs equivalent to its hourly running expenses, the network becomes an integral part of the business case.
This rapid transition from research papers to shipping product roadmaps in a few years highlights the importance of lossless fabrics, AI-tailored congestion control and topology-aware scheduling. Networking for AI is a fundamental rethinking of network assumptions, not an incremental evolution of DC design.
Campus and workplace networks were designed for predictable user bases, including employees and limited IT/IoT devices active during business hours with predictable traffic patterns. However, today’s workplaces require networks to support five to 10 times as many AI agents that work continuously alongside employees rather than clocking in and out. Instead of traffic networks designed for bursts of activity, we now face exponential north-south and east-west traffic that never truly quiets down. A nine-to-five network can’t handle an always-on agent population without rethinking capacity planning, segmentation and traffic prioritization.
The Blurred Boundary Between The Network And The AI System
AI is becoming essential for network operations due to the massive volumes of telemetry generated by enterprise and campus networks. AI enables real-time telemetry usability, unlike post-mortem analysis.
Production already sees AI-driven automation, such as radio management with independent power and channel assignments, anomaly detection and network health platforms correlating user complaints with root causes. The move from traditional NetOps to AgenticOps involves AI agents handling detection, correlation and remediation, with engineers setting policies and managing exceptions.
The direct benefit is operational efficiency, while the indirect benefit is a better platform for AI workloads, such as predicting and rerouting around congestion before the training job stalls. But the true opportunity lies in a closed-loop system, where AI-native environments directly influence network design decisions, such as buffer sizing, topology and load balancing algorithms, based on the workload characteristics of training and inference jobs.
Simultaneously, the operational intelligence running that network, AgenticOps, is an AI workload that benefits from the same low-latency, high-reliability infrastructure as training jobs. A network operations model relying on real-time inference for routing or remediation decisions can’t tolerate the congestion and jitter it aims to manage. This is AI depending on networking.
Extending this creates a feedback loop: Telemetry from AI workloads informs network design and operation, and the resulting network characteristics feed back into AI scheduling and placement. The operational AI continuously retrains on observed patterns, enhancing each other and blurring the boundary between “the network” and “the AI system.”
Implications For Infrastructure Construction
Networking and AI infrastructure decisions can’t be planned independently with separate owners who only compare notes quarterly. Network architecture and AI workload placement and orchestration teams need shared visibility into each other’s roadmaps because decisions in one domain have direct consequences in the other.
The skill set for infrastructure teams is evolving. Understanding AI workload behavior is crucial to network design, and understanding network telemetry is important to AI operations teams.
Networking and AI aren’t merging into a single function. They remain distinct disciplines with distinct expertise. What’s changing is the degree of integration between them, which is expected to increase as AI becomes a larger portion of the enterprise infrastructure.
Call To Action For Infrastructure Leaders
Leaders must consider the integration of networking and AI in their decisions. To deliberately close this loop, leaders can take three actions:
1. Rethink organizational design by aligning the reporting lines of network fabric designers and AI workload schedulers. This shared ownership helps ensure decisions in one domain directly impact the other.
2. Align planning and budgeting cycles by incorporating accelerator procurement, fabric upgrades and AI workload capacity planning into a single budgeting cycle. This helps prevent making decisions without visibility into the next training cluster’s scale or vice versa.
3. Establish a regular cadence of measuring key metrics, such as job completion time, mean time to detect and remediate, and agent-to-human traffic, such as quarterly.
Organizations that adopt these habits will continuously improve their infrastructure capabilities. Those that maintain separate tracks will spend years reconciling decisions that should have been made together.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?