Data Center Interconnect 2026: Planning 800G, 1.6T and Optics

As AI clusters grow, the scale-out network between servers has become a critical path for training and inference efficiency. Switch silicon has reached 102.4 Tb/s, NICs now deliver 1.6 Tb/s of network bandwidth per GPU, and optics have split into pluggable, linear-drive and co-packaged approaches. This article reviews the main technology milestones for data center interconnect in 2026 and the conditions to confirm during planning. It focuses on Ethernet and optical links between racks; for scale-up interconnect inside the rack, see the AI server rack article.

Key Takeaways

Switch silicon: Broadcom began shipping the 102.4 Tb/s Tomahawk 6 in June 2025 and announced production-volume shipments in March 2026.
Co-packaged optics: Broadcom Tomahawk 6–Davisson integrates sixteen 6.4 Tb/s optical engines; NVIDIA Spectrum-X Photonics offers the SN6810 at 102.4 Tb/s (128 ports of 800G) and the SN6800 at 409.6 Tb/s (512 ports of 800G).
NICs: NVIDIA ConnectX-9 SuperNICs provide 1.6 Tb/s of network bandwidth per GPU.
Standards: IEEE P802.3dj defines 200G through 1.6T Ethernet on 200 Gb/s-per-lane signaling; the Ultra Ethernet Consortium released its 1.0 specification in June 2025.
Optics: OSFP-XD doubles the electrical lanes from 8 to 16; the LPO MSA has released linear pluggable specifications at 100 Gb/s per lane and started work on 200 Gb/s per lane.

Scale-Up vs. Scale-Out: Two Different Networks

Both are called interconnect, but links inside the rack and links between racks solve different problems and should be planned separately.

Scale-up interconnect joins GPUs within one coordination domain into a single high-speed fabric, such as an NVLink rack. The scale-out network connects multiple nodes or racks and carries cross-node collective communication, data loading and storage traffic. The platform vendor sets the former as part of the rack design; the latter is assembled from switches, NICs, optics and cabling and planned by the operator around cluster size, topology and the existing network.

This article covers the scale-out network. Once per-GPU network bandwidth rises, switch port counts, optics power and data center cabling all need to be recalculated together.

Switch Silicon: The 102.4 Tb/s Generation

Total switch bandwidth determines how many ports a switch can offer at a given speed, which in turn affects how many network tiers are needed.

Broadcom announced in June 2025 that it was shipping Tomahawk 6, with 102.4 Tb/s of switching capacity on a single chip, and said in March 2026 that the Tomahawk 6 family had reached production-volume shipments. At 102.4 Tb/s, one chip can provide 64 ports of 1.6T or 128 ports of 800G (total bandwidth divided by port speed).

More ports per switch can mean fewer tiers and fewer switches for a cluster of the same size, with fewer hops between tiers. The actual design still depends on topology, oversubscription and how failure domains are drawn.

Port Speeds and NICs: From 800G to 1.6T

As NIC bandwidth rises, switch port speeds, optics and cabling need to be matched at the same rate.

In the Vera Rubin platform, NVIDIA's ConnectX-9 SuperNIC provides 1.6 Tb/s of network bandwidth per GPU. 1.6 Tb/s is about 200 GB per second and 800 Gb/s about 100 GB per second (at 8 bits per byte). When NIC and switch port speeds differ, breakout cables or speed conversion are needed, which changes cabling and the number of optical modules.

On the standards side, the IEEE 802.3 working group's P802.3dj project defines 200G, 400G, 800G and 1.6T Ethernet, with physical layers based on 200 Gb/s-or-greater per-lane signaling over copper, backplanes and single-mode fiber.

Optics: Pluggable, LPO and Co-Packaged

At higher port speeds, optics power and density become a primary constraint on switch design, and the three approaches involve different trade-offs.

Pluggable optics: Keep the field-replaceable module model. The OSFP MSA's OSFP-XD specification doubles the electrical lanes from 8 to 16, leaving headroom for higher total bandwidth.
Linear pluggable optics (LPO): Remove the in-module DSP to cut power and latency. The LPO MSA released its 100G-DR-LPO specification in March 2025 and its 400G-FR4-LPO specification in September 2025, both at 100 Gb/s per lane over single-mode fiber to at least 500 m, and has started work on 200 Gb/s per lane.
Co-packaged optics (CPO): Package the optical engines with the switch silicon to shorten electrical paths. Broadcom Tomahawk 6–Davisson uses sixteen 6.4 Tb/s optical engines; NVIDIA Spectrum-X Photonics includes the SN6810 at 102.4 Tb/s with 128 ports of 800G and the SN6800 at 409.6 Tb/s with 512 ports of 800G.

The approaches differ mainly in power, serviceability, supply chain maturity and fit with existing operations. Selection should account for how optics are replaced, how faults are handled and how spares are arranged.

The Protocol Layer: Ultra Ethernet

As hardware bandwidth rises, collective communication efficiency depends more on the transport protocol, congestion control and NIC support.

The Ultra Ethernet Consortium released the Ultra Ethernet 1.0 specification in June 2025. It covers the communication stack spanning NICs, switches, optics and cables, aiming to make Ethernet better suited to large-scale AI and HPC workloads; the specification is publicly downloadable and continues to receive revisions. Adopting a new protocol means confirming supported versions across NICs, switch firmware, operating system drivers and collective communication libraries.

Planning Framework

Planning item	Conditions to confirm
Bandwidth target	Whether per-GPU network bandwidth, NIC speed and switch port speed line up
Topology and tiers	Cluster size, switch port count, number of tiers and hops between tiers
Oversubscription	Whether training, inference and storage traffic share one network, and the oversubscription ratio at each tier
Optics approach	Pluggable, LPO or co-packaged; reach, power and replacement model
Cabling	Fiber type, breakout cables, distance between racks and pathway capacity
Protocol and software	RDMA transport, congestion control, Ultra Ethernet support and collective library versions
Operations	Failure domains, spares, firmware upgrades and monitoring

FAQ

How should I choose between 800G and 1.6T?: Start from per-GPU network bandwidth and NIC speed, then match switch port speed and optics. A 102.4 Tb/s switch chip provides 64 ports of 1.6T or 128 ports of 800G; the choice affects port count, cabling and the number of network tiers.
Will co-packaged optics replace pluggable modules?: The two coexist. Co-packaged optics shorten electrical paths, while pluggable modules keep field replacement. The choice depends on power targets, service processes and compatibility with the existing network.
How does in-rack NVLink differ from Ethernet between racks?: Scale-up interconnect such as NVLink joins the GPUs in a rack into one high-speed domain; scale-out Ethernet connects multiple nodes or racks and carries cross-node communication, data loading and storage traffic.