ConnectX-8 Connectivity Solution
matching the right interconnect strategy to our specific cluster architecture
As AI and HPC clusters move toward 800G and 1.6T interconnect speeds, network architecture design becomes increasingly critical.
ConnectX-8 delivers ultra-high-speed networking and higher throughput for modern AI and HPC infrastructure.
In this article, we’ll walk through practical connectivity solutions for CX-8 in different scenarios.
C8180 - One OSFP Cage(800G XDR or 2 x 400GbE)
The C8180 adapter provides one OSFP cage and supports:
800G XDR (single-port mode)
2 × 400GbE (breakout mode)
Scenario 1: 1.6T using 2 x C8180 in XDR 800G Mode
In this architecture, NVIDIA Quantum-3 based XDR InfiniBand Switch, Q3400-RA, 144 XDR Ports over 72 OSFP Cages, operates at 2x XDR800G. Each C8180 connects using one 800G link.
Two C8180 adapters together deliver 1.6T aggregate bandwidth.
Optical Solution
Use a twin-port 2x XDR800 OSFP module on the switch side
Each port connects to a single-port XDR800 OSFP module on C8180
Two single mode MPO-12/APC fibers are used for interconnection
This solution supports longer reach and is ideal for large-scale cluster deployment.
Copper Solution
For rack-level deployment:
1.6T OSFP to 2 × OSFP DAC (≤1m)
1.6T OSFP to 2 × OSFP AEC (≤3m)
This is suitable for Top-of-Rack GPU servers with minimal insertion loss requirements.
Scenario 2: 1.6T using 4 × C8180 in NDR 400G Mode
Using four C8180 adapters with Quantum-3 based XDR InfiniBand Switch (Q3400-RA) to hit 1.6T aggregate bandwidth.
Optical Solution
Use a twin-port 2 × XDR800 OSFP module on the switch side
Each port connects to a single-port NDR400 OSFP module on C8180
Two single mode fiber splitters are used for interconnection
Copper Solution
1.6T OSFP to 4 × OSFP DAC (≤1m)
1.6T OSFP to 4 × OSFP DEC (≤3m)
Scenario 3: 800G using one C8180 in 2x400G Mode
The SN5400 with 64 ports of 400GbE over 36 OSFP, operates at 800GbE. C8180 connects using one 800G link.
One C8180 adapter delivers 800G bandwidth.
Optical Solution
Use a twin-port 2 × 400G OSFP module on the switch side
Each port connects to a twin-port 2 x 400G OSFP module on C8180
Two Single mode MPO-12/APC fibers are used for interconnection
Copper Solution
800G OSFP to OSFP DAC (≤2m)
800G OSFP to OSFP DEC (≤7m)
C8240 - Two QSFP Cages(2 x 400G)
The C8240 adapter provides two QSFP cages and each cage support 400Gbps. Two cages together deliver 800G aggregate bandwidth.
Scenario: 800G using one C8240 in 2 x 400GbE or 2 x NDR 400G Mode
Optical Solution
Use a twin-port 2 × 400G OSFP module on the switch side
Each port connects to a single-port 400G QSFP module on C8240
Two single mode MPO-12/APC fibers are used for interconnection
Copper Solution
800G OSFP to 2 x QSFP DAC (≤2m)
800G OSFP to 2 x QSFP DEC (≤7m)
C8240 - Two QSFP Cages(400G or 2 x 200G)
The C8240 adapter provides two QSFP cages and supports:
400G(single-port mode)
2 × 200G (breakout mode)
Scenario 1: 800G using 2 x C8220 in 400GbE or NDR 400G Mode
Optical Solution
Use a twin-port 2 × 400G OSFP module on the switch side
Each port connects to a single-port 400G QSFP module on C8220
Two single mode MPO-12/APC fibers are used for interconnection
Copper Solution
800G OSFP to 2 x QSFP DAC (≤2m)
800G OSFP to 2 x QSFP DEC (≤7m)
Scenario 2: 800G using 2 x C8220 in 2 x 200GbE or 2 x HDR 200G Mode
Optical Solution
Use a twin-port 2 × 400G OSFP module on the switch side
Each port connects to a single-port 400G QSFP module on C8220
Two single mode fiber splitters are used for interconnection
Copper Solution
800G OSFP to 4 x QSFP DAC (≤2m)
800G OSFP to 4 x QSFP DEC (≤7m)
As we’ve seen, when multiple CX-8 adapters are deployed per node, aggregate bandwith can scale to 800G and 1.6T to meet demanding AI/HPC workloads.
Whether you prioritize the ultra-low latency of copper solutions for in-rack power efficiency or the long-reach reliability of optical solutions for spine-leaf interconnects, the key remains the same: matching the most suitable interconnect strategy to your specific cluster architecture.
Thanks to Hitesh’s article for the inspiration.