SRv6 and MRC for AI Data Centers#

Michal Styszynski – Juniper Networks

• Why: The backend AI fabric problem this program circles repeatedly — making RoCEv2 survive at 100k+ GPU scale without PFC — approached via an emerging OCP spec (MRC). Core to the PM's RDMA/lossless-fabric interest. Vendor session, but grounded in a specification rather than a product pitch.
• What: MRC (Multipath Reliable Connection) is modified RoCEv2 with reliability (SACK/NACK), packet trimming on congestion, packet spraying, and multiplanar fabric design — explicitly "no more PFC." Two transport flavors: IP-ECMP MRC (works with EVPN-VXLAN multitenancy, switch hashing) and SRv6 MRC (source-routed paths computed by a path orchestrator/fabric manager). Scale worked through an 8-plane design: ~131K GPUs, ~8K GPUs per pod, with optical shufflers breaking 2×800G into 8×200G plane links.
• How: SRv6 uSID shift-to-left: the orchestrator computes congestion-avoiding healthy paths and encodes each as a uSID chain in the outer IPv6 destination address; the NIC encapsulates RoCEv2, each transit node executes shift-left, and the leaf decaps. RoCEv2 Entropy Values are mapped to SRv6 Path-IDs at the NIC, which rotates across them; on congestion the switch trims the packet to headers, the receiver NACKs with source PSN, and the sender changes EV/Path-ID. The BTH header gains RTX/TSETH fields and new opcodes (e.g. reliable probe 0xDE), per the OCP MRC 1.0 spec.
• Where applicable: Requires MRC-capable NICs, AI Ethernet switches, and a path controller — an early, ecosystem-dependent stack. Multiplane designs assume no cross-plane links. Watch it as the OCP-specified alternative to PFC-based lossless fabrics; not something deployable on current commodity gear.