Introducing NCCL Extensions: Communication Patterns for Modern AI#
PyTorch Conference 2026 digest Program Core pick 20–21 October 2026
Sreeram Potluri, Artem Polyakov — NVIDIA
- Why: NCCL is the dominant collective library on GPU fabrics; these domain extensions change what traffic patterns the library can express — MoE dispatch/combine and cross-mesh resharding — which is precisely the traffic your inference and RL-rollout fabrics will carry.
- What: Two NCCL domain extensions built on NCCL's Device APIs. NCCL-EP brings MoE dispatch and combine natively into NCCL — one automatically tuned API spanning latency-optimized decode and throughput-optimized training, with GPU-initiated RDMA and NVLink out of the box. NCCL-M2N does zero-copy resharding between disjoint device meshes, moving weight snapshots from trainers to inference replicas with no CPU staging. Performance results promised in the talk. (from abstract only)
- How: Specialized libraries inheriting NCCL's topology awareness and transport stack, but designed for customization, kernel fusion, and rapid prototyping of emerging patterns rather than classic collectives.
- Where applicable: NVIDIA-centric (NCCL ecosystem), and a vendor talk by NVIDIA architects — but it defines the de-facto API surface that open alternatives (OpenCLIC, TorchComms backends) will be measured against.