Break Communication Barriers: Scale AI and HPC with NCCL, NIXL, and NVSHMEM#

Matthew Nicely (NVIDIA) — Talk, Thursday, March 19

• Why: The internals of the GPU communication stack — symmetric memory, copy-engine collectives, zero-SM communication, and NIXL's tier-spanning async transfers — are datapath and RDMA-adjacent content central to the PM's interests.
• What: A year of evolution across the three libraries: NCCL added Symmetric Memory and Copy Engine collectives enabling zero-SM communication; NVSHMEM enhanced GPU-initiated operations in the Python ecosystem; NIXL emerged for inference, providing async noncontiguous transfers across memory tiers (HBM, DRAM, local/networked storage) with GDS, UCX, and S3 backends. Plus developer-level material: custom NCCL collectives, KV-cache and storage innovations in NIXL, expert-parallelism patterns, and Spectrum-X plugins for multi-site scale-across. (from abstract only)
• How: Library-level mechanisms per above; the zero-SM collectives and NIXL's cross-tier transfers are the pieces that matter for disaggregated inference architectures.
• Where applicable: NVIDIA GPU clusters — vendor stack, but the de facto standard for GPU collective communication; the NIXL/disaggregated-inference material applies wherever KV cache and weights move between memory tiers over a fabric.