Skip to content

Demo Theater: OpenCLIC: Open Communication for Distributed Inference#

PyTorch Conference 2026 digest Program Core pick 20–21 October 2026

Abstract

Jesse Treger

  • Why: This is the Ultra Ethernet / connectionless-RDMA story arriving at the framework layer — squarely in physical-fabric territory. Its core argument is that MoE inference traffic (dynamic, many-destination all-to-all) is a structural mismatch for the connection-oriented RDMA transports most AI clusters run today.
  • What: OpenCLIC, an open communication framework for connectionless networks, being integrated with vLLM. First use case is expert-parallel dispatch/combine; planned support for one-sided operations — KV-cache transfer, cache reuse/offload, and model-state movement. It separates framework interfaces, GPU kernels, and transport implementations so inference code isn't tied to one accelerator or network stack. (from abstract only)
  • How: Built on Ultra Ethernet Transport, modeled on HPE Slingshot: no per-peer connections, improving multipathing, loss recovery, and scalability. The initial Slingshot implementation uses kernel-initiated OpenSHMEM so GPU code drives the network directly and the CPU stays out of the critical path.
  • Where applicable: Initial implementation is HPE Slingshot-specific (HPC heritage), with the architecture positioned to extend across accelerators and fabrics. Demo-theater short format, and the program attaches a sponsored-content notice — expect a direction statement and early integration, not a full evaluation.