Skip to content

From Kernels to Clusters: How PyTorch Powers High-Performance AI (Presented by Meta)#

GTC 2026 digest Program Adjacent pick San Jose · 16–19 March 2026

Abstract Talk page Slides Recording

Alban Desmaison (Meta) — Theater Talk, Monday, March 16

  • Why: Torchcomms — a vendor-neutral, scalable communication layer — is the open-ecosystem counterpart to NCCL/NIXL, and the PM's dataplane interest makes that layer's direction worth knowing.
  • What: PyTorch's high-performance stack: custom kernel authoring, the Helion compiler stack, Monarch's next-generation distributed runtime for reinforcement learning, Torchcomms' vendor-neutral communication layer, and fault tolerance/elasticity for reliable training at massive scale. (from abstract only)
  • How: Abstract describes scope but not mechanisms; the comm-layer and elasticity internals are what to look for in the full talk.
  • Where applicable: Open-source PyTorch ecosystem, Meta-scale training context; relevant to anyone evaluating whether the open stack can route around vendor-locked communication libraries.