Making Topology-Aware Scheduling Practical for AI Workloads: From Discovery to Simulation at Scale#

Weizhou Lan, Daocloud

• Why: Names the physical fabric explicitly — scale-up interconnect and RDMA spine–leaf — as the thing the scheduler has to understand. This is where DC topology and Kubernetes scheduling actually meet.
• What: A practical approach to multi-level topology discovery feeding topology-aware scheduling in Kueue, plus a way to validate it without buying the hardware.
• How: Three stated parts: dynamic topology discovery and health detection across scale-up, RDMA spine and RDMA leaf layers; topology-aware scheduling with priority-based placement so GPUs get optimal communication paths; and Kwok simulating thousands of virtual nodes with multi-level topologies for zero-hardware-cost validation at scale.
• Where applicable: Multi-tenant AI inference clusters with heterogeneous GPU interconnect. The Kwok simulation trick generalizes to any large-topology scheduling validation, independent of the GPU specifics.