Solve the Day 1 Challenge of Landing LLM Inference in Enterprise#

Tongyu Guo, Kai Zhang (Alibaba Cloud Intelligence) — Talk, Thursday, March 19

• Why: Kubernetes-native lifecycle management for prefill-decode disaggregated inference — the cloud-native-orchestration side of a problem the NCCL/NIXL talk covers from the datapath side.
• What: ACK RBG, Alibaba Cloud's tool for full-lifecycle LLM inference deployment on Kubernetes: multi-role management, automatic service discovery, topology-aware placement, elastic scaling, and atomic rollout/failure recovery for PD-disaggregation architectures, tightly integrated with NVIDIA Dynamo's AIConfigurator and Planner for performance profiling and smarter scaling in PD and MoE scenarios. (from abstract only)
• How: Kubernetes controllers/operators implementing the lifecycle features, plus Dynamo integration for profile-driven scaling decisions; abstract doesn't detail placement or recovery internals.
• Where applicable: Alibaba Cloud ACK, Kubernetes-based; specific to PD-disaggregation and MoE inference at scale. One of the few documented large-scale production deployments of NVIDIA Dynamo.