Skip to content

Mitigating Scalability Walls of RDMA-based Container Networks#

NSDI '25 digest Program Core pick 28–30 April 2025

Abstract Paper Recording

Play the recording: Mitigating Scalability Walls of RDMA-based Container Networks

  • Why: RDMA-offloaded container networks deliver big wins (the paper shows ~21 Gbps TCP rising to ~180 Gbps once forwarding and VXLAN move to the RNIC and RDMA carries packets), but performance collapses unexpectedly at the scale of millions of containers. Anyone betting on RNIC offload for container fabric needs to understand this ceiling.
  • What: The "scalability wall" is traced to the RNICs themselves — design/implementation defects that are invisible because RNIC internals are closed. They build ScalaCN, which infers the most likely causes despite that opacity and proactively reschedules NF offload.
  • How: Combinatorial causal testing reconstructs an approximate RNIC architecture and performance model from black-box behavior; ScalaCN then optimizes the network-function offload schedule against that model. On production workloads: ~1.4x end-to-end bandwidth and ~31% lower forwarding latency after resolving 82% of inferred causes. Issues were reported to and confirmed by vendors.
  • Where applicable: Production RDMA container networks at hyperscale; evaluation is on Alibaba's production RCN with NVIDIA ConnectX-6 RNICs, so specifics are vendor- and deployment-tied, but the causal-inference methodology for black-box NICs generalizes.