fabric-lib: RDMA Point-to-Point Communication for LLM Systems#

Nandor Licker, Kevin Hu, Vladimir Zaytsev, Lequn Chen (Perplexity AI, per the paper's open-source repository)

• Why: Disaggregated inference, MoE expert routing, and async RL fine-tuning all need flexible point-to-point RDMA, not just collectives — but today's implementations are locked to specific NICs. This is the clearest statement of the problem at the point where RoCE fabrics and LLM serving actually meet, and it ships as open source.
• What: A NIC-abstraction library that exposes uniform one-sided WriteImm operations across heterogeneous NICs, demonstrated at 400 Gbps on both NVIDIA ConnectX-7 and AWS EFA, and proven through three production use cases: KV-cache transfer for disaggregated inference, RL weight sync (1.3 s for trillion-parameter models), and MoE dispatch/combine that beats DeepEP decode latency on ConnectX-7 — with the first viable EFA latencies (from abstract only).
• How: Bridges common NIC functionality behind one interface; an ImmCounter primitive provides completion notification without assuming transport ordering; multiple NICs per GPU are managed transparently. It complements collectives rather than replacing them.
• Where applicable: Any RoCEv2 or EFA environment with point-to-point LLM traffic. The headline numbers are specific to ConnectX-7 and AWS EFA, and the MoE comparison is against DeepEP — but the portability argument and the code (pplx-garden on GitHub) are broadly usable.