Performance Comparison of Transport Mechanisms for LLM Inference KVCache Transfers#

Jamal Hadi Salim, Nabil Bitar, Pedro Tammela, Victor Nogueira

Why: The methodology is the contribution: evaluating AI-fabric transports without owning GPUs, which makes transport comparison accessible to anyone with switches and NICs.

What: A GPU-free emulation approach that reproduces LLM inference KV-cache traffic patterns on real NICs and fabrics, used to compare latency and throughput across existing transports.

How: Model GPU and model-execution behaviour with equations to estimate processing capacity, generate the resulting real-time traffic onto standard network infrastructure, and make the transport pluggable — the point being that different models produce different traffic patterns, so iperf-style generators can't express them. (From abstract only.)

Where applicable: Any lab evaluating fabric or transport options for AI infrastructure without GPU budget; fidelity depends entirely on how well the emulation model matches the real workload.