Fast Notification for tunnel-based lossless RDMA transmission in WAN#

Zehua Hu, Yongqing Zhu, Jiayuan Hu, Tanxin Pi (China Telecom)

• Why: Cross-AIDC LLM training and inference push RDMA through SRv6/GRE/VXLAN/MPLS tunnels over the WAN; a small number of losses collapses effective throughput, and neither ECN (end-to-end, long RTTs) nor PFC (link-local, DC-only) works there.
• What: A concrete Fast Network Notification (FANN) protocol: transit (P) nodes detecting failure, congestion, or state change send structured notifications to the ingress PE, which reroutes to backup paths, rebalances across available paths, and/or throttles the affected traffic and notifies the sender and controller.
• How: Defines a new ICMPv6 message — Code distinguishes event vs. recovery, a Hop Limit bounds propagation scope, Event Type covers link failure, congestion, performance degradation, microburst, signal degradation, and queue buildup (with sub-types, e.g., congestion severity tiers), a microsecond timestamp, the originating node's IPv6 address, and a 32-bit bitmap selecting fixed-length metadata fields (ingress/egress port IDs, link utilization, loss, latency, jitter, queue and buffer occupancy, flow 5-tuple, and a path ID carried as an SRv6 SID list or MPLS label stack). An identical structure can ride over UDP on a well-known port where ICMPv6 is filtered. Security: RFC 4443 rate limiting, length validation, and origination only from control-plane interfaces.
• Where applicable: Carrier inter-DC WANs carrying tunnelled RDMA (China Telecom is the driving operator — this is operator-specific in origin, general in technique). Individual draft at -03; the protocol encoding is draft-stage and the FANN umbrella has competing notification proposals (ARN etc.) it must converge with.