Rich Packet Metadata - The Saga Continues#

Jakub Sitnicki (Cloudflare)

• Why: The Linux network stack gives packets only 32 bits of sticky metadata (SO_MARK), and everyone is fighting over them. Cloudflare wants 128+ bytes for config consistency across layers, XDP classification, and drop attribution — and this is the unvarnished three-year engineering history, dead ends included.
• What: A progress report on three successive designs: (1) SKB Traits, a per-packet key-value store — abandoned ("don't introduce a second metadata area"); (2) XDP/skb metadata via a new bpf_dynptr_from_skb_meta() kfunc (v6.18), with helpers preserving metadata (v6.15) and metadata tracking decoupled from MAC-header offsets (v6.19) — necessary because L2 tunnels and header pushes corrupt or strand metadata otherwise; (3) the current idea: an skb_ext extension backed by BPF local storage (BPF_MAP_SKB_STORAGE), which lives separately from packet headers.
• How: Approach 3 reuses the existing bpf_local_storage machinery (flexible-array elements, map-backed) attached to sk_buff via the skb extension mechanism. The unsolved problem is memory allocation on the hot path — which is exactly why approaches 1 and 2 were tried first.
• Where applicable: Anyone building multi-layer in-kernel pipelines (XDP → TC → netfilter → socket) that need per-packet context to survive. Watch this if you do XDP classification or drop accounting; the landing kernel version will determine what's expressible.