One Layer Deeper: The 10-Layer Cake Under a Linux Network Interface#

Jesse Brandeburg (Cloudflare)

Why: The most useful non-feature talk of the program: a production-performance mental model from someone who wrote Intel Ethernet drivers and now runs performance at Cloudflare, framed around the observability gap rather than a knob list.

What: A tour of the ten layers hidden under the "interface, socket, queue" abstraction — NIC hardware, drivers, CPU placement, packet hooks, traffic control, firewalls, socket buffers, application runtimes, microarchitecture, observability — and how to reason about which one is spending your budget.

How: The framing is that the old playbook ("turn everything off and go fast") no longer applies, because firewalls, observability and multi-tenant isolation are production requirements. Packets can vanish at the NIC, XDP, tc, conntrack, qdisc, softnet backlog, socket receive queue or the application runtime queue — and a service can honestly report it processed everything it received while the kernel drops packets it never saw. Concrete examples: why sendmmsg()/recvmmsg() still matter, why treating the socket receive queue as a warehouse causes loss and latency, where UDP loses segmentation offload behaviour, where VLAN acceleration hides metadata userspace needs, and the real cost of stacked XDP/tc/netfilter/conntrack/BPF hooks. Ends below the packet path at cache efficiency, code size and TLB misses. (From abstract only.)

Where applicable: Universal. Cloudflare's 31.4 Tbps attack-mitigation scale is context, not a prerequisite.