Redefining Netflix BGP architecture#

Fred Cuiller — Cisco; Elise Vennegues — Netflix

• Why: A large, real BGP architecture migration with the validation methodology, defect discovery, and phased cutover shown — the parts that usually get omitted. Netflix's edge is 400+ PoPs, 19,000+ OCAs, ~175 countries, 20,000+ eBGP sessions to 2,800+ ASNs; moving that off iBGP full-mesh is a serious operational story.
• What: Replacement of iBGP full mesh between Edge Switches and Provider Edge routers with a route-reflector hierarchy built on Cisco IOS XRv9000 UCS M7 appliances (chosen over traditional routers and VNFs on integrated TPM, CoPP, operational simplicity). RRs placed in six global sites; production scale runs to 37M IPv4 paths / 300+ clients in North America, 20M paths / 250+ clients in EMEA. Interesting appliance-selection comparison table (router vs VNF vs appliance across scale, CoPP, security, cost).
• How: Early Field Trial with the vendor caught real defects before deployment (early-release memory leak, BGP process rlimit misassignment, duplicated LLDP entries on Mellanox NICs); one miss escaped lab — telemetry YANG path performance at scale with sub-60s polling, discovered only in pre-production, reinforcing that production churn/scale is unique. Migration: phased per-device (staging RR peering with pilot prefixes → release prefixes to RR → per-device depref-then-pref-up → remove mesh), driven by internal tooling with a per-device migration tag; telemetry dashboards track CPU/memory, session counts, and received paths per address family. BMP activation is the stated next step.
• Where applicable: The Cisco appliance choice is Netflix/Cisco-specific, but the RR placement reasoning, EFT-style validation discipline, phased-migration mechanics, and telemetry checklist transfer to any iBGP-mesh-to-RR program at hundreds-of-PoPs scale.