Sorry we messed up#

Stefan Funke

• Why: A concrete postmortem of a global route leak caused by a race condition in Arista's RCF (Routing Control Functions) agent — a failure mode (automated filter rename during live operation) that could happen on any programmable route-policy framework. The kind of hard-won failure story that transfers across organizations.
• What: How renaming RCF functions (to encode IP version, peer IP, and session type in function names) triggered a global route leak via Arista BUG 1060542. During RCF recompilation, BGP peers referenced undefined function names; update-group merging caused two low-scale peers to reconcile with a high-scale peer's advertised state — leaking all prefixes and causing session flaps via route-limit notifications. Affected a 34-PoP, ~70-device network across 15 countries.
• How: The root cause is a race between the RCF agent compiling new functions and the BGP agent using those names for filtering. Workaround: duplicate the RCF function or map the new one to the old, commit, wait for compilation, then update BGP config to use the new function — or shut peers before renaming. Fixed in EOS 4.31/4.32/4.33 maintenance releases and ≥4.34. The talk covers the full deployment pipeline: Ansible-generated config, session-based commits, checkpoints, commit timers.
• Where applicable: Arista EOS operators using RCF. The broader lesson — programmable filter renames can expose vendor-bug race conditions, and "test before deploy" doesn't catch what only manifests at scale — applies to any operator using automation to modify route policy in production.