A Marathon and Not a Sprint: Lessons Learnt from Trying (and sometimes succeeding!) to Automate Enterprise Campus Networks#

Robert Mertling-Blake, AEG (entertainment venues worldwide)

• Why: An unusually honest "we are not fully automated and here's why" operator story — the failure-mode content the operator-program criteria ask for, with lessons that transfer even to non-campus environments.
• What: AEG's journey automating campus networks across 100+ venues (>800 switches, thousands of APs), including the drivers, failures, and what finally worked.
• How: Key lessons from the slides: assurance matters more than self-healing when the event must work (their golden rule); DCIM cable documentation at remote/unmanned sites dramatically improved remote-hands resolution; "source of record" framing gets buy-in where "source of truth" evangelism doesn't; doing automation perfectly became the enemy of doing any automation (a rushed greenfield deployment without physical connectivity or PNP time); buy the SoR-driven process wins — NetBox-driven config generation, Oxidized backups, monitoring and DDI creation that manual SOPs always missed. They deliberately keep DDI changes non-event-driven while tooling confidence grows, and deploy sacrificial old kit during construction to eat the dust.
• Where applicable: Campus/venue-specific in details, but the meta-lessons (pick fewer battles, alignment with site refreshes, data quality as the persistent constraint) transfer to any heterogeneous multi-site estate.