ESnet's new Micro-Service based Discovery Service#

Marc Koerner — ESnet

• Why: ESnet operates a national science WAN with optical line systems — the closest thing in this program to the PM's DC/WAN interconnect and optics interests. This is a production (since June 2025) redesign of how bulk network data is collected and normalized, born from real operational pain.
• What: A Kafka-based, notification-driven discovery and document-repository architecture replacing per-application polling. Previously, multiple tools independently queried devices for overlapping purposes (optical circuit sync with the config DB, L2/L3 topology for the OSCARS L2VPN tool), causing redundant collection, inconsistent data across apps, and unnecessary device/network load. The Discovery Service centralizes this into one low-friction mechanism with pluggable normalizers.
• How: Agent Controller (scheduler, dedup of discovery requests) distributes work via Kafka to Discovery Agents, which pull data from sources — NETCONF from devices, RESTCONF from the TNMS management system, or even spreadsheets. Documents (up to 80MB, compressed) land in the "Andy Lake Service" repository with metadata and history; stacked/parallel normalizers (implementing the TOWL equipment ontology) generate normalized views and compute change notifications. Fine-grained controls: configurable audit intervals, namespace-level gets, regex filters to strip counters and sensitive data before storage. Deployed as microservices on Kubernetes (two prod clusters plus staging) with GitLab/Argo CD/Rancher.
• Where applicable: ESnet-specific stack (Kafka, TOWL, k8s), but the design pattern — notification-driven bulk data management with normalized views — generalizes to any multi-tool operator drowning in redundant polling. The honesty about equipment is valuable: vendor NETCONF/YANG standard violations, devices that lock up under discovery, models so large the device times out, and NSO being slow at non-config retrieval. Budget for these if you attempt the same.