Building AI That Works: ESnet's Pragmatic Approach to AI-Driven Operational Excellence#

Edward Balas, Jon-Paul Herron (ESnet)

• Why: One of the few AIOps talks with deployed results rather than demos. ESnet built ORBIT (Operations Response & Business Intelligence Toolkit) to address growing scale with fixed NOC staffing, and reports what actually worked — including the unglamorous lessons.
• What: Two production applications: Intelligent Resolution Guidance, which proactively surfaces relevant documentation for incoming alarms/tickets, and Adaptive Incident Summarization, which generates audience-tailored ticket summaries. A concrete example in the slides: an operator asks what to do about a "Span Loss Out of Range – High" alarm and gets guided resolution. Future roadmap items include Drishti (early-alpha unsupervised log analysis: identify interesting events, cluster, generalize into behavioral sequences, summarize) and off-the-shelf Elasticsearch anomaly detection — multivariate for data movement (limited dimensionality, weak explainability) and univariate for predicting resource exhaustion ("preserves our engineers' weekends").
• How: Unified data access fusing ticketing (ServiceNow), Slack chatter (context that never made it into tickets), documentation, live metrics, and logs, with LLM agents and semantic search over semi-structured data via MCP. A representative query — "customers in Chicago report unstable service, summarize what might be responsible" — has the agent walk the customer database, backbone topology, ticket history, metrics, and operations chats before synthesizing an answer. Humans stay in the loop throughout.
• Where applicable: ESnet-scale research WAN, but the architecture (chat + tickets + docs + telemetry into one agent environment, semantic search/MCP over ops data) is general. Key lessons are stated plainly: treat prompts as code (version, QA — 25% speedup from refinement alone), plan for stochastic outputs, your first use cases won't be your strongest, and develop organizational policies for AI data access before capability sprawl.