ASEA: Autonomous Agents for ML Serving Efficiency at Scale#

Bikash Sharma, Aashik Gowda, Angineh Keshishian, Mia Gu, Michael Fulthorp, Jinfu Leng (Meta)

• Why: "ML for infrastructure operations" is a core interest, and this is the most complete production example in the program: an agentic platform running the full optimization lifecycle over a serving fleet drawing hundreds of megawatts, with validated dollar savings.
• What: ASEA turns a multi-day, expert-only optimization workflow (stitching telemetry, profiling, and capacity data) into an autonomous fleet-wide loop, claiming tens of millions of dollars in validated efficiency savings; the motivating observation is that service overhead often reaches 50% of total serving footprint (from abstract only).
• How: Dozens of specialized single-skill LLM agents (capacity sizing, autoscaling, batching, routing, hardware migration, regression detection) share a unified fleet-wide data foundation and orchestrate discovery → root-cause diagnosis → configuration generation → canary validation → rollout.
• Where applicable: Meta-specific serving fleet (CPUs, GPUs, and Meta's MTIA ASICs) and the savings figures should be treated as operator-specific. The architecture — single-skill agents over a shared telemetry/profiling data plane with canary-gated rollout — is a transferable pattern for any large serving or infrastructure estate.