@Scale: Systems & Reliability 2026#

Core#

The Agentic Infrastructure Gap: In-Distribution Languages Make It a Coding Problem#

Abstract Recording

Joe Duffy (Pulumi)

• Why: A sharp thesis on how agents should manage infrastructure: LLMs are trained on billions of lines of Python/TypeScript/Go and almost nothing on your bespoke DSLs, consoles, and runbooks. If agents are going to run infra, the infra's interface needs to live in the distribution the model was trained on. Directly relevant to how network automation and IaC platforms should be shaped for agent consumption.
• What: The argument that "assembling IKEA furniture" — consoles, API keys, copy-pasted config — is the agent blocker, and that modeling infrastructure in real programming languages reduces agent management to a coding problem, given an oracle that maps code changes back to infrastructure outcomes. Includes learnings from Pulumi's work with frontier AI companies. (from abstract only)
• How: Infrastructure expressed as code in mainstream languages (Pulumi's model), plus a preview/plan-style oracle that predicts infrastructure outcomes from code diffs, closing the agent's feedback loop.
• Where applicable: General-purpose for IaC and cloud provisioning; expect vendor framing (Pulumi CEO) but the in-distribution-language argument applies to any automation surface, including network config generation.

Extending Meta to the Public Cloud#

Abstract Recording

Dinesh Govindasamy (Meta), Sargun Dhillon (Meta)

• Why: A hyperscaler with one of the world's largest private DC footprints deliberately adopting public cloud — for GPU capacity — is a rare, high-signal data point on hybrid strategy and capacity optionality. Sargun Dhillon (long-time Meta infra) presenting adds credibility that this is engineering substance, not a partnership announcement.
• What: How Meta integrated public cloud into its overall infrastructure footprint: initial forays driven by GPU compute demand, multi-cloud used explicitly as optionality ("capacity hyperliquidity"), and full integration into Meta's platform, OS, and fleet management stack so internal customers see cloud capacity as ubiquitous rather than a separate silo. (from abstract only)
• How: Not detailed in the abstract; the interesting engineering is presumably in extending Meta's host OS, provisioning, and fleet management to cloud instances so tooling parity holds across estates.
• Where applicable: Meta-specific (their internal stack is not reproducible), but the strategic and integration lessons — treating cloud as a fungible capacity pool behind a uniform fleet-management abstraction — transfer to any large hybrid operator.

Experience Productionizing and Operating GB200 clusters#

Recording

Ankur Srivastava (NVIDIA), Sachin Kumar Lakharia (NVIDIA), Douglas Wightman (NVIDIA)

• Why: Rack-scale NVLink systems (GB200 NVL72: 72 GPUs with an exascale-scale NVLink fabric in one rack) change how clusters must be scheduled and operated — fabric locality becomes the dominant scheduling constraint. First-party operational experience from NVIDIA on bringing up and running a large cluster is hard to get elsewhere.
• What: Production experience operating large GB200 clusters: topology-aware scheduling in Slurm to maximize NVLink fabric utilization, GPU-occupancy and scheduling recommendations validated with a Slurm simulator, and the unglamorous bring-up work — health checks, rack testing, slow-node identification, supporting large-scale training runs. Cites >2.6x MLPerf training improvement attributable to the NVL72 system. (from abstract only)
• How: Topology-aware Slurm scheduling (placing jobs to exploit intra-rack NVLink bandwidth) plus simulation-driven occupancy policy; operational practices for node/rack validation before admitting capacity to the pool.
• Where applicable: Tied to NVIDIA GB200 NVL72 silicon and Slurm specifically — the concrete recommendations don't port directly to non-NVL systems or Kubernetes schedulers. The method (topology as a first-class scheduling input, simulator-tuned occupancy, systematic slow-node quarantine) generalizes to any tightly-coupled accelerator fabric.

AI Storage Blueprint#

Abstract Recording

Sidharth Bajaj (Meta), Venkatraghavan Srinivasan (Meta)

• Why: In training clusters, storage is frequently what gates GPU utilization (checkpoint I/O, dataset streaming, data-loading jitter). Meta re-architecting its storage stack around AI workloads is the storage-side counterpart of the GB200 scheduling talk.
• What: How Meta evolved its storage architecture to serve an environment where frontier models ship in weeks, framed around two explicit optimization targets: GPU utilization and research velocity. (from abstract only)
• How: Not detailed in the abstract; the two stated hurdles imply work on throughput/latency consistency for training data paths and on flexibility/speed of the research workflow.
• Where applicable: Meta-specific production setup; the framing (storage judged by GPU-hours protected and researcher iteration speed) is a useful lens for any AI-cluster storage design even before the details land.

Adjacent#

Securing Production Debugging at Hyperscale#

Recording

Shridivya Sharma (Microsoft), Luke Kelly (Microsoft)

• Why: The "how do humans (and now agents) safely touch production boxes" problem, worked at Microsoft's K8s fleet scale. Relevant to anyone operating Kubernetes estates where break-glass access is the default risk story.
• What: Microsoft's approach to production debugging across a global fleet of Kubernetes clusters: SSH-based access integrated with just-in-time privilege elevation and task-based (rather than open-shell) execution, kept secure and auditable — then those same building blocks exposed to AI (Semantic Kernel) so on-call engineers get assisted resolution rather than raw shell. (from abstract only)
• How: Structured, auditable debugging primitives (JIT elevation + task-scoped SSH commands) that double as a safe action surface for AI assistance — the same "give agents constrained, task-shaped interfaces" theme running through the Meta talks.
• Where applicable: Microsoft's fleet, but the pattern is general for large K8s estates; the AI-assist portion is early-stage and Microsoft-stack-specific (Semantic Kernel).

Wildcard#

Building Privacy Aware Infrastructure in the AI-Native Era#

Recording

Rituraj Kirti (Meta)

• Why: Not about networking or AI infra per se — but it presents the most transferable AI systems architecture in the program: how to run an LLM-driven control loop in production while converging it to deterministic, auditable, near-zero-LLM operation. That pattern applies far beyond privacy, e.g., to policy enforcement and classification over network/infra assets.
• What: A case study in AI-native asset classification at the scale of millions of data assets: a hybrid deterministic-rules engine with LLM fallback that converts messy context (metadata, lineage, code references, scan signals) into enforceable privacy controls — with ground-truth + eval self-improvement loops and a distillation step that exports high-performing LLM behavior into an auditable rules.yaml, driving LLM usage toward near-zero while improving determinism, latency, and cost. (from abstract only)
• How: Active learning focuses human review on high-value examples; LLM-as-Judge provides scalable review signal (with kappa reliability reported); evaluation gates prevent regressions and circular reinforcement; the converged behavior is distilled into declarative rules with drift monitoring in production.
• Where applicable: The privacy use case is Meta-specific, but the loop design — LLM with fallback, eval gates, distillation to declarative rules, drift monitoring — is a general recipe for any "LLM in the infra control path" system where you eventually need auditability and predictable latency/cost.

Themes#

• Agents move from writing code to operating systems. Nearly every talk puts an agent on the production side — debugging GPU failures (Pariag), investigating incidents (Mitra), root-causing NCCL hangs (Liu), assisting on-call (Microsoft). The field's center of attention is no longer code generation; it is safe agentic action on live infrastructure.
• The trust boundary is the hard problem. Across Meta and Microsoft, the shared answer is the same shape: constrained, task-scoped, auditable action surfaces (guardrails, JIT elevation, task-based execution, supervised mitigation of low-risk reversible actions) rather than autonomous shells.
• GPU capacity is the forcing function. Meta going to public cloud for GPU compute, storage re-architected around GPU utilization, and GB200 operationalization all trace to the same pressure — and multi-cloud shows up explicitly as capacity optionality rather than ideology.
• Distilling AI behavior into deterministic artifacts. Two talks independently converge on exporting what the model learned into auditable, non-AI form: rules.yaml in the privacy talk, decision trees/runbooks in the NCCL talk. LLMs as a generation mechanism for deterministic policy, not a runtime dependency.
• Topology-aware scheduling is now an operational discipline. The GB200 talk treats fabric topology as the primary scheduling input, validated via simulation — a preview of how scale-up/scale-out asymmetry will dominate cluster orchestration generally.
• Scarcity caveat: this program publishes abstracts only, no papers or slides, and several sessions (keynotes, Q&A, panel) have no technical abstract at all. Depth here must come from the recordings and the companion articles on the @Scale site.