Great Agents Start with Great Infrastructure#

Akshat Bubna (Modal, CTO)

• Why: The clearest articulation in the program of how agent workloads change the cloud primitives underneath them — sandbox architecture, storage semantics, network policy, and scheduling — which is squarely a cloud-infrastructure PM's problem, illustrated with real scale numbers.
• What: A concrete architecture for agent execution: each agent's sandbox contains multiple containers/microVMs — the harness (model-calling loop) separated from the tool-execution sidecar with different permissions — plus deliberately shared volumes for collaboration (a spreadsheet where every cell is a different agent, per Hebbia). Requirements derived from real workloads: workspaces that persist for days while humans respond (Basis accounting), CPU-then-GPU environment changes mid-task, agent-visible observability (logs, GPU utilization, budget) so the agent can diagnose its own failed runs, and credential injection the agent cannot see or exfiltrate. Domain allowlists are shown to be too broad; the emerging pattern is user-written lightweight proxies enforcing path- and operation-level rules.
• How: Scale figures: 20M snapshots/week, ~8 PB/day processed, 2.6M volumes/week created; one personal-assistant customer needs ~10,000 sandbox creations/second, ~1M concurrent sandboxes, lifetimes of minutes, sub-second interactivity — making startup time the binding constraint. Bubna argues you cannot get all CPU capacity from one provider, so capacity must be aggregated across hyperscalers and neoclouds. The conceptual endpoint: pets → cattle → an "ocean of microbes" of short-lived, snapshot-able, heterogeneous environments that the harness spins up, snapshots, and swaps.
• Where applicable: Modal-specific primitives and scale numbers, but the architectural requirements (harness/hands separation, shared-but-isolated filesystems, per-call network policy, agent-facing observability) transfer to any platform running agents at scale. The "don't vibe-code your sandbox platform" argument (reliability under rare failures, sustained security work, scale) is general.