Skip to content

Agentic Data Flywheel: A Multi-Agent System for Scalable Multimodal Training Data Production#

@Scale: AI & Data 2026 digest Program Wildcard pick 17 June 2026

Abstract

Wendy Jiang, Selahattin Akkas, Alan Li, Antoine Simoulin (Meta)

  • Why: Off the PM's beaten path (data annotation), but a well-instrumented multi-agent design with a surprising result: multilingual bootstrapping turned a per-language staffing bottleneck into a prompt configuration change across 100+ languages.
  • What: A multi-agent annotation pipeline — specialist VLM labeler, parallel quality reviewers with summarizer, self-correcting adjuster, and a thinking-model judge — delivering a 3x efficiency gain with 60%+ of annotations needing no human involvement; remaining cases go to humans prefilled with agent corrections (from abstract only).
  • How: Probabilistic VLM reasoning wrapped in deterministic scaffolding: agent isolation to prevent confirmation bias, deterministic routing around probabilistic judgment, and structured closed-choice output constraints with every failure mode defaulting to human escalation.
  • Where applicable: Demonstrated on image-text labeling with claimed generalization to scene understanding, chart/document VQA, and grounding. The escalation-defaults-to-human principle is the transferable design lesson for any probabilistic-agent pipeline.