Chasing a Moving Target: Going AI-Native on Edits' Video Seek Engine#

Boris Baracaldo (Meta)

• Why: A well-described systems problem (video seek/scrub under continuous interrupting requests and mobile device limits) with a 10x measured improvement, plus a forward-looking "AI-native" refactor — a useful case study in engineering systems so coding and observability agents can extend them.
• What: An engine for serving frames during video scrubbing in Edits: handling a continuous stream of seek requests, abandoning stale work mid-flight, GOP-aware seeking off a keyframe list, finger tracking on backward scrubs while decode runs forward, and precision-for-speed trades where users can't perceive the difference. Delivered 10x improvement in seek latency and frames shown per second. The team is now refactoring toward portability and "agentic just-in-time observability," agent skills for extending the engine, and a vision of agents profiling production data and generating/experimenting with algorithmic variations. (from abstract only)
• How: Request-shaped adaptive behavior rather than any fixed seek policy — each scrub's speed, direction, and composition shape resolves keyframe distance, frame selection, and work-abandonment decisions differently.
• Where applicable: The engine is Edits/Meta-specific and mobile-constrained; the seek mechanics are general video engineering. The agentic layer is largely a stated vision rather than a validated result — treat it as direction, not a substantiated finding.