Shipping like a startup, regulated like a bank#

David Abitbol (Revolut, Ops & Infrastructure Leader)

• Why: One of the few talks with a complete self-hosted inference deployment story: real GPU counts, real tuning results, real unit economics, and the compliance wrapper — a template for private-cloud AI operations.
• What: Revolut moved ~2,000 engineers from budget-limited Claude Code usage (engineers hoarding tokens for hard tasks) to self-hosted open models (GLM 5.x) on dedicated GPUs. Reported outcomes: the switch to GLM 5.1 coincided with ~60% better coding time and ~70% better lead time (their internal chart, not a controlled study — the improvement is not isolated from other causes); blended cost ~$0.30 per million tokens, roughly one-tenth of the prior Anthropic/OpenAI invoice. The key tuning result: on a fixed fleet of 56 B200s, requests per second rose 76% while time-to-first-token fell 88% (14.26 s during migration → 1.68 s stable) — i.e., tune before you buy more GPUs. Sub-3 s TTFT is their empirical usability threshold; ~200 tok/s is fine because engineers now run parallel tasks and return when done.
• How: The parameter playbook: size the context window to actual workload (~256K, not the advertised 1M — "long context is a scam" for capacity purposes), FP4 quantization on Blackwell (~40% smaller footprint vs FP8, claimed zero precision loss), tensor-parallel degree chosen for KV-cache headroom (TP8 over TP4), cache hit rate above 90% for coding workloads, speculative decoding, prefill/decode disaggregation; OpenAI-compatible endpoints so any harness works; a local router sends image tasks to a different model rather than degrading the text model. Controls: sandbox rules that make destructive operations require human intervention (allow grep, not gradle), zero-retention GPU providers in data-equivalency jurisdictions, and strong OpenTelemetry for incident investigation.
• Where applicable: Revolut-specific deployment and economics (2,000 engineers, EU-regulated bank, one model family); the tuning methodology and the "resist scaling until tuned" lesson generalize to any self-hosted inference operation. The compliance claims are their circumstances, not a general regulatory determination.