Live Demo & Q&A with Together AI#
AI Engineer Europe 2026 digest Program Core pick 8–10 April 2026
Max Ryabinin, VP of Research & Development, Together AI — expo session "Road to 5M Sequence Length: Breaking Memory Barriers in Context Parallelism"
- Why: Long-context training (hundreds of thousands to millions of tokens for video generation and agentic pipelines) hits the memory and compute wall of attention — context parallelism is how training clusters get past it, which is model-training infrastructure directly.
- What: Techniques for finding and optimizing bottlenecks in long-context training, and Untied Ulysses, Together AI's latest method claiming a ~25% boost in achievable training context length over prior approaches.
- How: Context parallelism — partitioning sequence/attention work across devices to break the memory barrier, with the "Untied" variant relaxing a constraint of prior Ulysses-style methods for the additional headroom. Method details beyond this are not in the abstract.
- Where applicable: Training and fine-tuning very-long-context transformers on multi-GPU clusters. The 25% figure is the vendor's own claim, and the 18-minute expo format means breadth over depth — worth a look, not a deep dive.