MLCommons Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces#

Srinivas, Andrey Balogh, Brad B, et al. (MLCommons-led effort with industry participation from NVIDIA, AMD, Meta, Keysight, HPE, and others, per the paper)

• Why: There is no portable, vendor-neutral way to characterize distributed AI workloads' behavior on a fabric. Chakra's standardized execution-trace format is becoming the interchange layer for AI-cluster benchmarking and co-design — effectively the telemetry standard to watch.
• What: An open ecosystem whose core is Chakra Execution Traces (ETs): graph-based representations of compute, memory, and communication operations with data/control dependencies, timing, and resource constraints, plus tools for collection, analysis, generation, and replay into simulators and emulators. Includes analysis of traces from production AI clusters and real-world case studies; adopted by MLCommons (from abstract only).
• How: Workload traces are captured as interoperable graphs, then fed to simulators/emulators/replay tools for performance prediction and SW–HW co-design without needing the original cluster.
• Where applicable: Cross-vendor by construction — the whole point is portability across NVIDIA, AMD, and other stacks. Adoption is still growing; value scales with ecosystem uptake.