A Lightweight High-Throughput Collective-Capable NoC for Large-Scale ML Accelerators#

Luca Colagrande, Lorenzo Leone, Chen Wu, Tim Fischer, Raphael Roth, Luca Benini (affiliations not listed in the program)

• Why: With thousands of compute elements per die, on-chip fabrics now face the same problems as DC fabrics — collective offload, multicast, in-network reduction, buffer/area budgets. This is fabric design and in-network compute at the silicon level, and the boundary between NoC and scale-up interconnect is exactly where the field is heading.
• What: A lightweight NoC supporting barrier synchronization, scalable high-bandwidth multicast, and in-network reduction, co-designed for next-generation ML accelerators. Introduces Direct Compute Access (DCA), giving the interconnect direct access to the cores' computational resources for in-network reduction at a 16.5% router area overhead. Measured 2.9× and 2.5× geomean speedups on multicast and reduction (1–32 KiB), and up to 2.1× estimated end-to-end GEMM performance vs a unicast-only NoC baseline (from abstract only).
• How: DCA lets the fabric itself perform reductions using core compute resources, keeping communication off the critical path in GEMM-heavy workloads.
• Where applicable: Applies to accelerator ASIC design, not to retrofitting existing clusters — and the end-to-end GEMM gains are estimates from architectural simulation rather than silicon measurements. The in-network-reduction tradeoff (area vs throughput), however, is a broadly transferable design lesson.