TileLink: Generating Efficient Compute-Communication Overlapping Kernels using Tile-Centric Primitives#

Size Zheng, Jin Fang, Xuegui Zheng, Qi Hou, Wenlei Bao, Ningxin Zheng, Ziheng Jiang, Dongyang Wang, Jianxi Ye, Haibin Lin, Li-Wen Chang, Xin Liu

• Why: Overlapping intra-layer communication with compute is where GPU-cluster performance now lives, and today it's hand-written by kernel experts. This is a compiler approach to that problem — relevant background for anyone whose fabric teams collaborate with ML-infra teams on collective performance.
• What: A kernel-generation system that composes communication and compute into overlapped kernels via "tile-centric primitives": the frontend decouples the communication and computation design spaces and links them via tiles; the backend lowers primitives into low-level communication instructions for fused, overlapped execution. 1.17×–20.76× speedup over non-overlapping baselines, performance comparable to state-of-the-art hand-tuned overlapping libraries. (from abstract only)
• How: Tile-level abstraction — computation and communication are each expressed over tiles, and the compiler schedules them to overlap, avoiding the suboptimality of operator decomposition and the expertise barrier of manual kernel fusion.
• Where applicable: General technique for distributed intra-layer parallel operators on GPUs; the very wide 1.17–20.76× range says gains depend heavily on the operator and topology. Evaluated on GPUs; the communication backend targets standard collectives, so it complements rather than replaces whatever interconnect you run.