Skip to content- Sources: primary
- Summary: Meta built Triton Low-Level Extensions (TLX), which lets kernel authors write regular Triton tensor code and add explicit control over warp groups, shared-memory buffers, barriers and asynchronous data movement where they need it. Meta calls this model Multiple Instructions, Multiple Warps. It sits between CUDA's per-thread SIMT model and Triton's block-level model. TLX runs production RecSys training and inference and LLM pretraining at Meta, on NVIDIA and AMD GPUs. Over 10 months, Meta's Triton kernel count grew about 3.8 times while its CUDA kernel count grew about 2 times.
- Why it matters: Kernel authors can reach Blackwell features such as tensor memory and cluster execution from Python, without dropping to CUDA or waiting for the Triton compiler to support them.
- Background: Triton is a Python language for GPU kernels in which the compiler decides how threads and memory are scheduled. Newer NVIDIA GPUs get their speed from warp specialization, where some warp groups only move data with the Tensor Memory Accelerator while others run tensor-core math. Triton's compiler hides that structure, so authors had little control over it.