Skip to content- Sources: primary
- Summary: Together Compiler runs a continual agent hill-climbing loop over end-to-end inference, using a Tiered Persistent Engine to separate engine launch overhead from forward pass timing and a cost model to pick starting parallelism configurations. The paper reports up to 1.7 times over hand-written megakernels on Llama 1B and up to 2.6 times the tokens per second of the best open engines on Kimi K3, GLM 5.3, MiniMax M3.1 Flash and DeepSeek v4.1 Flash on InferenceX.
- Why it matters: It shows agent-driven optimization working against whole-engine measurements in production, not only on isolated kernels with offline benchmarks.
- Background: Agent kernel optimizers usually time single kernels offline, which misleads for data-dependent operations such as mixture-of-experts routing. Optimizing a full engine is more faithful but each engine restart is slow. TKCC targets that tradeoff between measurement throughput and fidelity.