Skip to content- Sources: primary
- Summary: TIDE targets synchronous, colocated RL post-training, where inference engines idle or decode at small batch sizes while waiting on a few long rollouts. It repurposes rollout GPUs for training while the long tail finishes and migrates remaining rollouts between engines to raise decode batch size, with an adaptive policy deciding when to switch. The authors report end-to-end speedups up to 1.36x over colocated systems and up to 97% of optimal throughput while keeping on-policy semantics.
- Why it matters: Teams running GRPO-style post-training can recover GPU time lost to long-tail rollouts without moving to off-policy asynchronous training.
- Background: In on-policy RL for LLMs, each training step must use samples from the current policy, so the step waits for the slowest rollout in the batch. Asynchronous systems avoid the wait by training on slightly stale samples, which changes the algorithm. Colocated setups share the same GPUs between the inference engine and the trainer and switch between phases.