Skip to content- Sources: primary
- Summary: Meta describes a mixture-of-experts retrieval model with ensembles of user towers and an attention-based gating layer that reaches near ranking-level quality. Model inference exceeds 10 billion ads per second on one GH200, and the production system surfaces candidates at over one billion ads per second, a 100x lift over 2024.
- Why it matters: It shows how fusing embedding gathers, model execution and scoring into one GPU kernel removes memory bandwidth and network bottlenecks at extreme throughput.