Cactus releases Needle 2, a 45M-parameter tool-calling model in a 14MB binary under Apache 2.0

Sources: primary, discussion
Summary: Cactus states Needle 2 is 45M parameters compressed to CQ2-bit, shipping as a single 14MB binary with a 28MB session RAM ceiling from a 256-token sliding attention window, which puts it on ESP32-S3 and Cortex-M class parts, with weights on Hugging Face under Apache 2.0. Reported speeds are 500 tokens per second decode on a Raspberry Pi 5 and 300 to 700 on sub-200-dollar phones. The evaluation names five public benchmarks (Google Mobile Actions, DroidCall, Seal-Tools in-domain and out-of-domain, BFCL v4 single-turn), scores them as ordered strict exact match, and states Needle numbers are measured end-to-end through the shipped engine at deployment precision while baselines run at f16 under vLLM.
Why it matters: Tool calling at this footprint puts the routing decision on microcontroller-class hardware, though the numbers are vendor-run and not independently reproduced.
Follow-up: Whether an independent party reproduces the benchmark scores or the on-device throughput.