• Sources: primary, discussion
  • Summary: Apple ML Research benchmarked MLX LM on M5 Ultra and M3 Ultra Mac Studios, reporting up to 4.6 times faster time to first token and 1.6 times faster decode on M5 Ultra. Muse Spark Open in NVFP4 needs about 493GB for weights, so it runs across machines using the jaccl backend over RDMA on Thunderbolt 5, reaching 36.73 tokens per second with four-way tensor parallelism at 8,192 tokens of context. The post includes mlx.launch and mlx_lm.server commands for serving the model to local coding agents.
  • Why it matters: Running a frontier-size open-weight model locally for agentic coding now has a documented recipe and measured numbers on Apple hardware.