Wafer reports GLM 5.2 serving on AMD MI355X at lower cost than Blackwell

Category: AI
Status: discussion
Sources: Wafer benchmark, HN discussion
Summary: Wafer published its own benchmark of the open-weight GLM 5.2 model on AMD MI355X hardware (TensorWave capacity), reporting 2626 tokens per second per node at 2.4 requests per second on a 20k-in/1k-out workload with 60 percent cache hits, and 213 tokens per second single-stream on 10k-in/1.5k-out following Artificial Analysis standards. It uses SGLang with MXFP4 quantization via AMD Quark, TP4xDP2 parallelism, and custom kernel tuning plus speculative decoding. Wafer states single-node performance reached about 80 percent of a B200 while the hardware costs roughly 2.75 times less than NVIDIA Blackwell.
Why it matters: Vendor-reported cost-per-throughput on AMD for a top open-weight model is another data point in whether AMD inference is becoming a viable alternative to NVIDIA for serving, though the numbers are the provider's own and not independently reproduced.
Follow-up: Track independent reproduction of the MI355X throughput and cost figures and whether the SGLang/Quark path stabilizes for GLM 5.2.