Skip to content- Sources: primary, discussion
- Summary: Dr Magnus Lynch tested GPT-6 Astra, Claude Fable 5.1 and an Inception v3 fine-tuned on 68,011 ISIC images against 200 MILK10k lesions, with the test lesions removed from training. The CNN named the correct diagnosis for 125 lesions against 84 and 85 for the LLMs, though at its preset threshold it discharged 18 of 30 melanomas. Cases, prompts, replies and scoring code are published, and each LLM case was run once.
- Why it matters: It is a reproducible data point that small task-specific vision models can still beat general multimodal LLMs on narrow clinical image classification.