• Sources: primary, discussion
  • Summary: On the prompt-injection benchmark a 200M-parameter pre-trained classifier, deberta-v3-base-prompt-injection-v2, scored 89.01% at 54.1 ms median latency, against Jev-1.13.0 at 86.35% and 348.1 ms and Qwen3.6-35B used as LLM-as-a-judge at 89.31% and 312.5 ms, with the 2019 BART-large-mnli zero-shot baseline last at 61.49%. The method is disclosed in full: nine guardrails across four paradigms, the EvalHub NeMo Guardrails prompt-injection and toxicity-profanity-safety benchmarks, class-balanced datasets, named hardware, and the risk-policy prompts. Two caveats a reader needs: Red Hat ships those classifiers as the default guardrail catalog in OpenShift AI 3.6 and used them as its own gold standard, and the authors state their risk definitions were adapted from LLM-as-a-judge prompts and may not be optimal for zero-shot models.
  • Why it matters: A small task-specific classifier holding its own against a decision model and an LLM judge argues against paying generative latency on every guardrail check.