Home / Granite Guardian
Granite Guardian
An open judge model for LLM agents. Give it a conversation and a criterion; it returns a verdict and a probability. I lead the team and own its releases, from data to open weights: five versions in eighteen months, shipped in watsonx and Red Hat’s Ask Red Hat.
Independent evaluation
On the leaderboards, and live
Releases
Five versions in eighteen months
- Oct 20243.0 Harm and RAG-hallucination detection, at 2B and 8B
- Dec 20243.1 Function-call hallucination detection for agents
- Feb 20253.2 Smaller 5B and 3B variants
- 20253.3 Hybrid thinking: a reasoning trace or a direct verdict
- Apr 20264.1 User-written criteria, and use as a reward model
What 4.1 added over 3.3
Balanced accuracy, 8B, no-think mode. Source: the 4.1 model card.
- Guardian 3.3
- Guardian 4.1
- IFEval multi-constraint0.4580.844
- InfoBench, human-annotated0.5350.706
- InfoBench, GPT-4-annotated0.5850.726
- FC Reward Bench0.740.79
- LLM-AggreFact average0.761 → 0.760
70.3
Best-of-N score on the verifiable JETTS tasks as a reward model, ahead of Skywork-Reward 27B (68.7) and SFR-Judge 70B (68.2).
0.57 → 0.84
IFEval balanced accuracy: the base Granite 4.1 8B prompted as a judge, then post-trained as Guardian.
Design
Three decisions
- 01
A judge, not a classifier
It answers with a yes/no token and its probability, so the whole language model stays in play. That is what later let it take new criteria and serve as a reward model.
- 02
Criteria at runtime
Since 4.1 the user writes the rule. Out-of-distribution safety F1 slipped from 0.81 to 0.79; custom-criteria accuracy on IFEval went from 0.46 to 0.84.
- 03
Reasoning from the same weights
Think mode writes a justification first. It costs 59 ms to 1.3 s and pays back in tuning custom criteria and debugging verdicts.
Adoption
In products, open source and other people’s studies
“Granite-Guardian-3.2-5B showed the best generalization with only a 6.5% gap.”
“IBM’s open-source safety models provide harm detection, jailbreak detection, topic control, hallucination detection, and RAG quality assessment, capabilities that other guard providers don’t yet offer.”
Hugging Face downloads by Guardian version
All time, summed over the model sizes in each release. Read October 13, 2026.
- 3.0 Oct 2024331K
- 3.1 Dec 2024122K
- 3.2 Feb 2025150K
- 3.3 2025290K
- 4.1 Apr 2026213K
Independent benchmarks
- LLM-AggreFact: Granite Guardian 3.3 (8B) ranked third of 39 in 2025; with 47 models listed today, it is still ahead of GPT-4o and Llama 3.1 405B.
- GuardBench (EC Joint Research Centre): six of the top ten places across 40 datasets, April 2025.
- Artificial Analysis, with NVIDIA: fifth of 20, first among policy-at-runtime models, June 2026.
In products
- watsonx.governance: “The AI guardrails now use a Granite Guardian model as a filter.”
- watsonx.ai: the AI guardrails filter for prompts, and a hosted foundation model.
- Ask Red Hat, on the Red Hat Customer Portal: “Guardrails consists of IBM granite-guardian-3.3-8b.”
- Traefik Hub: an LLM Guard built on Granite Guardian, next to guards from Microsoft and NVIDIA.
In open-source projects and tutorials
The same post-training recipe adds capabilities to Granite models, composed by Granite Switch.