Prasanna Sattigeri

Home / Research

Research

Post-training that teaches a base model to check its own work, the generative modeling before it, and the papers behind both. Granite Guardian has its own page; the full paper list is under Publications.

Impact

Team, releases and recognition

I lead the Granite Guardian team, set its direction and own its releases, from data to open weights. Five versions in eighteen months, each with new capabilities and improvements.

  1. Oct 20243.0 Harm and RAG-hallucination detection, at 2B and 8B
  2. Dec 20243.1 Function-call hallucination detection for agents
  3. Feb 20253.2 Smaller 5B and 3B variants
  4. 20253.3 Hybrid thinking: a reasoning trace or a direct verdict
  5. Apr 20264.1 User-written criteria, and use as a reward model

Recognition

  1. 2025IBM Research Outstanding Accomplishment. AI governance innovations to watsonx.governance and open source
  2. 2025IBM Research Technical Achievement Award. Granite Guardian
  3. 2024IBM Research Outstanding Accomplishment. Granite 3.0 release
  4. 2024IBM Research Technical Achievement Award. Causal inference for business decision making
  5. 2020–21Four IBM Research Technical Achievement Awards. Uncertainty quantification, robust and generalizable AI, learning with less labels, efficient AI
  6. 2019IBM Outstanding Technical Achievement Award. Contributions to trustworthy AI

Joint projects

Principal investigator on two MIT-IBM Watson AI Lab projects, trustworthy learning with limited data with Gregory Wornell and human-centric AI for shared decisions with David Sontag, and on joint work with RPI on causal inference and agent safety with Ali Tajer and Mohammad Mohammadi Amiri. The MIT projects produced Thermometer, the EDL paper, the COLM paper, and the learning-to-defer and onboarding papers.

Mentoring

Thesis committees for Joshua Lee and Maohao Shen at MIT and Burak Varıcı at RPI. Interns most years for a decade; two of the 2026 papers below started as internships.

Post-training

Teaching a base model to check its own work

Post-training, as I use the word, is everything that happens to a pretrained model before it can be trusted: judges, calibration, capabilities trained into the base model one at a time, and the memory and guardrails an agent runs with. The recipe is the same each time.

  1. 1Define

    One output you can check: a verdict, a score, a ranking, a rewrite.

  2. 2Build data

    Human labels where judgment is needed. Synthetic data where scale is. Red-teaming for the rest.

  3. 3Train

    A full fine-tune for the judge; capabilities trained into the base model, one at a time, for the libraries.

  4. 4Measure

    Held-out sets, and the same base model prompted, so the gain is visible.

  5. 5Release

    Open weights, Apache 2.0, signed.

The gain over the same model, prompted

Requirement checking

IFEval multi-constraint, balanced accuracy. Chance is 0.5.

  • Base model, prompted
  • Post-trained
  • Granite 4.1 3B0.5120.843
  • Granite 4.1 8B0.5690.855
  • Granite 4.1 30B0.6240.849
Held-out IFEval set. Source: Requirement Check model card.

Calibrated uncertainty

MMLU, AUROC for telling right answers from wrong. Chance is 0.5.

  • Base model, sequence probability
  • Post-trained
  • Granite 4.0 micro0.675 → 0.690
  • Granite 4.1 3B0.6290.771
  • Granite 4.1 8B0.6440.778
  • Granite 4.1 30B0.6510.802
14,042 MMLU questions, not used in training. Calibration error: 0.161 → 0.057 (micro), 0.288 → 0.099 (3B), 0.112 → 0.038 (8B), 0.092 → 0.100 (30B). Source: Uncertainty model card.

Where each check runs

  1. 01

    A request arrives

    Is it safe and in policy?

    • Harm and jailbreak screeningGuardian
    • Policy compliance, with an “ambiguous” verdictGuardian Library
  2. 02

    The agent calls a tool

    Is the call valid for these tools and this request?

    • Function-call hallucinationGuardian
  3. 03

    Context comes back

    Is the retrieved material relevant?

    • Context relevanceGuardian
  4. 04

    The model drafts an answer

    Is it grounded, and does it meet the requirements?

    • Groundedness and answer relevanceGuardian
    • Factuality detection against conflicting contextGuardian Library
    • Requirement checkCore Library
  5. 05

    The draft is repaired

    Can a failing draft be fixed or replaced?

    • Factuality correctionGuardian Library
    • Best-of-N selection, Guardian as reward modelGuardian
  6. 06

    The answer goes out

    How sure is the model, and what is it based on?

    • Calibrated uncertainty scoreCore Library
    • Context attribution, sentence by sentenceCore Library
Guardian
Granite Guardian 4.1, or its criteria trained onto a base Granite model
Guardian Library
Policy compliance, factuality detection and factuality correction
Core Library
Requirement checking, calibrated uncertainty and context attribution

The seven capabilities

CapabilityReturnsTrained on
Judging against a criterionyes or no, with a risk scorehuman-labeled and synthetic data for safety, RAG and tool calls
Policy compliancecompliant, non-compliant or ambiguoushuman-written policies across about 40 domains
Factuality detectionyes if any claim is unsupported by the context17,522 ELI5 instances, half with injected errors
Factuality correctiona corrected response, or nonethe same corpus, supervised by FactCorrector (ACL 2026)
Requirement checkingyes or no, with P(yes)multi-constraint prompts with programmatic labels
Calibrated uncertaintya confidence that is right about X% of the time at X%240K answers per base model, labeled by a Thermometer calibrator
Context attributionthe context sentences behind each response sentence, ranked4,101 conversation rounds labeled with leave-one-out MExGen

Released as the Guardian and Core capability libraries for granite-4.0-micro and granite-4.1 at 3B, 8B and 30B, so each capability can be added, swapped or upgraded on its own. Two of the seven are papers from this page, distilled into one forward pass: Thermometer became calibrated uncertainty, MExGen became context attribution.

Featured · COLM 2026

Decocted Experience: context as a second axis for scaling agents

Test-time scaling means more compute: longer reasoning, more samples, more search. For agents much of that is spent exploring paths that go nowhere. This paper asks what happens if you scale the input instead and build the agent’s context from its own past runs.

Three results. Lessons distilled from experience beat raw experience and keep scaling as context grows. Memory has a sweet spot; an intermediate size beats keeping everything. And context with higher information gain about the output gives more efficient inference, which in practice means balancing relevance against diversity. Validated on math, web shopping and software engineering agents, with Maohao Shen, Kaiwen Zha, Zexue He and Gregory Wornell among others.

Test-time compute: longer reasoning, more samples, more searchContext: distilled experiencemore compute: cost up, much of it wastedthis paper: better context, same compute
Experience is distilled into lessons, consolidated so memory stays small, and organized as a concept tree for retrieval.

The papers

Agents, memory and guardrails

Agents fail between the turns: a bad plan before any action, a risk that builds over a long session, a cache shared with another agent. These papers put the check at that point.

Ships as Granite Guardian

  1. ICLR2026Co-corresponding author

    Building a Foundational Guardrail for General Agentic Systems via Synthetic Data

    A guardrail that runs before an agent acts. AuraGen synthesizes risky plans, Safiron flags and explains them, Pre-Exec Bench measures both.

    Y. Huang, H. Hua, Y. Zhou, P. Jing, M. Nagireddy, I. Padhi, G. Dolcetti, Z. Xu, et al., P. Sattigeri, X. Zhang

  2. arXiv2026Senior author

    BLINDSPOT: A Benchmark for Safety and Refusal Calibration in Long-Horizon Tool-Using Agents

    Judges whole agent trajectories, not turns: 2,500+ runs averaging 14.7 turns, each labeled safe, correctly refused, unsafe, over-refused or unclear.

    S. Asif, M. M. Amiri, M. Abbas, T. Pedapati, P. Sattigeri

  3. arXiv2026

    LCGuard: Latent Communication Guard for Safe KV Sharing in Multi-Agent Systems

    Agents that share KV caches leak private context. A learned transform of the cache keeps the task signal and blocks reconstruction.

    S. Asif, M. M. Amiri, M. Abbas, P. Sattigeri, K. N. Ramamurthy

  4. NAACL2025IndustrySenior author

    Granite Guardian: Comprehensive LLM Safeguarding

    One model for harm, jailbreaks and RAG hallucination, trained on human labels plus synthetic data. AUC 0.871 and 0.854 on the two benchmark families.

    I. Padhi, M. Nagireddy, G. Cornacchia, S. Chaudhury, T. Pedapati, P. Dognin, K. Murugesan, E. Miehling, et al., P. Sattigeri

  5. AIES2025Senior author

    When in Doubt, Cascade: Towards Building Efficient and Capable Guardrails

    Traced a bias detector’s errors to the use–mention distinction, then generated 300K contrastive examples to fix it at a fraction of the compute.

    M. Nagireddy, I. Padhi, S. Ghosh, P. Sattigeri

Calibration, uncertainty and abstention

A model that knows when it is likely wrong. Calibration that transfers, uncertainty for long outputs, abstention for reasoning models.

Ships as a calibrated-uncertainty capability for Granite models

  1. ICML2024

    Thermometer: Towards Universal Calibration for Large Language Models

    An auxiliary model, trained across tasks, that calibrates an LLM on tasks it has not seen. Accuracy unchanged, inference cost negligible.

    M. Shen, S. Das, K. Greenewald, P. Sattigeri, G. Wornell, S. Ghosh

  2. NeurIPS2024

    Graph-based Uncertainty Metrics for Long-form Language Model Generations

    Treats samples and their claims as a bipartite graph; claim uncertainty is graph centrality. Self-consistency is the degree-centrality special case.

    M. Jiang, Y. Ruan, P. Sattigeri, S. Roukos, T. Hashimoto

  3. NeurIPS2024

    Are Uncertainty Quantification Capabilities of Evidential Deep Learning a Mirage?

    Shows evidential deep learning’s epistemic uncertainty does not behave as claimed, and that the methods are better read as energy-based OOD detectors.

    M. Shen, J. J. Ryu, S. Ghosh, Y. Bu, P. Sattigeri, S. Das, G. W. Wornell

  4. ACL2026Senior author

    Answering the Wrong Question: Reasoning Trace Inversion for Abstention in LLMs

    Reconstructs the question from a model’s reasoning trace; if it drifts from the question asked, abstain. Best in 33 of 36 settings.

    A. Gourabathina, I. Padhi, M. Nagireddy, S. Chaudhury, P. Sattigeri

Attribution and interpretability

Which part of the context, and which part of the model, produced this sentence.

Ships as a context-attribution capability for Granite models, and the ICX360 toolkit

  1. ACL2025Oral

    Multi-Level Explanations for Generative Language Models

    LIME- and SHAP-style attribution for long-context generation, at a cost that scales. More faithful than LLM self-explanations in human and automated tests.

    L. M. Paes, D. Wei, H. J. Do, H. Strobelt, R. Luss, A. Dhurandhar, M. Nagireddy, K. N. Ramamurthy, P. Sattigeri, W. Geyer, S. Ghosh

  2. ACL2026

    Multi-component Causal Tracing in Large Language Models

    Finds the set of attention heads and MLP neurons that jointly drive a metric, by relaxing the combinatorial search to a continuous one.

    Z. Yan, D. Wei, D. A. Katz, P. Sattigeri, A. Tajer

Generative modeling

A decade of generative models

Before LLMs I worked on what a generative model learns and how to control it. The same questions now show up as synthetic data: what to generate so that a judge learns the right thing.

  1. 2017Semi-supervised GANs

    NeurIPS. Tangent space from the generator, invariance into the classifier.

  2. 2018DIP-VAE

    ICLR. Disentangled latents without labels; the SAP score.

  3. 2019Fairness GAN

    Data that meets demographic parity or equal opportunity.

  4. 2020not-so-BigGAN

    Wavelet-domain sampling. FID 10.59 on ImageNet 512 at half the compute.

  5. 2025–26Data engines

    Synthetic trajectories and 300K contrastive samples that train judges. Next: diffusion.

DIP-VAE is one of the six methods in the ICML 2019 best paper by Locatello et al., which trained more than 12,000 disentanglement models. It is implemented in google-research/disentanglement_lib and AntixK/PyTorch-VAE.

Research

Other threads

Full list under Publications and on Google Scholar.

Human–AI collaboration

Who should act, and when a person should trust the model.

  1. AISTATS2023

    Who Should Predict? Exact Algorithms for Learning to Defer to Humans

    Learning a linear classifier–rejector pair is NP-hard even when a perfect one exists. An exact MILP for small cases, a consistent surrogate for the rest.

    H. Mozannar, H. Lang, D. Wei, P. Sattigeri, S. Das, D. Sontag

  2. NeurIPS2023Spotlight

    Effective Human-AI Teams via Learned Natural Language Rules and Onboarding

    Finds where people over- or under-trust a model, describes each region in plain language, and teaches it. About 5% more accurate human–AI teams.

    H. Mozannar, J. J. Lee, D. Wei, P. Sattigeri, S. Das, D. Sontag

  3. AIES2021

    Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty

    Treats communicated uncertainty as a form of transparency: how to measure it, show it, and act on it with stakeholders.

    U. Bhatt, J. Antorán, Y. Zhang, Q. V. Liao, P. Sattigeri, R. Fogliato, et al.

Fairness and transfer

Changing what a trained model does without retraining it.

  1. NeurIPS2022First author

    Fair Infinitesimal Jackknife: Mitigating the Influence of Biased Training Data Points Without Refitting

    Uses infinitesimal-jackknife influence to find the training points that hurt a fairness metric, and removes their effect without refitting.

    P. Sattigeri, S. Ghosh, I. Padhi, P. Dognin, K. R. Varshney

  2. NeurIPS2023

    Efficient Equivariant Transfer Learning from Pretrained Models

    Makes a pretrained model equivariant by averaging over group-transformed features, with the weights learned rather than fixed.

    S. Basu, P. Katdare, P. Sattigeri, V. Chenthamarakshan, K. Driggs-Campbell, P. Das, L. R. Varshney

  3. NeurIPS2018

    Co-regularized Alignment for Unsupervised Domain Adaptation

    Aligns source and target in several diverse feature spaces at once and makes their target predictions agree.

    A. Kumar, P. Sattigeri, K. Wadhawan, L. Karlinsky, R. Feris, W. T. Freeman, G. Wornell

Adoption

Independent evaluation and adoption

“Granite-Guardian-3.2-5B showed the best generalization with only a 6.5% gap.”
R. J. Young, independent robustness study of ten guardrail models, 2025The gap is the accuracy drop from public benchmarks to novel attacks. The largest drop in the study was 57 points.
“IBM’s open-source safety models provide harm detection, jailbreak detection, topic control, hallucination detection, and RAG quality assessment, capabilities that other guard providers don’t yet offer.”
Traefik Labs, on adding Granite Guardian to Traefik Hub, March 2026One of three guard providers in the product’s safety pipeline.
Hugging Face downloads, all time, across the Granite Guardian models and the capability libraries
1.16M
Ollama pulls of granite3-guardian and granite4.1-guardian
371K
community quantized builds on Hugging Face
45
downloads of Granite Guardian 4.1 8B in the past 30 days
56K

Hugging Face downloads by Guardian version

All time, summed over the model sizes in each release. Read October 13, 2026.

  1. 3.0 Oct 2024331K
  2. 3.1 Dec 2024122K
  3. 3.2 Feb 2025150K
  4. 3.3 2025290K
  5. 4.1 Apr 2026213K
4.1 has been out six months and accounts for most of the past month’s downloads.

Independent benchmarks

In products

In open-source projects and tutorials

Research others have built on

Citation counts from Google Scholar, October 2026.

Open source

Models and toolkits

  • Granite Guardian

    Judge and guardrail models for LLM systems. Project and training lead.

    1.16M Hugging Face downloads across versions

  • Granite capability libraries

    Modular post-training: capabilities for reasoning, safety and agentic workloads, added to Granite models. Lead.

    58K Hugging Face downloads

  • AI Fairness 360

    Fairness metrics and bias mitigation algorithms for datasets and models. Core contributor.

    2,878 GitHub stars; the paper has 2,937 citations; an LF AI & Data (Linux Foundation) project

  • AI Explainability 360

    Interpretability and explanation methods for data and models. Core contributor.

    1,809 GitHub stars; the taxonomy paper has 732 citations

  • Uncertainty Quantification 360

    Estimating, communicating and using uncertainty in model predictions. Core contributor.

    270 GitHub stars

  • ICX360

    In-context explainability for LLMs, including the MExGen attribution method. Contributor.

    Released with the ACL 2025 paper

Collaborations

Academic collaborations and service

  • MIT

    Gregory Wornell, David Sontag, William Freeman, Aude Oliva; with Maohao Shen, Joshua Lee, Abhin Shah, J. Jon Ryu, Hussein Mozannar, Hunter Lang, Abinitha Gourabathina

    Calibration and uncertainty, learning to defer, abstention, agents that learn from experience, domain adaptation

  • Rensselaer Polytechnic Institute

    Ali Tajer, Mohammad Mohammadi Amiri; with Zirui Yan, Burak Varıcı, Sadia Asif

    Causal bandits and causal discovery, causal tracing in LLMs, agent safety benchmarks, multi-agent communication

  • Stanford University

    Tatsunori Hashimoto; with Mingjian Jiang, Yangjun Ruan, Zexue He

    Uncertainty for long-form generation, experience-driven agents

  • Mila and McGill University

    David Rolnick; with Paula Harder, Qidong Yang, Alex Hernandez-Garcia, Venkatesh Ramesh

    Hard-constrained deep learning for climate downscaling

Affiliations as on the joint papers.

Service

  • Associate Editor, Pattern Recognition
  • Area chair and senior PC: NeurIPS, ICML, ICLR, AAAI
  • IEEE Senior Member
  • Invited talks at the National Academy of Sciences, the MIT AI Conference and NAACL TrustNLP (2024)