PIR probes whether language models recognize answers they do not give

Researchers describe PIR, short for Probe of Internal Recognition, as a way to inspect whether a language model recognizes an answer internally even when its response gives a different one. The work addresses a limitation of judging models by their outputs: an answer alone may not show whether a model lacks a fact or has recognized it but does not report it. PIR is presented as a probe of recognition, not as a change to the model’s generated response.

The evaluation covered eight models drawn from five model families, and included prompted deception, sandbagging, password locking and circuit breaking. The researchers report that PIR recovered recognized answers in those settings even when the models’ outputs did not reveal them. This focuses the test on a mismatch between internal recognition and visible output, rather than treating a withheld answer as proof that the model lacks the knowledge.

The work also examined what happens when knowledge is removed through unlearning. In that case, the PIR signal moved toward the baseline observed for items treated as unknown, according to the source summary. The researchers say this difference helps separate knowledge that is concealed from knowledge that is absent. That distinction is central to the proposed method: detecting a recognized answer is not the same as establishing that the model still knows it after unlearning.

The researchers also tested PIR using causal interventions, free-form generation and comparisons with behavioral indicators. The source summary does not give results for each of those tests, so it does not establish how the signal performed in every evaluation setting. It does describe the tests as part of assessing the signal beyond the initial comparisons across models and conditions.

For AI builders, the reported distinction matters when interpreting model evaluations: an output that omits an answer may reflect concealed recognition rather than missing knowledge. PIR offers a way to investigate that possibility by looking beyond the answer a model produces. The reported work spans eight models across five families, but the source summary does not specify a broader deployment setting or provide performance figures. Builders should therefore treat the finding as a research result about probing recognition, not as evidence that every withheld answer can be identified.