DerivAudit framework tests whether persistent-agent memories follow from past interactions

Hongjun Liu and Chen Zhao introduce DerivAudit, a framework for checking whether a persistent agent’s stored memory is supported by the interaction history available when it was written. The arXiv listing shows the paper was submitted on 28 Sep 2026. The work focuses on agents that condense earlier exchanges into durable records that may become premises for later tasks. Its central question is whether those records are justified by the history, not merely whether a memory writer has attached citations.

The authors describe two ways that judging support can go wrong. Evidence for a memory may be spread across earlier exchanges, so the excerpts cited by its writer might not show all the relevant support. At the same time, compression can combine details into a stronger claim, relationship or event status that the conversations never established. The paper frames the audit around three linked concerns: evidence scope, compositional validity and admission reliability. These address where supporting evidence can be found, whether the combined statement remains warranted, and how decisions to store memories affect their later use.

In evaluations using two natural memory corpora, the researchers broadened the evidence check to cover more of the history preceding each memory write. This recovered support for nearly 60% of memories that looked unsupported when assessed using citations alone. But the expanded review did not resolve every case: 17-21% remained unsupported afterward. The results therefore distinguish a memory that appears unsupported because its citations are incomplete from one that remains unsupported after more of the available history is considered.

The paper also reports that verification models frequently admitted unsupported memories, and that expanding the evidence by itself made admission reliability worse on two model backbones. For AI builders, the findings suggest that searching a larger history and deciding whether a memory is safe to retain are separate evaluation problems. A memory pipeline that checks only its cited excerpts could miss support elsewhere, while a broader search alone does not ensure that unsupported claims are rejected. DerivAudit offers a framework for examining both the evidence behind a stored statement and the decision to admit it.