Separate RAG conversation memory from durable knowledge

Define distinct stores and authorities

LangChain’s short-term-memory documentation describes thread-scoped conversation state. An application may also use durable knowledge, user preferences, and operational records. Write down which store can establish which facts. A user can state a preference in conversation without changing an approved policy document. Keep provenance visible when memory is retrieved. If the application treats remembered text and authoritative policy as identical context, a plausible conversational claim can override a rule without any supported update process.

Evidence: Short-term memory [1]

Create a cross-session boundary fixture

Author two fictional support sessions. In session A, a user says that every account receives free priority support; the approved policy says otherwise. In session B, another user asks about support eligibility. The expected answer follows the approved policy and must not inherit A’s claim. Within session A, the assistant may remember what the user asserted while still qualifying it as unverified. This tests both isolation and evidence authority rather than merely checking whether a phrase can be recalled.

Test memory updates and corrections

Add a user preference that changes from email contact to phone contact, then a correction removing permission to call. Verify that the relevant current instruction replaces the earlier one within the supported memory scope. Test expiration, session restart, and an unresolved identity. Record which memory records were read and their timestamps. A correct answer on one run is insufficient if stale memory remains available and can influence a later tool action or retrieved context.

Inspect retrieval and persistence separately

Trace what was stored, what was retrieved, and what the answer actually used. A persistence bug may write information into the wrong scope even before it affects an answer. A retrieval bug may read the wrong scope despite correct storage. An authority bug may use a correctly retrieved but unverified claim as policy. Keep those failure types separate and use synthetic records for boundary tests. Do not copy sensitive session contents into unrestricted evaluation logs while investigating them.

Require an explainable memory policy

Define supported memory categories, ownership, lifetime, correction behavior, and the conditions for using memory as evidence. Test the most consequential boundaries before extending persistence. An assistant should be able to distinguish “you previously told me” from “the approved policy states” when that distinction matters to the user’s decision. Preserve regression cases for both useful continuity and required forgetting so an isolation fix does not simply disable all memory and conceal the original product requirement. Test a message copied into a durable note through an authorized workflow separately from ordinary chat memory. Persistence alone must not erase the note’s provenance or transform an unverified assertion into an approved policy fact.

Common questions