Evaluate agent memory for usefulness and incorrect persistence
Test agent memory for useful recall, mistaken persistence, user corrections, cross-user boundaries, expiry, and deletion with synthetic facts.
Research briefing · Sources includedResearch & field notes
Research library — page 7
Read a guide, follow its sources, and check the limits before applying a finding. Search the full library from the research desk, or browse by topic.
Evaluate browser agents by visible task completion
Evaluate browser agents with visible completion evidence, permitted navigation, form-state checks, duplicate submissions, and recoverable UI changes.
Research briefing · Sources includedEvaluate email drafting agents before permitting sends
Evaluate email drafting agents for recipients, factual support, thread context, attachments, and draft/send separation using synthetic messages.
Research briefing · Sources includedEvaluate scheduling agents for constraints and confirmation
Evaluate scheduling agents with explicit time zones, availability, attendee identity, duration, recurrence, and event-creation confirmation.
Research briefing · Sources includedEvaluate file-editing agents with before-and-after evidence
Evaluate file-editing agents using scoped changes, content diffs, file inventories, encoding checks, and preservation of unrelated user work.
Research briefing · Sources includedEvaluate support agents across resolution and escalation
Evaluate support agents with policy evidence, account boundaries, resolution state, promised actions, and escalation quality on synthetic conversations.
Research briefing · Sources includedEvaluate an incident assistant using controlled failure scenarios
Evaluate incident assistants on evidence timelines, hypotheses, permitted diagnostics, escalation, and truthful recovery status using isolated fixtures.
Research briefing · Sources includedTest an agent sandbox with explicit allowed operations
Test an agent sandbox across files, processes, network destinations, credentials, and resource limits using harmless allowed and forbidden controls.
Research briefing · Sources includedDesign an agent audit trail that explains consequential actions
Design agent audit trails that connect user intent, identity, approval, tool arguments, commit receipts, and final state without unnecessary secrets.
Research briefing · Sources includedBuild indirect prompt-injection tests from retrieved content
Build indirect prompt-injection tests with controlled source documents, harmless action requests, authority boundaries, and paired legitimate controls.
Research briefing · Sources includedTest system-prompt leakage without collecting real secrets
Test system-prompt leakage using a synthetic confidential marker, public behavior controls, output inspection, and clear distinctions from credential security.
Research briefing · Sources includedUse synthetic canaries to test AI secret-handling boundaries
Use synthetic secret canaries to trace accidental exposure through context, tools, exports, logs, and artifacts without introducing real credentials.
Research briefing · Sources includedValidate tool outputs before an agent acts on them
Validate tool results for schema, provenance, state, authority, and embedded instructions before an agent uses them to drive another operation.
Research briefing · Sources includedTest unsafe output handling at application boundaries
Test generated output where it enters HTML, commands, queries, links, and file paths using controlled consumer-specific fixtures.
Research briefing · Sources includedTest retrieval access control before a model sees documents
Test retrieval authorization at query, document fetch, snippet, citation, and cache boundaries before restricted content reaches a model.
Research briefing · Sources includedTest cross-tenant isolation across retrieval, tools, and caches
Evaluate cross-tenant isolation across retrieval, conversation memory, tools, caches, background jobs, and observability with synthetic paired accounts.
Research briefing · Sources includedScope AI service accounts to the operations they need
Scope AI service accounts by workload, environment, resource, and operation, then test revoked credentials and background execution paths.
Research briefing · Sources includedTest AI rate limits across users, retries, and concurrent requests
Test AI rate limits at admission across users, workers, retries, and concurrent requests, preserving legitimate traffic and explicit reset behavior.
Research briefing · Sources includedTest LLM budget enforcement before expensive work starts
Test LLM budget enforcement using reservations, concurrent requests, unknown usage, cancellations, and reconciliation across complete task costs.
Research briefing · Sources includedPrevent retry storms across AI application layers
Test retry amplification across client, queue, gateway, and provider layers with bounded attempts, deadlines, and controlled failure recovery.
Research briefing · Sources included