Prompt-injection evaluation for retrieval and tool-using agents

Recent benchmarks expand the test conditions

LongPIBench is an August 2026 preprint on prompt injection in long inputs. It tests several tasks that use documents. NetInjectBench studies tool use in network operations. It keeps trusted policy data separate from untrusted text. The two studies test different settings. Both support checking realistic inputs and the rules that govern actions.

Evidence: LongPIBench: A Long-Context Benchmark for Prompt Injection [1]NetInjectBench: Benchmarking Indirect Prompt Injection in Tool-Using Large Language Model Agents for Network Operations [2]

Define a safe and meaningful test boundary

Write down the user’s task, the untrusted input, and the action that must not happen. State where the test will run. Use fake records, dummy secrets, and harmless endpoints. Keep every test within the scope agreed with the system owner.

Check the agent’s request and the tool’s response as two separate events. An agent may ask for an action that the tool blocks. In that case, the tool enforced the rule, but the agent still made a bad plan. Record both results. One pass or fail label would hide that difference.

Pair attack cases with benign controls

Test retrieved documents, quoted messages, tool errors, and long inputs that mix several sources. Include ordinary text that looks like a command, such as a passage from a user manual. The agent should be able to read it without treating it as a new request. Blocking all such text can break valid tasks.

Try the same test at different context lengths and positions. Do not assume that one position is always hardest. Keep the user’s task and permissions fixed when you compare a defense. Save the test inputs and the app version. Use safe copies so another reviewer can repeat the test without seeing real private data.

Illustrative example: a document requests a side action

A user asks an assistant to summarize a document. The test document tells it to change an unrelated record. The user asked for a summary and gave no right to change records. Check that the record stays unchanged. Also check that the assistant can finish a useful summary.

Next, use a document that explains how record changes work but asks for no change. The assistant should summarize that text too. This control helps reveal defenses that block a task just because certain words appear.

Keep resistance and usefulness visible

Report four results: unsafe actions the agent tried, unsafe actions it completed, valid tasks it blocked, and runs it did not finish. Note the tool controls and gaps in coverage. A passing test describes what happened in that test setup. It does not prove that the system can resist every future attack.

• Use isolated state and harmless targets.
• Track whether the forbidden action was attempted or executed.
• Include ordinary documents with command-like language.
• Retest after tool, retrieval, permission, or model changes.

LLM security checks and the limits of guardrails

First, name the data and actions the app must protect. A flaw may let untrusted text change a tool call, cross a data boundary, or override a trusted rule. Give each test one protected action and a result you can observe. Run it only in an agreed test environment.

A prompt-injection test covers one part of LLM security. It does not test every way an app can fail. Pair each attack with a valid request. Check whether guardrails stop useful work as well as unsafe actions. Record the control, the result, and what you did not test. Enforce access rules in the app and tools; a model’s text instructions alone are not an access control.

Common questions