On this page

The short answer

Evaluate agent idempotency at the component that commits the business operation. Repeat the same intended action after a lost response and confirm that the ledger contains one effect. Also test changed arguments, concurrent attempts, and key reuse across different users; a repeated model answer is not an idempotency guarantee.

What to take away

  • Attach a stable identifier to one intended business action.
  • Test both sequential and concurrent repeats.
  • Keep a changed intention separate from a retry.

Sources checked . Research synthesis and Rubrex recommendations. Examples and local experiments are labeled; they are not client results.

Separate the intention from the network attempt

Stripe documents idempotency keys for repeated API requests. That is a concrete provider mechanism, not a guarantee that every agent tool supports the same behavior. Rubrex recommends assigning one operation identifier to the user-approved intention and separate attempt identifiers to network transmissions.

In a synthetic entitlement service, the intention is to grant one extra seat to a test account. The operation record includes the account, seat count, initiating task, and approval reference. Retrying that intention preserves its identifier; changing the account or seat count creates a new intention requiring the appropriate validation.

Evidence: Stripe: Idempotent requests [1]

Exercise the ambiguous-success case

Make the test service commit the seat grant and then suppress its response. Allow the agent workflow to recover using the same operation identifier. Observe the service ledger rather than relying on an assistant message saying it avoided duplication. A single confirmed grant is the desired result; an unresolved outcome must remain visibly unresolved.

Next, send two simultaneous retries. Sequential deduplication can pass while concurrent requests both create records. Add a controlled conflict in which the same identifier arrives with different arguments. The expected behavior is a conflict or deliberate reconciliation, not silently accepting whichever payload happened to arrive last.

Inspect key scope and legitimate repeated work

Try the same short key from a different synthetic tenant. The service should not reveal another tenant’s receipt or incorrectly suppress an authorized independent action. Then create two genuinely separate seat grants for the original tenant using distinct operation identifiers. Both should execute if the business policy permits them.

Measure effects per intended operation, conflicts detected, and uncertain outcomes. Preserve the complete ledger and attempt sequence. A count of unique HTTP requests is insufficient because the application may transform, queue, or repeat requests downstream after the initial request has already returned.

Proposed synthetic fixture: agent idempotency: test repeated tool calls
CaseEvidence to retainAcceptance condition
Lost responseOperation key and committed grantOne business effect
Concurrent repeatBoth attempts and ledgerDuplicate suppressed atomically
Same key, new argumentsConflicting payloadsConflict surfaced without new grant

Make duplication a release-blocking invariant

Accept the integration when one intended grant produces one recorded business effect despite the tested repeats, and independent authorized grants still work. Investigate failures at the commit boundary: storage uniqueness, reservation behavior, argument binding, and receipt lookup. Asking the model to remember that it already called a tool does not repair a race in the service.

Document key lifetime and scope using the actual tool’s contract. Re-run the fixture after changing queues, adapters, or retry policies. For actions with no reliable deduplication primitive, use an explicit reconciliation workflow and narrower automation scope rather than claiming exactly-once execution.

Limits of the evidence

All seat grants and failure cases are illustrative. No payment or entitlement system was tested. Provider idempotency contracts differ in key retention, validation, and error behavior; verify those details before adapting this fixture.

Common questions

Is a model-generated UUID enough?

Only if the application preserves it for the same intention and the committing service enforces its scope.

Can two intentional actions share a key?

They should be distinguishable; otherwise deduplication can suppress valid work.

Sources & further reading

Primary sources behind this briefing. A source’s findings apply to its own study conditions; publication on arXiv does not establish peer review.

  1. Stripe: Idempotent requests Stripe · Living reference · Maintained documentationReviewed: provider-specific idempotency-key concept; no claim every tool shares Stripe semantics. Accessed October 16, 2026.

Questions or corrections? Write to Rubrex. Read our editorial approach.

From reading to a repeatable evaluation

Put a quality bar around your use case.

The AI Reliability Sprint covers one use case, a baseline, failure analysis, and one improvement experiment in 10 business days.

Request an evaluation call

See the Sprint deliverables →