idempotent-retry-safety
Use this when designing retry behaviour for an SDK method that causes a side effect — picking the IsRetryable predicate, requiring an idempotency envelope (Idempotency-Key header, MessageID, upsert ke
SKILL.mdskills/idempotent-retry-safety/SKILL.md
GitHub<!-- Cross-language: body is language-neutral; cross-references name language-pack siblings (`go-idempotent-retry-patterns`, `python-idempotent-retry-patterns`, `python-asyncio-leak-prevention`). The leakage scripts honor `cross_language_ok: true`. -->
idempotent-retry-safety (v1.1.0)
Scope
Defines the language-neutral rules for safe retries:
- A predicate that only retries errors whose causes admit repetition.
- An idempotency envelope so the receiver can dedupe.
- Capped exponential backoff with jitter.
Language-specific code lives in go-idempotent-retry-patterns and python-idempotent-retry-patterns.
Rationale
A retry that isn't proved idempotent is a correctness bug, not a resilience feature. Duplicated side effects (double-charge, double-publish, double-insert) are far worse than the transient failure the retry was meant to mask. Retry safety rests on three legs:
- Predicate. Only retries errors whose causes admit repetition (transient network blip, server overload, pool timeout). Deterministic failures (invalid input, config error, permission denied, duplicate-already-detected) MUST NOT retry — same input → same failure.
- Idempotency envelope. A caller-generated identifier (HTTP
Idempotency-Keyheader, message ID, upsert key) that the receiver uses to dedupe. Without it, the caller cannot prove a retry didn't double-effect. - Backoff with jitter. Capped exponential backoff prevents accidental DoS of a recovering dependency. Jitter (±10% baseline) prevents thundering-herd lockstep retries.
Activation signals
- A method causes a side effect (publish, write, mutation) and the TPRD requests retry behavior.
- An error taxonomy is being designed and you need to decide which classes are retriable.
- A publisher/client is getting
MaxAttempts/MaxRetriesconfiguration. - Integration tests observe duplicate deliveries or double-processing.
- HTTP client work: deciding whether POST/PATCH can retry on 5xx.
- Reviewer cites "retries on non-idempotent op" or "no jitter → thundering herd".
Decision criteria (language-neutral)
| Situation | Retry? | Notes |
|---|---|---|
| GET / idempotent query returned 5xx | YES | Safe — no side effect |
| PUT / upsert with deterministic key returned 5xx | YES | Key dedupes server-side |
| POST without idempotency key returned 502 | NO | Cannot prove request didn't commit |
POST with Idempotency-Key header returned 502 |
YES | Server-side dedup makes retry safe |
| Publish with dedup-message-id returned transient error | YES | Server dedup handles it |
| Duplicate-already-detected sentinel | NO | Already delivered; retry = bug |
| Invalid-config sentinel | NO | Deterministic failure; same result on retry |
| Caller cancellation | NO | Caller no longer wants the op |
Tuning baseline: MaxAttempts=3, Initial=100ms, Multiplier=2.0, Max=5s, Jitter=0.1. Raise MaxAttempts only for long-deadline batch ops; never raise Max above the caller's likely deadline.
Universal anti-patterns (language-agnostic)
- Retrying a non-idempotent op with no envelope. POST with no
Idempotency-Key, retry on 502 — every retry may have committed. - Backoff without jitter. N clients all back off for exactly 1s, all retry simultaneously, dependency gets clobbered.
- Retry predicate that returns true on caller cancellation. Wastes resources; can mask deadline-exceeded into ResourceExhausted.
- Blocking sleep that ignores cancellation. Use the language's cancellation-aware sleep primitive, not the blocking variant.
Language realizations
go-idempotent-retry-patterns— Go realization (sentinel-based predicate, middleware composition, context-aware backoff via channel-select, server-side message-ID dedup envelope).python-idempotent-retry-patterns— Python realization (exception-class predicate with cause-chain walk, decorator-driven retries, async-cancellable backoff, idempotency-key headers on HTTP, idempotent-producer config).
Cross-references
network-error-classification— sentinel taxonomy that the predicate reads- shared-core
tdd-patterns— every retry path needs a RED test that exercises it go-circuit-breaker-policy/python-circuit-breaker-policy— breaker + retry are dual; same error taxonomy feeds bothgo-context-deadline-patterns/python-asyncio-leak-prevention— retry MUST respect the caller's deadline / cancellation