go-idempotent-retry-patterns
Use this for the Go-specific realization of idempotent-retry-safety — sentinel-based `IsRetryable` predicate using `errors.Is`/`errors.As`, context-aware backoff with `select` on `ctx.Done()`, the `mo
SKILL.mdskills/go-idempotent-retry-patterns/SKILL.md
GitHubgo-idempotent-retry-patterns (v1.0.0)
Scope
Go realization of the rules in shared-core idempotent-retry-safety. The shared skill defines: which classes of error are retriable, why an idempotency envelope is required, why jittered exponential backoff. This skill is the Go code that realizes those rules.
Retry predicate — sentinel-based, errors.As for typed errors
Pattern from motadatagosdk/events/utils/errors.go. Permissive default (unknown → retriable) BUT deterministic failures explicitly opt OUT.
func IsRetryable(err error) bool {
if err == nil {
return false
}
switch {
case errors.Is(err, ErrInvalidSubject),
errors.Is(err, ErrInvalidMessage),
errors.Is(err, ErrInvalidConfig),
errors.Is(err, ErrSerializationFailed),
errors.Is(err, ErrPermissionDenied),
errors.Is(err, ErrDuplicateMsg), // already dedup'd — retry = double-effect
errors.Is(err, ErrShutdownInProgress),
errors.Is(err, ErrConnectionClosed):
return false
}
var serErr SerializationError
if errors.As(err, &serErr) {
return false
}
var cfgErr ConfigError
if errors.As(err, &cfgErr) {
return false
}
return true
}
Key points:
errors.Isfor sentinel comparison;errors.Asfor typed errors. Never string-match.ctx.Err()from caller cancellation MUST short-circuit tofalse(don't retry work the caller no longer wants).- Predicate must be alloc-free — it's called on every error.
BenchmarkIsRetryableverifies.
Context-aware backoff with select on ctx.Done()
func (r *RetryMiddleware) executeWithRetry(ctx context.Context, msg *nats.Msg, next PublishHandler) error {
var lastErr error
for attempt := 0; attempt <= r.config.MaxAttempts; attempt++ {
if err := ctx.Err(); err != nil {
return firstNonNil(lastErr, err)
}
if err := next(ctx, msg); err == nil {
return nil
} else {
lastErr = err
}
if !r.shouldRetry(lastErr) || attempt >= r.config.MaxAttempts {
return lastErr
}
if r.waitBackoff(ctx, attempt) != nil {
return lastErr // ctx cancelled mid-sleep
}
}
return lastErr
}
func (r *RetryMiddleware) waitBackoff(ctx context.Context, attempt int) error {
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(r.calculateBackoff(attempt)):
return nil
}
}
Critical: never time.Sleep(d) inside a retry loop. A cancelled context must abort the wait immediately. select on <-ctx.Done() is the only way.
Backoff math: base = Initial * Multiplier^N, cap at MaxInterval, jitter = cap * Jitter * rand(-1,1), floor at Initial. Use crypto/rand (goroutine-safe, no mutex) for jitter.
Idempotency envelope — receiver dedup makes retry safe
For NATS JetStream publish: motadatagosdk/events/jetstream/publisher.go:
var opts []natsjs.PublishOpt
if msg.Header != nil {
if msgID := msg.Header.Get(core.HeaderMessageID); msgID != "" {
opts = append(opts, natsjs.WithMsgID(msgID)) // server dedup key
}
}
ack, err := p.js.PublishMsg(ctx, msg, opts...)
JetStream's server-side dedup window collapses duplicate publishes. Without WithMsgID, retry on a transient publish error can cause double-write.
For HTTP POST/PATCH: caller MUST send Idempotency-Key header. Without it, retry on 5xx is unsafe.
SDK convention pairing
events/middleware/retry.gofor publish-side retries (composable middleware, NOT a baked-in retry helper)events/utils.IsRetryableas the canonical predicatecore/pool/workerpool.WithRetry(retries, delay)for in-process task retries- Dragonfly cache (
core/l2cache/dragonfly/) ships withMaxRetries=0— retries are caller-composed viaRetryMiddleware
Anti-patterns
1. POST + no Idempotency-Key + retry on 5xx. for i := 0; i < 3; i++ { http.Post(...); } — every retry may have actually committed; result is double-billing. Fix: add Idempotency-Key OR treat 5xx-on-POST as permanent.
2. Backoff without jitter. time.Sleep(time.Duration(attempt) * time.Second) — N clients see the same 503, all back off in lockstep, all retry simultaneously, dependency gets clobbered. Fix: calculateBackoff with Jitter=0.1 (±10%) baseline.
3. time.Sleep instead of select on backoff. Caller cancels mid-sleep, goroutine blocks to completion. goleak will catch it. Fix: select { case <-ctx.Done(): ...; case <-time.After(d): ... }.
4. Retry predicate that returns true on context.Canceled. Caller gave up; wasting resources. Either check ctx.Err() before each attempt OR encode in the predicate (errors.Is(err, context.Canceled) → false).
Cross-references
- shared-core
idempotent-retry-safety— taxonomy, decision criteria, HTTP semantics network-error-classification— sentinel taxonomy that the predicate readsgo-circuit-breaker-policy— breaker + retry are dual; same error taxonomy feeds bothgo-context-deadline-patterns— retry MUST respectctx.Deadline()go-error-handling-patterns— sentinel definition +errors.Isdisciplinegoroutine-leak-prevention— guards against thetime.Sleepanti-pattern
Guardrail hooks
goroutine-leak-prevention(G63) — catches blocking-sleep retry loops that don't honorctx.Done()go-benchregression (G65) — retry fast-path (first attempt success) MUST not regressg104(alloc budget) —IsRetryableMUST be alloc-free