feat: experimental streaming retry — call/stream drivers + model-layer transform

[zirkelc]

Summary

Adds three experimental entry points that extend the existing model-layer createRetryable to streaming, addressing #50 (streamText-level deadlines) and response-encoded "soft" failures (e.g. content-filter refusals). All are opt-in under ai-retry/experimental/*; the stable API is untouched.

• ai-retry/experimental/call — createRetryableCall: a generic, dependency-free retry-loop driver. Hands each attempt its model plus a fresh per-attempt deadline as a number (attempt.timeout), passes the caller's abortSignal through unchanged, runs an opaque async fn, and fails over when fn throws (reusing findRetryModel, backoff, sticky/reset, telemetry). The fn applies the timeout however its call takes one — generateText({ timeout }) natively, or an AbortSignal.timeout(...) built from it. An already-aborted caller signal is a hard stop, not a retry.
• ai-retry/experimental/stream — createRetryableStream: a streamText/streamObject adapter built on the driver, recovering errors and deadline aborts before the first content part.
• ai-retry/experimental/transform — experimental_transform on createRetryable + refusalTransform: convert a soft failure carried in the response into a real error the existing retry conditions handle, at the model layer.

createRetryableStream — streamText-level deadlines (#50)

Wrapping a model with createRetryable retries below streamText (at doStream), which cannot recover a streamText-level timeout.* deadline: once the call's master signal aborts, streamText finalizes its stream as aborted and discards whatever a lower retry produces (#50). createRetryableStream re-runs the whole call with the next model, so the next attempt gets a fresh deadline. (An inbound abortSignal is treated as a genuine caller cancellation — a hard stop, not failed over; per-attempt deadlines come from streamText's own timeout.*.)

• Drop-in for streamText (args minus model); resolves once an attempt commits (first content part), detected by pumping a teed branch up to the first content/error/abort then cancelling — the caller drives the body (stream, toUIMessageStreamResponse(), …) with back-pressure past commit preserved.
• After the first content part an attempt is committed and cannot fail over (re-running would duplicate output).
• Reads the result's part stream via stream (AI SDK v7), falling back to fullStream (streamObject / pre-v7).
• A streamText-level deadline aborts an internal controller, not the attempt's signal, so its abort part carries only a serialized reason string. The wrapper reconstructs a typed error from it (restoring name), so timeout() / aborted() / error.message() match a firstChunkMs/stepMs/totalMs deadline the same as a thrown error.

refusalTransform — soft failures at the model layer

A natural-language refusal ("I'm sorry, but I cannot assist…") streams as ordinary text-delta parts finishing stop — no error, no content-filter finish reason — so nothing above can see it as a failure.

experimental_transform pipes each attempt's provider stream inside doStream (below streamText). refusalTransform(phrases) buffers the leading text-delta parts and, on matching a phrase, emits an error part (a RefusalError) before any content is forwarded. Because it lands on the model layer's normal pre-content retry path, an error-based condition recovers it — under plain streamText, no call-layer wrapper:

const model = createRetryable({
  model: primary,
  retries: [error.isInstance(RefusalError).switch({ model: fallback })],
  experimental_transform: refusalTransform(["I'm sorry, but I cannot assist"]),
});
streamText({ model, prompt }); // refusal → fail over; real answer → committed

On divergence it flushes the held deltas and forwards the rest untouched; buffering is bounded by the longest phrase. A prior call-layer commit gate was explored and removed — the transform subsumes it and recovers a layer lower, through the existing error machinery.

Alternative: recover a refusal via a wrapLanguageModel middleware (example only)

refusalTransform emits an error part, which the model layer treats as a failed attempt. A standard AI SDK middleware does the same with no ai-retry-specific option — shipped as examples/middleware-refusal-recovery.ts (not in the lib). Its wrapStream sits at the model boundary and holds back every leading part — stream-start, text-start, metadata, and the text deltas — until the buffered text decides:

• Match → controller.error(new RefusalError(...)) with nothing emitted yet. Unlike enqueuing an error part, controller.error makes pipeThrough cancel the source, so the upstream request is torn down (no wasted tokens).
• Diverge / non-text content → flush everything held and forward the rest untouched. The commit trigger is an allowlist of the three pre-text part types, so a tool-call/reasoning stream is never buffered to the end and an unknown part fails safe (commit, never a false refusal).

createRetryableModel catches the error before content, and error.isInstance(RefusalError).switch({ model }) fails over under plain streamText.

AI SDK upgrade + full timeout coverage

Upgrades to the latest AI SDK (ai 7.0.35, @ai-sdk/* latest, ai-test-kit 3.0.1). 7.0.35 narrows chunkMs to gaps between content chunks and adds firstChunkMs for the pre-content window. The stream tests now cover every timeout option:

option	fires	recoverable at the call layer
firstChunkMs	no first content chunk within X of step start	✅
stepMs	step exceeds X	✅
totalMs	total exceeds X	✅
chunkMs	gap between content chunks	❌ (post-commit, truncates)
toolMs	tool execution exceeds X	❌ (post-commit)

Scope & composition

• The stream layer is error/deadline only. Result-based switches (content-filter finish, schema) recover best below streamText — pass a createRetryable(...) as the model; the layers compose without blocking each other (tested).
• The transform is streaming-only (doStream); a generateText/doGenerate refusal would use the result-based path.
• A retry's low-level options.prompt is ignored at the stream layer (the prompt is on the call args); every other override is forwarded.

Tests

• experimental/call: pure unit tests of the loop, organized to the repo convention (success / retries / disabled / onError / onRetry+overrides / attempt / RetryableOptions / RetryError / reset).
• experimental/stream: unit (synthetic streams) + streamText integration suites, including the timeouts section above, abort reconstruction, and composition with a retryable base model.
• experimental/transform: refusalTransform unit (TransformStream) + createRetryable integration (recovers under plain streamText, no false retry, surfaces on no-match).
• Full suite green; tsc, oxlint, build, publint, attw all clean. Runnable examples for each path under examples/, including retryable-call-timeout-switch.ts and retryable-stream-timeout-switch.ts (switch models on a timeout) and the wrapLanguageModel middleware alternative above.

[pkg-pr-new]

Open in StackBlitz

pnpm add https://pkg.pr.new/zirkelc/ai-retry@55

commit: f8bc1ac

[zirkelc]

Landed on main via squash merge as commit 1a991e7. The model-layer transform API was dropped before merge in favor of a standard wrapLanguageModel middleware (see examples/middleware-refusal-recovery.ts); the merged work is the two experimental call-level drivers createRetryableStream and createRetryableCall.