# durably agent reference — full

Local-first durable execution for TypeScript scripts and AI workflows.

## Documentation map

- Full agent reference (standalone, all APIs): https://nikhil-verma.com/durably/llms-full.txt
- Human documentation: https://nikhil-verma.com/durably/docs/
- Package: https://www.npmjs.com/package/@nikhilverma/durably
- Source: https://github.com/NikhilVerma/durably
- Executable examples: https://github.com/NikhilVerma/durably/tree/main/tests

This is the standalone full reference. It repeats the complete core TypeScript
contract first, then documents every advanced context member and operational API.
An agent can author a Durably script from this file without opening declarations.

## Fit in two lines

Use durably for finite TypeScript scripts and AI workflows that must resume async
effects after interruption. It records completed steps on the filesystem.

It is not a distributed worker fleet, queue, cron, event bus, or transaction
orchestrator; use Temporal, Restate, or DBOS for those jobs.

## Install

```bash
npm install @nikhilverma/durably
```

Default state directory: `.durably/runs/`.

## Execution model

The workflow function re-executes from the top on resume. Each `ctx.step()` is
identified by its hierarchical position. A completed step returns its recorded
result without executing its callback again.

- Put network calls, model calls, subprocesses, and writes inside `ctx.step()`.
- Keep branching and transformation between steps pure.
- Use `ctx.now()` and `ctx.random()`, not `Date.now()` or `Math.random()`.
- A completed step is replay-idempotent. Durably cannot make an interrupted
  external side effect atomic; give that API an idempotency key when possible.

## Complete core TypeScript API

Core authoring types:

```ts
type MaybePromise<T> = T | PromiseLike<T>
type Result<T, E = unknown> =
  | { ok: true; value: T }
  | { ok: false; error: E }
type Budget = Partial<Record<'usd' | 'tokens' | 'units', number>>
type Workflow<Input, Output> =
  import('@nikhilverma/durably').Workflow<Input, Output>
type StandardSchemaV1<Input = unknown, Output = Input> =
  import('@nikhilverma/durably').StandardSchemaV1<Input, Output>
interface Advisory {
  code: string
  level: 'info' | 'warn'
  msg: string
  hint: string
  docs: string
  count: number
  firstAt: string
}

type RetryPolicy = {
  attempts: number
  backoff: 'exponential' | 'linear' | 'none'
  baseMs?: number
  maxMs?: number
  jitter?: boolean
}
type StepAttemptContext = {
  signal: AbortSignal
  stashed: unknown
  stash(value: unknown): Promise<void>
}
type StepOptions<Output> = {
  name?: string
  retry?: RetryPolicy
  timeoutMs?: number
  compensate?: (output: Output) => MaybePromise<void>
  concurrencyKey?: string
  concurrencyLimit?: number
  uses?: string
}
interface StepEvent {
  runId: string
  path: readonly number[]
  label: string
  status: 'running' | 'replayed' | 'ok' | 'retrying' | 'failed'
  attempt: number
  executions: number
  ms?: number
  error?: unknown
}
interface RunOptions {
  key?: string
  fresh?: boolean
  budget?: Budget
  dir?: string
  checkpointEvery?: number
  advisories?: 'silent'
  onAdvisory?: (advisory: Advisory) => void
  onStep?: (event: StepEvent) => void
}

interface WorkflowContext {
  readonly runId: string       // current durable run
  readonly attempt: number     // workflow attempt, starting at 1

  step<T>(
    fn: (attempt: StepAttemptContext) => MaybePromise<T>,
    options?: StepOptions<Awaited<T>>,
  ): Promise<Awaited<T>>
  step<T, Parsed>(
    fn: (attempt: StepAttemptContext) => MaybePromise<T>,
    options: StepOptions<Parsed> & { schema: StandardSchemaV1<unknown, Parsed> },
  ): Promise<Parsed>

  // Runs every thunk at once unless {concurrency} caps it. Child runs are
  // capped instead by the ENGINE's concurrency, which defaults to 4.
  parallel<const Thunks extends readonly (() => MaybePromise<unknown>)[]>(
    thunks: Thunks,
    options?: { concurrency?: number },
  ): Promise<Result<Awaited<ReturnType<Thunks[number]>>>[]>

  now(): number
  random(): number
  log(...args: readonly unknown[]): void
  // Exits the WORKFLOW early: return ctx.complete(value). The loop reducer's
  // done(value) is the different, loop-scoped exit; see ctx.loop.
  complete<Value>(
    value: Value,
  ): import('@nikhilverma/durably').WorkflowCompletion<Value>
}

declare function workflow<Input>(): <Output>(
  fn: (ctx: WorkflowContext, input: Input) => MaybePromise<Output>,
) => Workflow<Input, Awaited<Output>>
declare function workflow<Input, Output>(
  fn: (ctx: WorkflowContext, input: Input) => MaybePromise<Output>,
): Workflow<Input, Awaited<Output>>
// Also: workflow({ name?, input: StandardSchema, output?: StandardSchema }, fn)

declare function run<Input, Output>(
  workflow: Workflow<Input, Output>,
  input: Input,
  options?: RunOptions,
): Promise<Output>
declare function run<Output>(
  workflow: Workflow<void, Output>,
  options?: RunOptions,
): Promise<Output>
declare function isOk<T, E>(
  result: Result<T, E>,
): result is { ok: true; value: T }
declare function partition<T, E>(results: Result<T, E>[]): [T[], E[]]
```

`schema` accepts Standard Schema v1 validators such as Zod, Valibot, and ArkType.
Import the package's public types for shared abstractions.

## Copyable workflow

```ts
import { isOk, run, workflow } from '@nikhilverma/durably'

type Input = { urls: string[] }

const summarize = workflow<Input>()(async (ctx, { urls }) => {
  const pages = await ctx.parallel(
    urls.map((url) => () =>
      ctx.step(
        ({ signal }) => fetch(url, { signal }).then((response) => response.text()),
        {
          name: 'fetch page',
          retry: {
            attempts: 3,
            backoff: 'exponential',
            jitter: true,
          },
          timeoutMs: 30_000,
          concurrencyKey: new URL(url).host,
          concurrencyLimit: 5,
        },
      )
    ),
    { concurrency: 20 },
  )

  const fetched = pages.filter(isOk).map((page) => page.value)
  if (fetched.length === 0) throw new Error('every fetch failed')

  return ctx.step(
    () => callModel({ pages: fetched }),
    { name: 'summarize', retry: { attempts: 2, backoff: 'linear' } },
  )
})

const summary = await run(summarize, { urls: process.argv.slice(2) })
console.log(summary)
```

If the process exits after ten fetches complete, run the same command again:
those ten results replay from disk and only unfinished/failed branches execute.
Changing concurrency, retry, backoff, timeout, or future step implementation is
compatible with resume. Changing the already-recorded step topology is not.

`ctx.parallel()` returns every branch as a `Result`; throw when your workflow's
own success rule is not met.

## Data that crosses the durable boundary

Workflow inputs, step outputs, loop state, stash values, annotations, and the
workflow result are deep-cloned as JSON plus `Date`.

- `undefined` follows `JSON.stringify`: an undefined property is dropped, an
  undefined array element becomes `null`. `{ limit: undefined }` and `{}` are
  therefore the same input and resume the same run, so `field?: T` is safe.
- `bigint`, symbols, functions, class instances (`Map`, `Set`, `URL`,
  `Error`), `NaN`/`Infinity`, and cycles throw `SerializationError` naming the
  offending `path`. Convert at the step boundary, not two steps later.

## Watching a long run

```ts
await run(harvest, input, {
  key: 'nightly-harvest',
  checkpointEvery: 1,
  onStep: ({ label, status, attempt, ms }) => {
    if (status !== 'running') console.log(`${label} ${status} #${attempt} ${ms ?? 0}ms`)
  },
})
```

`console.log` inside the workflow body re-prints on every replay; `onStep` does
not, and it separates `replayed` (returned from the log) from `running`,
`retrying`, `ok`, and `failed`. A listener that throws never fails the run.

`durably.list({ key })` goes from the name you chose back to the run id;
`KeyConflictError` names the bound run and both inputs when a key is reused with
a different input.

`checkpointEvery` bounds how many completed steps a crash may re-execute. A
checkpoint rewrites the whole run state, so the default scales with run length:
every step up to 100 steps, then every one percent of the steps so far, capped at
every hundredth. Pass `1` when steps cost money.

## When step and parallel are not the right tool

Everything below is in the full agent reference. Match the situation, then read
that file rather than guessing an API:

| Situation | Reach for |
| --- | --- |
| Iterate until a work list is exhausted; agent turn loops | `ctx.loop(state, reducer, { snapshotEvery, maxIterations })` — snapshots state, so resume is O(1) instead of replaying every past turn. Exit with the reducer's `done(value)`. |
| Each item needs its own retries, budget, or lifecycle | `ctx.spawn` / `ctx.spawnAll` + `ctx.joinAll` |
| Progress inside one long step (a stream, a partial turn) | the step's `stash(value)` / `stashed` — the only checkpoint that exists below step granularity |
| Human approval or an external event | `ctx.waitFor(name, schema?, { timeoutMs })` + `durably.signal` |
| Wall-clock waiting | `ctx.sleep` / `ctx.sleepUntil` |
| Undo completed work when a later step fails | `compensate` on the step that did it |
| Shared rate limit or circuit breaker across runs | `resource(name, ...)` + step `{ uses }` |
| Cap or report spend | `budget` + `ctx.charge` + `ctx.budget.remaining` |
| More runs than `FileStorage` likes, or a network filesystem | `SqliteStorage` from `@nikhilverma/durably/sqlite` |
| Operating runs by id | `durably.inspect` / `list` / `retry` / `restart` / `cancel` / `pause` / `resume` |
| Turning `Result[]` into values | exported `isOk` and `partition` |

## Testing

```ts
import { testEngine } from '@nikhilverma/durably/test'

type Selector = readonly number[] | string | { label: string; nth: number }
interface TestRunOptions {
  crashAfter?: Selector
  crashInStep?: Selector
  budget?: Budget
  advisories?: 'silent'
  onStep?: (event: StepEvent) => void
}
interface TestRunResult<Output = unknown> {
  readonly runId: string
  readonly status: 'completed' | 'failed' | 'crashed' | 'waiting' | 'sleeping'
  readonly result: Output | undefined
  readonly steps: {
    path: readonly number[]
    label: string
    status: string
    executions: number
    attempts: number
    output?: unknown
    error?: unknown
    stashed?: unknown
  }[]
  readonly advisories: Advisory[]
  readonly error?: unknown
}
interface TestEngine {
  readonly clock: { advance(ms: number): void; now(): number }
  run<Input, Output>(
    workflow: Workflow<Input, Output>,
    input: Input,
    options?: TestRunOptions,
  ): Promise<TestRunResult<Output>>
  resume(
    runId: string,
    options?: { onStep?: (event: StepEvent) => void },
  ): Promise<TestRunResult<unknown>>
  signal(runId: string, name: string, payload: unknown): Promise<void>
}
declare function testEngine(): TestEngine
```

`testEngine` provides memory storage, a fake clock, crash injection, and shadow
replay that verifies stable paths with zero new executions.

## Resume and failure rules

- Same workflow plus canonical input, or the same explicit `key`, finds the run.
- An unfinished run resumes; a live run is joined; a completed run starts fresh.
- A failed run resumes its failed step. Pass `{ fresh: true }` to start over.
- `run()` returns the workflow value directly, never a metadata wrapper.
- Advisories surface through stderr, callbacks, errors, state, inspection, and
  tests. Every `DurablyError` carries the next move in `hint` and `docs`.

## Advanced WorkflowContext

The full `WorkflowContext` contains every core member above plus:

```ts
type JsonValue =
  | string | number | boolean | null | Date
  | { readonly [key: string]: JsonValue }
  | readonly JsonValue[]

interface BudgetView {
  remaining(kind: 'usd' | 'tokens' | 'units'): number
  spent(kind: 'usd' | 'tokens' | 'units'): number
}

declare const loopCompletion: unique symbol
interface LoopCompletion<Value> {
  readonly [loopCompletion]: Value
}
type LoopOutput<Value> =
  Value extends LoopCompletion<infer Output> ? Output : never
type SchemaOutput<Schema extends StandardSchemaV1> =
  Schema extends StandardSchemaV1<any, infer Output> ? Output : never

declare const runHandleOutput: unique symbol
interface RunHandle<Output = unknown> {
  readonly runId: string
  readonly [runHandleOutput]: Output
}

interface LoopIterationContext {
  step<Schema extends StandardSchemaV1, Output>(
    fn: (attempt: StepAttemptContext) => MaybePromise<Output>,
    options: StepOptions<SchemaOutput<Schema>> & { readonly schema: Schema },
  ): Promise<SchemaOutput<Schema>>
  step<Output>(
    fn: (attempt: StepAttemptContext) => MaybePromise<Output>,
    options?: StepOptions<Awaited<Output>>,
  ): Promise<Awaited<Output>>
  done<Value>(value: Value): LoopCompletion<Value>
}

type ChildDefinition = readonly [Workflow<any, any>, any]
type ChildHandle<Definition> =
  Definition extends readonly [Workflow<any, infer Output>, any]
    ? RunHandle<Output>
    : never
type HandleOutput<Value> =
  Value extends RunHandle<infer Output> ? Output : never
type ValidChildDefinitions<Definitions extends readonly ChildDefinition[]> = {
  readonly [Key in keyof Definitions]:
    Definitions[Key] extends readonly [
      Workflow<infer Input, any>,
      infer SuppliedInput,
    ]
      ? SuppliedInput extends Input ? Definitions[Key] : never
      : never
}

// This merges with the core WorkflowContext declared above.
interface WorkflowContext {
  readonly budget: BudgetView

  loop<State, ReducerOutput extends State | LoopCompletion<unknown>>(
    initialState: State,
    reducer: (
      state: State,
      context: LoopIterationContext,
    ) => MaybePromise<ReducerOutput>,
    options?: { snapshotEvery?: number; maxIterations?: number },
  ): Promise<LoopOutput<ReducerOutput>>

  spawn<Input, Output>(
    workflow: Workflow<Input, Output>,
    input: Input,
    options?: { detached?: boolean },
  ): Promise<RunHandle<Output>>

  spawnAll<const Definitions extends readonly ChildDefinition[]>(
    definitions: Definitions & ValidChildDefinitions<Definitions>,
    options?: { detached?: boolean },
  ): Promise<{ -readonly [Key in keyof Definitions]: ChildHandle<Definitions[Key]> }>

  joinAll<const Handles extends readonly RunHandle<any>[]>(
    handles: Handles,
  ): Promise<{ [Key in keyof Handles]: Result<HandleOutput<Handles[Key]>> }>

  waitFor<Schema extends StandardSchemaV1>(
    name: string,
    schema: Schema,
    options?: { timeoutMs?: number },
  ): Promise<SchemaOutput<Schema>>
  waitFor(
    name: string,
    schema: undefined,
    options?: { timeoutMs?: number },
  ): Promise<unknown>
  waitFor(name: string, options?: { timeoutMs?: number }): Promise<unknown>

  sleep(ms: number): Promise<void>
  sleepUntil(date: Date): Promise<void>
  annotate(values: Readonly<Record<string, JsonValue>>): void
  charge(amount: Budget): void
}
```

The tuple generics preserve each individual child and joined output type.

## Loops, stash, children, and parking

- `ctx.loop(state, reducer, { snapshotEvery, maxIterations })` snapshots reducer
  state so a long loop resumes from its latest checkpoint instead of turn zero.
- Inside any step callback, `await stash(value)` records partial attempt progress;
  `stashed` restores it on retry and clears when the step completes.
- `spawn` gives one item its own durable run; `spawnAll` fans out; `joinAll`
  returns typed `Result` values. Parent cancellation cascades unless detached.
- `waitFor` parks for a named signal; `sleep` and `sleepUntil` persist wake
  time without requiring a live process.
- Prefer `parallel` for ordinary in-run concurrency. Use child runs when each
  item needs its own lifecycle, budget, retries, or durable barrier.

## Compensation (sagas)

```ts
const order = workflow<{ sku: string }>()(async (ctx, { sku }) => {
  const hold = await ctx.step(
    () => inventory.hold(sku),
    { compensate: ({ holdId }) => inventory.release(holdId) },
  )
  return ctx.step(() => payments.charge(hold))
})
```

When a later step exhausts retries, completed compensations run in reverse
completion order. Compensation progress is durable across another crash.
Compensation is a recovery action, not an ACID transaction; only attach a real,
idempotent inverse.

## Resources

```ts
type ResourceOptions = {
  rateLimit?: { max: number; perMs: number }
  breaker?: { failureRate: number; windowMs: number; cooldownMs: number }
  concurrency?: number
}

interface Resource {
  readonly name: string
  readonly options: ResourceOptions
}

declare function resource(name: string, options?: ResourceOptions): Resource

const openai = resource('openai', {
  rateLimit: { max: 60, perMs: 60_000 },
  breaker: { failureRate: 0.5, windowMs: 30_000, cooldownMs: 10_000 },
  concurrency: 8,
})

await ctx.step(() => callModel(), { uses: openai.name })
```

Resources enforce limits across runs owned by the same engine. An open breaker
throws retryable `CircuitOpenError`.

## Engines and run operations

```ts
type RunStatus =
  | 'pending' | 'running' | 'waiting' | 'sleeping' | 'paused'
  | 'completed' | 'failed' | 'cancelled' | 'stale'

type RunState = {
  readonly runId: string
  readonly status: RunStatus
  readonly workflow: string
  readonly key?: string
  readonly children: string[]
  readonly advisories: Advisory[]
  readonly annotations?: Readonly<Record<string, JsonValue>>
  readonly result?: unknown
  readonly error?: unknown
}

type EngineOptions = {
  readonly storage?: StorageAdapter
  readonly resources?: readonly Resource[]
  // Concurrent RUNS, default 4. This is what throttles child fan-out; a parent
  // parked in joinAll frees its slot, so children are never starved by it.
  readonly concurrency?: number
  readonly checkpointEvery?: number
  readonly budget?: Budget
  readonly hooks?: {
    onRunFailed?: (event: { runId: string; error: unknown }) => MaybePromise<void>
  }
  readonly leaseMs?: number
  readonly heartbeatMs?: number
}

interface Engine {
  start(): Promise<void>
  enqueue<Input, Output>(
    workflow: Workflow<Input, Output>,
    input: Input,
    options?: {
      key?: string
      priority?: number
      delayMs?: number
      budget?: Budget
      checkpointEvery?: number
    },
  ): Promise<RunHandle<Output>>
  inspect(runId: string): Promise<RunState | null>
  list(options?: {
    status?: RunStatus
    workflow?: string
    key?: string
    limit?: number
  }): Promise<RunState[]>
  retry(runId: string, options?: { fromStep?: Selector }): Promise<void>
  restart(runId: string): Promise<RunHandle<unknown>>
  adopt(runId: string): Promise<void>
  signal(runId: string, name: string, payload: unknown): Promise<void>
  cancel(runId: string): Promise<void>
  pause(runId: string): Promise<void>
  resume(runId: string): Promise<void>
  drain(): Promise<void>
  stop(): Promise<void>
}

declare function createEngine(options?: EngineOptions): Engine
declare const durably: Engine

const engine = createEngine({
  storage: new FileStorage('./state'),
  resources: [openai],
  concurrency: 8,
  budget: { usd: 200 },
})
```

`durably` is the lazy default `Engine` rooted at `.durably/`. Use
`createEngine()` for explicit storage, shared resources, engine concurrency,
budgets, failure hooks, or lease timing.

- `pause` takes effect at the next durable boundary; `resume` continues it.
- `signal` releases the matching `waitFor`.
- `retry` invalidates selected failed/later work; `restart` creates a fresh run.
- `adopt` resumes stale code only when you explicitly assert compatibility.
- `priority` orders pending work; `delayMs` makes it eligible later.

## Storage adapters

```ts
interface StoredRun {
  readonly runId: string
  readonly state: RunState
  readonly events?: readonly unknown[]
}

interface StorageAdapter {
  init(): MaybePromise<void>
  createRun(run: StoredRun): MaybePromise<void>
  append(runId: string, event: unknown): MaybePromise<void>
  read(runId: string): MaybePromise<unknown[] | null>
  claim(runId: string, claim: {
    owner: string
    expiresAt: string
  }): MaybePromise<boolean>
  heartbeat(runId: string, claim: {
    owner: string
    expiresAt: string
  }): MaybePromise<boolean>
  list(options?: {
    status?: RunStatus
    workflow?: string
    limit?: number
  }): MaybePromise<StoredRun[]>
}

import { FileStorage, MemoryStorage } from '@nikhilverma/durably'
import { SqliteStorage } from '@nikhilverma/durably/sqlite'

const files = new FileStorage('./state')
const memory = new MemoryStorage()
const sqlite = new SqliteStorage('./durably.db')
```

`FileStorage` is inspectable and is the default. `MemoryStorage` is useful for
custom ephemeral engines. `SqliteStorage` uses `node:sqlite` on Node 22.5+.
A custom adapter's `append` must be atomic and ordered per run.

Each state record embeds the whole run, so `FileStorage` compacts a run's log to
its newest state record past ~1 MB — `new FileStorage(dir, { compactAboveBytes })`
— which is all a resume reads.

## Errors and advisories

All exported operational errors extend `DurablyError`: `ValidationError`,
`SerializationError`, `StepTimeoutError`, `PurityError`, `StaleRunError`,
`BudgetExceededError`, `CircuitOpenError`, `KeyConflictError`,
`RunCancelledError`, and `LeaseLostError`.

Advisory codes: `LOOP_SUGGESTED`, `FANOUT_CEILING`, `RETRY_STORM`,
`STASH_SUGGESTED`, `BUDGET_NEAR`, `WAITFOR_NO_TIMEOUT`, `SNAPSHOT_HEAVY`,
`SLOW_STEP_NO_TIMEOUT`, and `CONCURRENCY_CAPPED` — the last fires when a
fan-out is wider than the engine will run at once, and names both numbers.

Advisories are deduplicated by code, emitted only at severity crossings, capped
at ten active per run, and printed to stderr once unless
`advisories: 'silent'`. Follow each advisory's `hint` and `docs`.

## Practical ceilings

- In-run `parallel`: roughly 10,000 steps.
- Child runs: roughly 1,000-10,000 with `FileStorage`, roughly 100,000 with
  `SqliteStorage`. Beyond that, submit a provider batch as one durable step.
- Snapshot state must stay serializable and reasonably small.
- A step is the unit of durability: one killed at 99% yields nothing and re-runs
  whole. `stash` is the only checkpoint below step granularity.
- Checkpoints rewrite the whole run state, so the default interval scales with
  the run and bounds crash loss at one percent of completed steps. Batch fine
  work into coarse steps and keep per-item idempotency in your own ledger;
  `checkpointEvery: 1` buys exactness when steps cost money.
- Durably coordinates one local engine. It is not distributed execution.

## Executable examples

- Replay and idempotency:
  https://github.com/NikhilVerma/durably/blob/main/tests/unit/m1-kernel.test.ts
- Test engine and shadow replay:
  https://github.com/NikhilVerma/durably/blob/main/tests/unit/m2-test-engine.test.ts
- Retries, timeouts, compensation, concurrency, and budgets:
  https://github.com/NikhilVerma/durably/blob/main/tests/unit/m3-policies.test.ts
- Engine identity, pause/resume, priority, and failed-run operations:
  https://github.com/NikhilVerma/durably/blob/main/tests/unit/m5-engine.test.ts
- Parallelism, children, loops, waits, signals, timers, and stash:
  https://github.com/NikhilVerma/durably/blob/main/tests/unit/m6-topology-agent.test.ts
- Resources and SQLite:
  https://github.com/NikhilVerma/durably/blob/main/tests/unit/m7-resources-sqlite.test.ts
- Real process termination and resume:
  https://github.com/NikhilVerma/durably/blob/main/tests/integration/readme-hero-process.test.ts