0006. Read content-addressed chunks, write ProseMirror steps

• Status: Accepted
• Date: 2026-10-01 (recorded with @nextgensoftwares/folio-sync, commit 0544841)

Context

A 7,000-page book is tens of megabytes of JSON. Fetching and saving it whole makes opening slow, saving O(document) per batch of keystrokes, offline editing fragile, and concurrent editing lossy (last writer wins). The editor already has a precise description of every edit: ProseMirror steps.

Decision

Reads are chunked and content-addressed. A small manifest lists the document's chunks at a version: runs of top-level blocks cut at chapter boundaries, each identified by the hash of its content. A chunk URL never changes meaning, so HTTP caches, CDNs and IndexedDB keep chunks forever; an edit invalidates only the chunks it touched. Loading is progressive: manifest, then chunks (cache first, 6 in parallel, delivered in order), the editor usable after the first chunk, the rest appended outside undo history and collaboration.

Writes are ProseMirror steps tagged with the version they apply to (the central-authority protocol of prosemirror-collab): debounced, batched pushes; pull and rebase on conflict; an IndexedDB outbox of confirmed and unconfirmed steps for offline work. DocumentApi describes the five calls; createHttpApi maps them to REST; MemoryDocumentServer is the reference backend.

Consequences

• First page in ~0.1 s on the stress book; a warm reopen is one 12 KB manifest request; saves are O(edit).
• Concurrent editors converge by rebasing instead of overwriting.
• Offline edits survive reloads (base manifest + confirmed + unconfirmed steps).
• Servers must keep a step log and apply steps with the same schema; any out-of-band change must itself be steps, or clients won't rebase onto it.
• Loading must never produce steps (otherwise a partial load could be "saved"): appended content is excluded from history and collab.
• Integrity becomes checkable: a chunk must hash to its id, and the server can verify a client's result (0007).
• Page numbers are not part of storage (0008).