The replay harness: a group's corpus through one carried window into a scratch catalog

replay:corpus runs edge-bench --group over a group's complete corpus ranges
in article order (one window, carried from range to range, finished after
the last range; edge/src/lease.rs holds the lease JSON the worker and the
bench share) and stores every batch through BatchStore as the worker
would, then reports releases finished, provisional, complete and the bytes
per header of the NVMe and HDD tables. Refuses a database not named
*_replay or *_scratch. The app image carries edge-bench; compose gets a
replay profile for homelab.

Design amended (REBUILD.md 1, 2, 5): the window is carried by one worker a
group, no block leases and no block merger; provisional writes stay for
the lease ends.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

REBUILD.md

@@ -390,13 +390,25 @@ little in a backfill. That churn is most of the WAL (0.6 TB a day) and the
 releases, a third of them one poster's image dump (27.7 M rows, LOG.md
 (102)).
 
-1. **Blocks.** A group's history is still fetched as ranges of 2 M
-   articles in chunks of 100 k (the spool, the corpus, the prefetcher stay),
-   but it is *leased* as blocks: consecutive ranges covering at least 30
-   days of posting time (`ranges.posted_low/high`), so the window below is
-   small against a block (≤ 3% of a block's releases touch an edge). A
-   small group is one block; boneless has a block a week or so. Blocks of
-   one group run in parallel on different workers, in any order.
+1. **One cursor a group (amended 2026-10-26; was: blocks).** A group's
+   history is still fetched as ranges of 2 M articles in chunks of 100 k
+   (the spool, the corpus, the prefetcher stay) and leased as ranges, but
+   a group's ranges are indexed in article order by one worker at a time,
+   and the window below is *carried* from range to range: the next range
+   finishes what the previous one left open. The first design leased
+   blocks of 30 days of posting time, ran a group's blocks in parallel and
+   joined their edges afterwards (a block merger). The user asked why
+   (LOG.md 2026-10-25 (110)): downloading stays parallel through the
+   prefetcher and a Rust thread parses faster than a connection delivers,
+   so one worker a group loses nothing but the end of a reset, when the
+   last big group runs alone. The merger, the block cutting and the
+   `blocks` lease go. Costs: one seam a group where the history meets the
+   tail (the releases in flight then end as two incomplete halves; history
+   runs forward in one pass a group, priority only between groups), and
+   the carried state must survive a restart (5 below). Measured on six
+   consecutive ranges of a.b.ath (2016): one range is 15 hours of posting,
+   so without the carry no release would finish inside a range (LOG.md
+   2026-10-26 (111)).
 
 2. **The window.** The worker reads a block in article order and keeps its
    open files and releases in memory (counters, names, sizes; the segment
@@ -412,15 +424,16 @@ releases, a third of them one poster's image dump (27.7 M rows, LOG.md
    decides later for those without a counter), never updated by the header
    pass again. Incomplete releases are kept (user, 2026-10-24). A fill
    posted after the window is a release of its own (the judge's `repeated`
-   rule already knows reposts). At the block's end the releases the window
-   hasn't closed on are written **provisionally** (`settled_at` null) and
-   flagged `edge`, as are the finished releases posted within 24 h of the
-   block's first frontier; the block merger (replaces the fragment merger)
-   joins the edge releases of adjacent blocks of a group by identity key
-   once both are done, finishes them and clears the flag. The settler goes:
-   `settled_at` is the finish. The **tail** needs no path of its own: a
-   tail lease's open releases at its end are such provisional edges, the
-   next lease's continuation joins them.
+   rule already knows reposts). At a lease's end (the tail's poll, a worker
+   stopping; amended 2026-10-26, was: the block's end) the releases the
+   window hasn't closed on are written **provisionally** (`settled_at`
+   null, `finished: false`, flagged `edge`): a release being uploaded
+   shows in the catalog, and the carried window finishes the same row
+   later, since the release key is the identity, the group and the first
+   article; the store's finish path adds the files not sent before. No
+   merger: the carry does the joining. The settler goes: `settled_at` is
+   the finish. The **tail** needs no path of its own: its lease ends leave
+   such provisional releases, the next lease's continuation finishes them.
 
 3. **Albums.** Files of one poster in one owner group whose subjects differ
    only by a number sequence, or share a directory-like prefix, posted
@@ -445,7 +458,16 @@ releases, a third of them one poster's image dump (27.7 M rows, LOG.md
    a day; at a file per ~50 articles that is ~1 M open files of ~100 B in
    memory per block, and their segment lists (~25 B each) in the block's
    spool file, read back when a release is finished. Per worker thread a
-   few hundred MB at worst.
+   few hundred MB at worst. The carried window (amended 2026-10-26) has to
+   survive a restart (deploys restart the workers daily): its counters and
+   names are small (kilobytes for most groups, ~100 MB for the busiest),
+   the open files' segment lists are the bulk (~1.25 GB for a day of the
+   busiest group). A snapshot at every 2 M range would be ~600 B/article
+   of writes, so the state is snapshotted at a lease's end and on a timer,
+   to a file on bulk, and a crash redoes the ranges since the last snapshot
+   from the corpus or the prefetch cache: the deterministic keys make the
+   redo idempotent. Not built yet; the replay harness (8) keeps a group's
+   window in one process.
 
 6. **The corpus stays as it is.** Measured on 300 unfiltered chunks
    (30 M lines, LOG.md (102)): 43% of lines lack a "(k/N)" counter, and of
@@ -603,11 +625,37 @@ indexer; CI runs `edge/bench-fixtures.sh` instead), the `incomplete` count
 window holds a day of a group's segments in memory (the busiest group's
 ~50 M); the spool file per block of the design is not built yet.
 
-Next: albums in the edge (design 3), block leases and the planner (then
-the store keys by the leased block and the lease columns leave `ranges`;
-until then every range end leaves provisional edges), the block merger
-(joins and finishes the edges), the prober writing `release_texts` kind 2,
-the corpus replay harness.
+### The replay harness (built 2026-10-26)
+
+`node ace replay:corpus --group <name> [--dir CORPUS_DIR] [--from article]
+[--limit ranges] [--batch-chunks 5] [--no-finish]` (commands/
+replay_corpus.ts; design 8): a group's complete corpus ranges in article
+order through one window in the Rust edge (`edge-bench --group <dir>
+--rules rules.json --wire -`: the window carried from range to range, the
+end finished after the last range; `edge/src/lease.rs` has the lease JSON
+the worker and the bench share) and the batch store into a scratch
+catalog, as the worker would store them: each range leased to a node named
+`replay`, its batches stored at its resume point (pulled from the bench's
+stdout as the store keeps up), the range finished when its last batch is
+in. It refuses a database not named `*_replay` or `*_scratch`.
+The rules are the scratch catalog's current set and group priorities:
+copy `groups` and `rule_sets` from the catalog for a faithful replay. The
+report: releases finished, provisional, edges; complete, incomplete,
+undecided; readable, with par2; files, segments as a share of the
+headers; bytes per header of the NVMe tables (releases, release_texts,
+posters, group_sets, rule counts and samples) and of those on
+CHUNK_TABLESPACE (release_chunks, probe_candidates, release_inspections).
+The app image carries `edge-bench` for it (Dockerfile); on homelab the
+scratch catalog is a second database on bulk (`usenet_replay`, migrated
+with the voyager-3 image, `groups` and `rule_sets` copied over), the
+corpus mounted read-only: not run there yet (LOG.md 2026-10-26 (111) has
+the first local numbers).
+
+Next: the replay on homelab over the 2016 window (the NVMe bytes per
+header for real), albums in the edge (design 3), the ordered range leases
+with the carried window in the edge-node and its snapshot (design 1 and
+5), the prober writing `release_texts` kind 2. Gone from the list: block
+leases, the planner and the block merger (design 1, amended).
 
 ## Decisions