Commit Graph
2 Commits
Author SHA1 Message Date
Daan MeijerandClaude Fable 5.1 1f6e3316ec The window worker: group leases, the stream serving the corpus, the snapshot through the API (design 1 and 5)
Test and deploy / test (push) Successful in 55s
Test and deploy / edge (push) Successful in 2m9s
Test and deploy / deploy (push) Successful in 1m16s
A worker thread leases a group and takes its ranges in article order through
one carried window; the coordinator keeps one cursor a group (group_leases,
migration 6500) and hands out the next range, the rules and nothing about
where the chunks are: the range stream serves a stored range's chunks from the
corpus file and a live one's from the spool. The window's snapshot (bincode:
the frontier, the open files, releases and albums, the upload numbers, the
batch-counter table) goes to the coordinator every EDGE_SNAPSHOT_MINUTES and
when the group is given back, and the next holder restores it; a takeover
after LEASE_TTL_SECONDS plans the ranges from the snapshot's article again.
The prefetcher runs ahead of held groups' cursors and skips chunks the corpus
has. LEASE_MIN_PRIORITY..LEASE_MAX_PRIORITY bound what is leased. edge-node
holds groups by default; --ranges keeps the old per-range path.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-26 18:22:18 +01:00
Daan MeijerandClaude Fable 5.1 72e973d006 The replay harness: a group's corpus through one carried window into a scratch catalog
Test and deploy / test (push) Successful in 48s
Test and deploy / edge (push) Successful in 1m43s
Test and deploy / deploy (push) Skipped
replay:corpus runs edge-bench --group over a group's complete corpus ranges
in article order (one window, carried from range to range, finished after
the last range; edge/src/lease.rs holds the lease JSON the worker and the
bench share) and stores every batch through BatchStore as the worker
would, then reports releases finished, provisional, complete and the bytes
per header of the NVMe and HDD tables. Refuses a database not named
*_replay or *_scratch. The app image carries edge-bench; compose gets a
replay profile for homelab.

Design amended (REBUILD.md 1, 2, 5): the window is carried by one worker a
group, no block leases and no block merger; provisional writes stay for
the lease ends.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-26 00:27:09 +01:00