Commit Graph
2 Commits
Author SHA1 Message Date
Daan MeijerandClaude Opus 5.5 7b436f3f77 Idle connections fetch capture ranges into the corpus
Test and deploy / test (push) Successful in 42s
Test and deploy / edge (push) Successful in 1m48s
Test and deploy / deploy (push) Successful in 1m6s
The spool only fetches what workers lease soon, in priority order, and the
tail never ends: after a reset the 1,400 capture ranges waited behind it
while 34 of 40 connections idled. Up to CORPUS_AHEAD (8) planned capture
ranges whose chunks aren't all stored are now targets after everything else,
fetched into the corpus only (no spool); a later lease reads them from disk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 20:11:57 +02:00
Daan MeijerandClaude Opus 5.5 9ea2669b2a Corpus: stored XOVER chunks for the grouping sweeps
Test and deploy / test (push) Successful in 43s
Test and deploy / edge (push) Successful in 1m48s
Test and deploy / deploy (push) Successful in 1m23s
Ranges marked `capture` keep their XOVER chunks in CORPUS_DIR
(corpus/<group>/<start>-<end>/, edge-bench's layout, range.json once
complete; only chunks that inflate). The prefetcher reads a stored chunk
instead of fetching it, so a reset re-indexes them without the news
server; a chunk a worker reports corrupt goes from the corpus too.

edge-bench gets --releases (release chunks without payload, with their
first subjects) and ships in the edge image; deploy/corpus_sweep.py marks
more ranges (capture), replays the corpus with the deployed build (replay)
and reports suspects and the change since the previous build (report).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 09:28:22 +02:00