4 Commits

Include renames

Author	SHA1	Message	Date
Daan MeijerandClaude Opus 5.5	2bbabc08de	corpus_sweep capture: budget for the marked ranges still filling ... Only the bytes on disk were checked: at 296.6 GB with 2,300 ranges complete it marked 100 more (~310 GB). The marked ranges not complete count at the stored ranges' mean size, and no more are marked than fit under 300 GB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-13 05:55:26 +02:00
Daan MeijerandClaude Opus 5.5	8e07478659	Counters: a total padded wider than its index is none; report diff on shared ranges ... - "Very Best Of The Who [01/010]", "[02/020]", "[07/070]": a posting tool's total of index×10, so every track had a total of its own and was a release of its own. A total zero-padded wider than its index is no counter ("[001/010]" still is). - corpus_sweep.py report compares builds over the ranges both replayed (the corpus grows between builds). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-12 10:12:28 +02:00
Daan MeijerandClaude Opus 5.5	3d0dc6ca1f	The par2 merge: releases whose files one par2 set describes become one ... The prober stores, per probed article that starts a file, the MD5 of its first 16 KiB (file_hashes), and per par2 body the set's whole file list (par2_files, par2_sets): FileDesc holds the same hash per file, whatever the file was renamed to. releases:merge (the merger service, no NNTP) merges every minute the releases of one group, posted within 48 hours of a release holding the set's par2, whose files the set describes, into the one with the most files: chunks, probe candidates, inspections, hashes and counts move; the par2's shared name when none was readable; release_merges keeps what went where. Obfuscated uploads whose files were each a release of their own ("[9/20] - "14f5999a…"", hep: 13,232) become one. Also: corpus_sweep.py counts the marked ranges without psql's command tag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-12 09:37:35 +02:00
Daan MeijerandClaude Opus 5.5	9ea2669b2a	Corpus: stored XOVER chunks for the grouping sweeps ... Ranges marked `capture` keep their XOVER chunks in CORPUS_DIR (corpus/<group>/<start>-<end>/, edge-bench's layout, range.json once complete; only chunks that inflate). The prefetcher reads a stored chunk instead of fetching it, so a reset re-indexes them without the news server; a chunk a worker reports corrupt goes from the corpus too. edge-bench gets --releases (release chunks without payload, with their first subjects) and ships in the edge image; deploy/corpus_sweep.py marks more ranges (capture), replays the corpus with the deployed build (replay) and reports suspects and the change since the previous build (report). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-12 09:28:22 +02:00