4 Commits

Include renames

Author	SHA1	Message	Date
Daan MeijerandClaude Opus 5.5	d0f3497ec9	Rebuild: completeness as indexed, file totals from any counter, pausable tails, progress by articles ... - releases.complete is computed with every batch (files >= the counter's total) instead of only when a release settles after 6 quiet hours: the 'complete' filter matched nothing in a fresh catalog. - A release's file total comes from any counter form (counterTotal: '(01/10)', 'File 27 of 132', …), not only [i/M]; the post rule's key keeps the [i/M] total, so keys are unchanged. Both implementations. - groups.tail: a group's live tail can be paused (its history and owner priority stay), e.g. to let other groups' history go first. - Stats: history by articles, and the current pass (history ranges before those deferred 2000 priorities behind) as its own figure; the header showed 0.1% of ranges counting 145k deferred ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-09 13:06:03 +02:00
Daan MeijerandClaude Opus 5.5	b862201991	Rebuild: the fragment merger ... Files cut by a range's edge are put together once the ranges around them are done (coordinator:run, every minute): a complete file is judged by the rules and written as a release chunk with probe candidates; one still missing parts while both neighbouring ranges are done is dropped and counted (incomplete/edge). A neighbour not planned yet counts as not done, except before the group's oldest article. BatchStore.writeReleases is the shared write path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-09 02:19:53 +02:00
Daan MeijerandClaude Opus 5.5	cd06f96f85	Rebuild: first end-to-end runs; prefetch per chunk, executable rule, wire fixes ... End to end on the workstation (coordinator:run, the API, edge:node, real NNTP): a.b.e-book.french and a.b.cd.image.playstation2.dvdiso ranges. - Prefetcher: every connection takes the next missing chunk (leased ranges first, in order); one connection per range (~26k lines/s) kept the worker waiting - executable rule: releases of bare executables (malware floods named after films and software) are dropped; they were 92% of the catalog rows of the test ranges, 21% of the segments - fragments per file (details once) and gzipped batch bodies; the JSON limit (1 MB by default) raised to 256 MB, empty strings kept (a rule count's key is '') - release_chunks.batch_from is a bigint (boneless article numbers) - tests get their own database; edge:bench --store times BatchStore: 1.3 s for a boneless range (1.24 ms per release chunk) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-09 01:07:11 +02:00
Daan MeijerandClaude Opus 5.5	20910647fa	Rebuild: the batch store (catalog upserts, write-once chunks, candidates, rule counts, fragments, resume point) ... A batch is stored in one transaction and only at its range's resume point, so a retried batch can't count twice. Garbage is judged before per-article: a dropped tool isn't worth reading bodies for either. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-09 00:21:17 +02:00