Rebuild spec: rules and recovery (counts per range and rule, rule versions, a sample)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

REBUILD.md

@@ -51,21 +51,35 @@ dropping garbage; 28 TB of compressed overview traffic. Throughput goal:
    aggregates once per batch (one upsert per release per batch), plus a
    small name index for search. ~115 M rows, ~50–100 GB.
 
-5. **Probe queue**. The candidates, in the catalog's database: one small row
+5. **Rules and recovery**. What a worker skips isn't stored (message-ids
+   are the bulk: garbage alone would be ~2.5 TB); a skipped range is read
+   again instead (headers are cheap: ~53 B per overview line compressed).
+   - *Per range, per rule and signature*: how many articles and files the
+     rule dropped (garbage per (tool, group), per-article random subjects,
+     obfuscated subjects, crossposts). A few rows per range.
+   - *Rule versions*: every range records the version of the rule set it ran
+     with. Retracting or changing a rule makes the ranges where it matched
+     stale; the coordinator leases them again for that rule only, and the
+     worker returns just what the rule had dropped.
+   - *Sample*: ~1% of what each rule drops (by a hash of the release key) is
+     kept with full file details and probe candidates, so a rule can be
+     checked, and its losses measured in phase 2, before it is retracted.
+
+6. **Probe queue**. The candidates, in the catalog's database: one small row
    each (release, kind, message-id, chunk reference), priority by the
    release's state (settled first) and kind (random names before readable
    ones). Phase 2 takes them from here; phase 1 only fills it.
 
-6. **Fragments**. Files crossing a range edge: kept by range and article
+7. **Fragments**. Files crossing a range edge: kept by range and article
    number; merged when the neighbouring range is done, then written as a
    chunk. Never staged per segment.
 
-7. **Settling**. A release is settled when the ranges covering its posting
+8. **Settling**. A release is settled when the ranges covering its posting
    span are done and no chunk arrived for a day: completeness is computed
    then (expected vs found files and segments). Phase 2 picks its work from
    settled releases.
 
-8. **NZB and API**. NZB built from a release's chunks; a minimal Newznab
+9. **NZB and API**. NZB built from a release's chunks; a minimal Newznab
    search (names) and get. Web UI later.
 
 ## Not in phase 1