diff --git a/REBUILD.md b/REBUILD.md index 0b32822..6415663 100644 --- a/REBUILD.md +++ b/REBUILD.md @@ -1,4 +1,9 @@ -# Rebuild: header-first indexer (spec, draft for approval) +# Voyager 2: header-first indexer (concept) + +The concept for what runs (since 2026-10-11; it started as the spec of a +rebuild, then a draft for approval). Voyager 1's CONCEPT.md and its +decisions D1–D42 are superseded; its domain model still holds, and its +features not yet here are the phase 2 backlog below. Branch `rebuild`. Phase 1 finds releases from headers and makes their NZBs. Everything learned from articles (names from yEnc headers, par2, archive @@ -203,11 +208,31 @@ of other groups whose counts show crossposts skipped for it (part 5). Start: the 1000 largest `alt.binaries.*` groups by LIST ACTIVE (~266 k ranges of 2 M articles; boneless alone 80 k). -## Not in phase 1 +## Phase 2 -Inspection and archive walks, par2/SFV naming, spots, NZB imports, the -after-probe garbage deletes, live indexing (the current indexer keeps -running and does those meanwhile). +Built: the prober (2026-10-11; `probe:run`, LOG.md 2026-10-11 (2)). It +reads the probe queue's first articles (`lone`, `par2`, `archive`) and +stores per release whether it's passworded and its inner name, which search, +the web interface and Newznab use. + +**Backlog: voyager 1's features not in voyager 2** (CONCEPT.md decisions; +voyager 1's data exported to `/mnt/bulk/usenet-rebuild/v1-export/` on +homelab: `voyager1-tables.dump` with 2.76 M spots, upload signatures, +signature probes, dropped releases and groups; `voyager1-release-passwords.csv` +with 68,032 releases with a password): + +| Feature | Voyager 1 | In voyager 2 | +|---|---|---| +| Spotnet spots | D29 | not yet; 2.76 M spots in the export | +| Posted NZBs read into releases (also NZBs inside archives) | D26, D28 | not yet | +| Passwords (from spots, NZB meta, names) | D29, D26 | only *whether* passworded (the prober); the export's 68 k passwords not imported | +| SFV grouping and naming | D34 | not yet (SFV files aren't probe candidates) | +| Complete RAR listings by walking volume headers | D40 | no: the prober reads the first volume's first article only | +| 7z listings | D19 | no: `7z-tail` candidates are queued, not probed | +| Names from yEnc bodies for obfuscated subjects | D13, D21, D36 | no: obfuscated and per-article subjects are only counted (part 5); their message-ids aren't stored, so a body path needs those ranges read again | +| Garbage dropped after probing (no real name, no password) | D42 (its second filter) | not yet; the header-time garbage rules are in (part 5) | + +Live indexing (the tail) runs since 2026-10-09. ## Decisions