REBUILD.md is the concept for what runs; the phase 2 backlog
Test and deploy / test (push) Successful in 44s
Test and deploy / deploy (push) Failing after 1m4s

From "draft for approval" to the concept of voyager 2 (voyager 1's
CONCEPT.md and D1-D42 are superseded). Phase 2: what's built (the
prober), and voyager 1's features not yet here: spots, NZB import,
passwords, SFV, RAR walks, 7z listings, body naming of obfuscated
subjects, after-probe garbage drops; where its exported data is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-11 19:48:49 +02:00
1 parent 93eaec8d27
commit 34fea342f5
1 file changed
+30 -5
+30 -5
View File
@@ -1,4 +1,9 @@
# Rebuild: header-first indexer (spec, draft for approval)
# Voyager 2: header-first indexer (concept)
The concept for what runs (since 2026-10-11; it started as the spec of a
rebuild, then a draft for approval). Voyager 1's CONCEPT.md and its
decisions D1–D42 are superseded; its domain model still holds, and its
features not yet here are the phase 2 backlog below.
Branch `rebuild`. Phase 1 finds releases from headers and makes their NZBs.
Everything learned from articles (names from yEnc headers, par2, archive
@@ -203,11 +208,31 @@ of other groups whose counts show crossposts skipped for it (part 5). Start:
the 1000 largest `alt.binaries.*` groups by LIST ACTIVE (~266 k ranges of
2 M articles; boneless alone 80 k).
## Not in phase 1
## Phase 2
Inspection and archive walks, par2/SFV naming, spots, NZB imports, the
after-probe garbage deletes, live indexing (the current indexer keeps
running and does those meanwhile).
Built: the prober (2026-10-11; `probe:run`, LOG.md 2026-10-11 (2)). It
reads the probe queue's first articles (`lone`, `par2`, `archive`) and
stores per release whether it's passworded and its inner name, which search,
the web interface and Newznab use.
**Backlog: voyager 1's features not in voyager 2** (CONCEPT.md decisions;
voyager 1's data exported to `/mnt/bulk/usenet-rebuild/v1-export/` on
homelab: `voyager1-tables.dump` with 2.76 M spots, upload signatures,
signature probes, dropped releases and groups; `voyager1-release-passwords.csv`
with 68,032 releases with a password):
| Feature | Voyager 1 | In voyager 2 |
|---|---|---|
| Spotnet spots | D29 | not yet; 2.76 M spots in the export |
| Posted NZBs read into releases (also NZBs inside archives) | D26, D28 | not yet |
| Passwords (from spots, NZB meta, names) | D29, D26 | only *whether* passworded (the prober); the export's 68 k passwords not imported |
| SFV grouping and naming | D34 | not yet (SFV files aren't probe candidates) |
| Complete RAR listings by walking volume headers | D40 | no: the prober reads the first volume's first article only |
| 7z listings | D19 | no: `7z-tail` candidates are queued, not probed |
| Names from yEnc bodies for obfuscated subjects | D13, D21, D36 | no: obfuscated and per-article subjects are only counted (part 5); their message-ids aren't stored, so a body path needs those ranges read again |
| Garbage dropped after probing (no real name, no password) | D42 (its second filter) | not yet; the header-time garbage rules are in (part 5) |
Live indexing (the tail) runs since 2026-10-09.
## Decisions