REBUILD.md is the concept for what runs; the phase 2 backlog
From "draft for approval" to the concept of voyager 2 (voyager 1's CONCEPT.md and D1-D42 are superseded). Phase 2: what's built (the prober), and voyager 1's features not yet here: spots, NZB import, passwords, SFV, RAR walks, 7z listings, body naming of obfuscated subjects, after-probe garbage drops; where its exported data is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
93eaec8d27
commit
34fea342f5
1 file changed
+30
-5
+30
-5
@@ -1,4 +1,9 @@
|
||||
# Rebuild: header-first indexer (spec, draft for approval)
|
||||
# Voyager 2: header-first indexer (concept)
|
||||
|
||||
The concept for what runs (since 2026-10-11; it started as the spec of a
|
||||
rebuild, then a draft for approval). Voyager 1's CONCEPT.md and its
|
||||
decisions D1–D42 are superseded; its domain model still holds, and its
|
||||
features not yet here are the phase 2 backlog below.
|
||||
|
||||
Branch `rebuild`. Phase 1 finds releases from headers and makes their NZBs.
|
||||
Everything learned from articles (names from yEnc headers, par2, archive
|
||||
@@ -203,11 +208,31 @@ of other groups whose counts show crossposts skipped for it (part 5). Start:
|
||||
the 1000 largest `alt.binaries.*` groups by LIST ACTIVE (~266 k ranges of
|
||||
2 M articles; boneless alone 80 k).
|
||||
|
||||
## Not in phase 1
|
||||
## Phase 2
|
||||
|
||||
Inspection and archive walks, par2/SFV naming, spots, NZB imports, the
|
||||
after-probe garbage deletes, live indexing (the current indexer keeps
|
||||
running and does those meanwhile).
|
||||
Built: the prober (2026-10-11; `probe:run`, LOG.md 2026-10-11 (2)). It
|
||||
reads the probe queue's first articles (`lone`, `par2`, `archive`) and
|
||||
stores per release whether it's passworded and its inner name, which search,
|
||||
the web interface and Newznab use.
|
||||
|
||||
**Backlog: voyager 1's features not in voyager 2** (CONCEPT.md decisions;
|
||||
voyager 1's data exported to `/mnt/bulk/usenet-rebuild/v1-export/` on
|
||||
homelab: `voyager1-tables.dump` with 2.76 M spots, upload signatures,
|
||||
signature probes, dropped releases and groups; `voyager1-release-passwords.csv`
|
||||
with 68,032 releases with a password):
|
||||
|
||||
| Feature | Voyager 1 | In voyager 2 |
|
||||
|---|---|---|
|
||||
| Spotnet spots | D29 | not yet; 2.76 M spots in the export |
|
||||
| Posted NZBs read into releases (also NZBs inside archives) | D26, D28 | not yet |
|
||||
| Passwords (from spots, NZB meta, names) | D29, D26 | only *whether* passworded (the prober); the export's 68 k passwords not imported |
|
||||
| SFV grouping and naming | D34 | not yet (SFV files aren't probe candidates) |
|
||||
| Complete RAR listings by walking volume headers | D40 | no: the prober reads the first volume's first article only |
|
||||
| 7z listings | D19 | no: `7z-tail` candidates are queued, not probed |
|
||||
| Names from yEnc bodies for obfuscated subjects | D13, D21, D36 | no: obfuscated and per-article subjects are only counted (part 5); their message-ids aren't stored, so a body path needs those ranges read again |
|
||||
| Garbage dropped after probing (no real name, no password) | D42 (its second filter) | not yet; the header-time garbage rules are in (part 5) |
|
||||
|
||||
Live indexing (the tail) runs since 2026-10-09.
|
||||
|
||||
## Decisions
|
||||
|
||||
|
||||
Reference in new issue
Block a user