Voyager 3 schema: the slim catalog (migration 6000, the reset) and blocks (6100)
Test and deploy / test (push) Failing after 50s
Test and deploy / edge (push) Successful in 1m53s
Test and deploy / deploy (push) Skipped

releases rebuilt without its text copies: poster_id, group_set_id,
bytea keys, source smallint, edge, settled_at as the finish. posters by
id, group_sets. blocks as the lease unit, ranges.block_id. fragments,
range_blocks, release_final, the settler and the fragment merger gone;
release_chunks, rule_counts and rule_samples keyed by block. Cold tables
on CHUNK_TABLESPACE. Migration 6000 truncates the catalog and refuses
without VOYAGER3_RESET=1.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Fable 5.1 committed 2026-10-25 13:44:54 +01:00
1 parent 1a448f37f9
commit 346493b6c3
17 files changed
+527 -545

No files matched your search

+49
View File
@@ -472,6 +472,55 @@ releases, a third of them one poster's image dump (27.7 M rows, LOG.md
verdicts, file hashes. Resets are paused (2026-10-12); this one waits
for the user's go-ahead, and for the new NVMe.
### Schema (built 2026-10-25; migrations 6000 and 6100)
Migration 6000 *is* the reset: it truncates every table the indexing
recomputes and refuses to run on a catalog with releases unless
`VOYAGER3_RESET=1`, so merging `voyager-3` into `rebuild` is the go-ahead.
Survives: `groups`, `ranges` (planned again from their start, with their
priorities, `capture`, `est_date`, `stored_at`, `plain_chunks` and measured
`posted_low/high`), the corpus, `probe_cache`, `spots` (unlinked from
releases; the inspector matches them again), `nntp_shares`, `nodes`,
`users`, the rule sets.
- `releases`: created anew, 8-byte columns first: `id`, `posted_at`,
`last_posted_at`, `first_article`, `last_article`, `segments`, `bytes`,
`settled_at` (the finish; null while a tail release is provisional),
`inspected_at`, `first_seen_at`; `group_id`, `group_set_id`, `poster_id`,
`file_total`, `files`, `chunks`, `category`; `source smallint`
(`lib/release/source.ts`: 1 subject, 2 post, 3 stem, 4 spot, 5 nzb, 6
album); `readable`, `has_par2`, `complete`, `passworded`, `sidecar_only`,
`edge` (open at its block's end); `release_key` and `identity_key` as
20-byte `bytea`; `name`, `inner_name`. Gone: `poster`, `groups`,
`last_seen_at`. Indexes: pkey, release_key unique, (identity_key,
group_id), name and inner-name trigram, newest, (poster_id, newest),
passworded, `(group_id, identity_key) WHERE edge` for the block merger,
`id WHERE settled_at IS NULL` for the provisional ones. The category
trigger reads the groups from `group_sets`; the judge's dirty trigger
watches `poster_id` instead of `poster`.
- `posters(id serial, poster unique)` with its trigram index;
`group_sets(id serial, groups jsonb unique)`, a sorted array of names.
- `blocks` (6100): `group_id`, `start_id`, `end_id`, `next_id` (the resume
point), `posted_low/high`, `status`, `priority`, `node_id`, `attempts`,
`rule_version`, `headers`, `releases`, `leased_at`, `heartbeat_at`,
`done_at`, `error`; indexes todo (priority, id) and leased.
`ranges.block_id` points at the block; the ranges' own lease columns stay
until the leases move (then a migration drops them).
- `release_chunks`, `rule_counts`, `rule_samples`: `range_id` → `block_id`.
- Gone: `fragments`, `range_blocks`, `release_final()` (final = finished:
`settled_at IS NOT NULL`), the settler, the fragment merger,
`releases_category_newest_idx`.
- `release_inspections` and `probe_candidates` on `CHUNK_TABLESPACE` (the
HDD) with their indexes. Note LOG.md 2026-10-19 (81): the judge's fake
rule read all of `release_inspections` every round, 17 min on the HDD;
the judge must read it by release or keep what it needs on the NVMe.
Next: the batch store (write-once releases, posters and group sets by id,
bytea keys, blocks), then the window and albums in the Rust edge, block
leases and the block merger, the corpus replay harness. Until the batch
store is done the 38 functional tests that write the old columns fail on
`voyager-3`.
## Decisions
a. Chunk payload: a write-once `bytea` table in Postgres, partitioned by