REBUILD.md: voyager 2's final numbers for group sets and albums (LOG 112)
Test and deploy / test (push) Successful in 47s
Test and deploy / edge (push) Successful in 1m47s
Test and deploy / deploy (push) Skipped

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Fable 5.1 committed 2026-10-26 08:12:03 +01:00
1 parent f437929181
commit 219fe4fd4c
2 files changed
+10 -3

No files matched your search

+9 -2
View File
@@ -441,11 +441,18 @@ releases, a third of them one poster's image dump (27.7 M rows, LOG.md
shared template (or the prefix), its files kept by name so each stays
findable; at least two files, else the single stays a release. This is
for subjects that name no release (today's `post`/`stem` singles); a
poster's ten different cracks in a day stay ten releases.
poster's ten different cracks in a day stay ten releases. The stake,
from voyager 2's final analysis (LOG.md (112)): 77% of its 83.7 M
releases had one file, 50% one segment (0.3% of the segments);
a.b.pictures.erotica.porn alone was 27.7 M single-file releases of one
poster. A counter total shared by many stems of one poster within the
window is a batch counter, not a file count (Yenc-PP-A&A's [n/68067]):
`file_total` null for them, not 68,067 incomplete singles.
4. **A slimmer catalog.** `releases` loses its text copies: `poster_id` →
`posters(id)` (the table gets an id; 33 B → 4), `group_set_id` →
`group_sets` (the ~2,600 distinct group combinations; 85 B jsonb → 4),
`group_sets` (the distinct group combinations: 139,927 in voyager 2's
83.7 M releases, LOG.md (112); 85 B jsonb → 4),
`release_key` and `identity_key` as 20-byte `bytea` (41 B each → 20),
`source` a smallint. ~150 of ~430 B a row; the poster and key indexes
shrink with them (~12 GB of 50 today); `releases_category_newest_idx`
@@ -7,7 +7,7 @@ import env from '#start/env'
* inspections, candidates, par2 data, file hashes, verdicts, merges, rule counts and samples; what survives:
* groups, ranges, the corpus, probe_cache, spots, nntp_shares, nodes, users), so it refuses to run on a catalog
* with releases unless VOYAGER3_RESET=1 (the user's go-ahead; resets are paused since 2026-10-12). Then:
* - posters get an id; group_sets holds each distinct group combination once (~2,600); releases is created anew
* - posters get an id; group_sets holds each distinct group combination once (~140k); releases is created anew
* without its text copies (poster_id, group_set_id, the keys as 20-byte bytea, source a smallint), 8-byte
* columns first, with `edge` (still open at its block's end, for the block merger) and settled_at as the finish
* (null: a tail release still provisional). The category and judge triggers follow the new columns;