diff --git a/REBUILD.md b/REBUILD.md index 9897a75..6fc6b11 100644 --- a/REBUILD.md +++ b/REBUILD.md @@ -441,11 +441,18 @@ releases, a third of them one poster's image dump (27.7 M rows, LOG.md shared template (or the prefix), its files kept by name so each stays findable; at least two files, else the single stays a release. This is for subjects that name no release (today's `post`/`stem` singles); a - poster's ten different cracks in a day stay ten releases. + poster's ten different cracks in a day stay ten releases. The stake, + from voyager 2's final analysis (LOG.md (112)): 77% of its 83.7 M + releases had one file, 50% one segment (0.3% of the segments); + a.b.pictures.erotica.porn alone was 27.7 M single-file releases of one + poster. A counter total shared by many stems of one poster within the + window is a batch counter, not a file count (Yenc-PP-A&A's [n/68067]): + `file_total` null for them, not 68,067 incomplete singles. 4. **A slimmer catalog.** `releases` loses its text copies: `poster_id` → `posters(id)` (the table gets an id; 33 B → 4), `group_set_id` → - `group_sets` (the ~2,600 distinct group combinations; 85 B jsonb → 4), + `group_sets` (the distinct group combinations: 139,927 in voyager 2's + 83.7 M releases, LOG.md (112); 85 B jsonb → 4), `release_key` and `identity_key` as 20-byte `bytea` (41 B each → 20), `source` a smallint. ~150 of ~430 B a row; the poster and key indexes shrink with them (~12 GB of 50 today); `releases_category_newest_idx` diff --git a/database/migrations/1770000006000_voyager3_catalog.ts b/database/migrations/1770000006000_voyager3_catalog.ts index d08a8bb..a2eacd8 100644 --- a/database/migrations/1770000006000_voyager3_catalog.ts +++ b/database/migrations/1770000006000_voyager3_catalog.ts @@ -7,7 +7,7 @@ import env from '#start/env' * inspections, candidates, par2 data, file hashes, verdicts, merges, rule counts and samples; what survives: * groups, ranges, the corpus, probe_cache, spots, nntp_shares, nodes, users), so it refuses to run on a catalog * with releases unless VOYAGER3_RESET=1 (the user's go-ahead; resets are paused since 2026-10-12). Then: - * - posters get an id; group_sets holds each distinct group combination once (~2,600); releases is created anew + * - posters get an id; group_sets holds each distinct group combination once (~140k); releases is created anew * without its text copies (poster_id, group_set_id, the keys as 20-byte bytea, source a smallint), 8-byte * columns first, with `edge` (still open at its block's end, for the block merger) and settled_at as the finish * (null: a tail release still provisional). The category and judge triggers follow the new columns;