REBUILD.md: voyager 2's final numbers for group sets and albums (LOG 112)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
f437929181
commit
219fe4fd4c
2 files changed
+10
-3
No files matched your search
+9
-2
@@ -441,11 +441,18 @@ releases, a third of them one poster's image dump (27.7 M rows, LOG.md
|
||||
shared template (or the prefix), its files kept by name so each stays
|
||||
findable; at least two files, else the single stays a release. This is
|
||||
for subjects that name no release (today's `post`/`stem` singles); a
|
||||
poster's ten different cracks in a day stay ten releases.
|
||||
poster's ten different cracks in a day stay ten releases. The stake,
|
||||
from voyager 2's final analysis (LOG.md (112)): 77% of its 83.7 M
|
||||
releases had one file, 50% one segment (0.3% of the segments);
|
||||
a.b.pictures.erotica.porn alone was 27.7 M single-file releases of one
|
||||
poster. A counter total shared by many stems of one poster within the
|
||||
window is a batch counter, not a file count (Yenc-PP-A&A's [n/68067]):
|
||||
`file_total` null for them, not 68,067 incomplete singles.
|
||||
|
||||
4. **A slimmer catalog.** `releases` loses its text copies: `poster_id` →
|
||||
`posters(id)` (the table gets an id; 33 B → 4), `group_set_id` →
|
||||
`group_sets` (the ~2,600 distinct group combinations; 85 B jsonb → 4),
|
||||
`group_sets` (the distinct group combinations: 139,927 in voyager 2's
|
||||
83.7 M releases, LOG.md (112); 85 B jsonb → 4),
|
||||
`release_key` and `identity_key` as 20-byte `bytea` (41 B each → 20),
|
||||
`source` a smallint. ~150 of ~430 B a row; the poster and key indexes
|
||||
shrink with them (~12 GB of 50 today); `releases_category_newest_idx`
|
||||
|
||||
@@ -7,7 +7,7 @@ import env from '#start/env'
|
||||
* inspections, candidates, par2 data, file hashes, verdicts, merges, rule counts and samples; what survives:
|
||||
* groups, ranges, the corpus, probe_cache, spots, nntp_shares, nodes, users), so it refuses to run on a catalog
|
||||
* with releases unless VOYAGER3_RESET=1 (the user's go-ahead; resets are paused since 2026-10-12). Then:
|
||||
* - posters get an id; group_sets holds each distinct group combination once (~2,600); releases is created anew
|
||||
* - posters get an id; group_sets holds each distinct group combination once (~140k); releases is created anew
|
||||
* without its text copies (poster_id, group_set_id, the keys as 20-byte bytea, source a smallint), 8-byte
|
||||
* columns first, with `edge` (still open at its block's end, for the block merger) and settled_at as the finish
|
||||
* (null: a tail release still provisional). The category and judge triggers follow the new columns;
|
||||
|
||||
Reference in new issue
Block a user