Nothing enters the catalog without a usable name: the pending store (decision f amended)
Test and deploy / test (push) Successful in 56s
Test and deploy / edge (push) Successful in 2m2s
Test and deploy / deploy (push) Skipped

An unnamed release (no readable file name) is written to pending_releases
(migration 6300, the HDD) with its keys, counts, article span, upload tool
and probe targets, instead of releases, release_texts, posters and a chunk.
The edge sends it with an empty payload and its tool; the batch store splits
a batch by readable. The pending ladder (probes cheapest first, signature
verdicts, the move into the catalog) comes next. The replay report shows the
pending releases.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Fable 5.1 committed 2026-10-26 13:18:14 +01:00
1 parent 6860e6ccaf
commit 9cf85e09d8
8 files changed
+286 -15

No files matched your search

+40 -9
View File
@@ -550,6 +550,13 @@ releases; the inspector matches them again), `nntp_shares`, `nodes`,
`release_texts`. The category trigger reads the groups from
`group_sets`; the judge's dirty trigger watches `poster_id` instead of
`poster`.
- `pending_releases` (migration 6300, the HDD): the unnamed releases
outside the catalog (decision f): `release_key` (primary), `identity_key`,
`group_id`, `name` (the random stem), `poster` text, the posting span,
`first_article`, `last_article`, counts, `partial`, `finished`, `tool`,
`candidates` jsonb (the probe targets by effort), `status` (0 waiting, 1
named, 2 discarded), `tries`, `created_at`, `probed_at`; indexed by
(group, tool, status, created_at) for the ladder.
- `release_texts(release_id, kind smallint, text varchar(255))` (6200),
primary key on all three, one trigram index (fastupdate): every string
search matches against, appended by whoever learns one and never an
@@ -682,10 +689,12 @@ until the window is done (LOG.md 2026-10-26 (111) has the first local
numbers, (112) the homelab run).
Next: the replay on homelab over the 2016 window (the NVMe bytes per
header for real), the ordered range leases with the carried window in
the edge-node and its snapshot (design 1 and 5), the prober writing
`release_texts` kind 2, random-named singles without a chunk (decision f:
2.3 M of the first replay's 4.0 M singles are "Bob"'s 32-hex posts). Gone from the list: block
header for real), the pending ladder and the signature learner (decision
f, amended 2026-10-26: the prober working `pending_releases` cheapest
first, verdicts per tool and group, a named release moved into the
catalog with its segments read again), the ordered range leases with the
carried window in the edge-node and its snapshot (design 1 and 5), the
prober writing `release_texts` kind 2. Gone from the list: block
leases, the planner and the block merger (design 1, amended).
## Decisions
@@ -697,11 +706,33 @@ c. Group list: editable, owner by priority number (above). **Accepted.**
d. Reposts: separate releases in phase 1, merged in phase 2. **Accepted.**
e. Headers through the coordinator's relay, with prefetch (part 1).
**Accepted.**
f. Random-named releases (no file with a readable name) keep no chunk in
the header pass: their file keys and probe candidates only, until
phase 2 names them (bodies) and fetches their segments again. The full
history is ~3 TB with them, ~1.2–1.4 TB without (725 M of the 1.25 B
kept files; LOG.md 2026-10-09). **Accepted** 2026-10-09, not built yet.
f. **Nothing enters the catalog without a usable name** (amended
2026-10-26, the user's direction; before: random-named releases keep
no chunk but their rows). A release the window closed on whose files
all have random names is no release yet: it goes to `pending_releases`
(migration 6300, the HDD) with its keys, counts, article span, upload
tool and probe targets, no row in `releases`, `release_texts` or
`posters`, no chunk. The pending ladder works them **from least to most
effort**: (0) no fetch: a signature verdict (tool and group, poster)
learned from earlier probes, a spot or NZB that names the message-ids;
(1) a par2 main file's first article (it names the files); (2) the first
article of the first file (the yEnc name, an archive's listing in its
first 16 KiB); (3) a 7z's tail; then nothing more. A probe that yields
a usable name moves the release into the catalog, its segments read
again from the corpus (the range stored) or the server; one that
yields none counts against its signature, and a signature with
≥100 probes and ≤0.5% named (voyager 2's garbage learner, LOG.md
2026-10-08 (5)) is **discarded** from then on, a sample of it still
probed to notice a change. The reason, in the 2016 replay: one poster's
hash-named pieces ("Bob", 11 aliases, 3.2 M releases in five small
groups) were 76% of the rows and three quarters of the NVMe bytes for
11% of the segments and nothing anyone can find; the full history has
725 M random-named files (LOG.md 2026-10-09). Built 2026-10-26: the
edge sends an unnamed release with an empty payload and its tool, the
batch store writes `pending_releases` (LOG.md (114)). Not built: the
ladder itself (the prober working `pending_releases` cheapest first,
the signature verdicts, the move into the catalog with the segments
read again), and the spot/NZB match on pending keys.
g. The account's NNTP connections shared by priority (`nntp_shares`): the
coordinator (headers) before the prober, each between its minimum and
maximum, 5 kept for the spotter, the spot inspector, the plain