The live tail reads an upload in pieces while it is posted, and each
piece queued a 7z-tail candidate for the last volume it had seen so far
(release 306905984: seven, for .002-.008). Only the last volume's end
holds the 7z listing. When candidates are stored, and when releases are
merged, waiting tails below the release's highest volume are deleted.
All tails of the highest volume stay: a piece that saw only part of it
queued a segment that is not its last. RAR is unaffected: it lists its
files in the first volume's header and gets no tail candidate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merges delete releases, so the count of a key's releases fell behind its
numbers (83 releases up to key:129): the 20 numbers tried were all taken
and the range's batch failed for good. Spot releases (spot:key) are not
numbered.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Most par2 sets describe readable uploads whose files are never hashed, so they
were never complete and kept forever (19 GB). Candidates added to a release
after it settled were never marked ready (93,642 since 2026-10-13); the batch
store and merge mark them now, migration 5100 repairs the waiting ones
(LOG.md 2026-10-20 (87)).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FTD posts one upload's files to different groups ("Knowsum-Loops_Are_My_
Religion-WEB-2026-FTD": par2 and part1 to a.b.music.misc, part2 to
a.b.sounds.music, each article in one group only; releases 81825221 and
81825412): the catalog matched a chunk only to releases of its own
group. A chunk without one there now joins a release of its key in
another group whose window is within 3 hours, when none of its file
names are in that release yet; a post of the same files to another group
(other message-ids, same names) stays a release of its own, or its NZB
would hold every file twice. Functional tests for both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Writers lock their group only, so two groups' batches can number the
same identity key at once (a spam post crossposted to 67 groups:
duplicate key ...:65, HTTP 500, the batch retried). Releases are now
inserted with ON CONFLICT (release_key) DO NOTHING, and those skipped
take the next number (up to 20 tries).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A poster re-uploading under the same name made one release of all
uploads: in a sample, 24% of releases with 2+ files had a file name more
than once (4.3% of files extra copies), 70% of those copies posted a day
or more apart; their NZBs listed each copy.
- File assembly keeps a re-upload's copy apart: a part already in every
open copy, posted more than 3 h from them, starts another copy; a
missing part fills the nearest copy (a re-send of what was missing),
a re-sent part near a copy that has it is ignored as before.
- A key's files in a batch split into uploads where posting pauses for
more than 3 h, counted from the latest article posted so far (a file
that takes hours to post doesn't split it); each upload is judged and
chunked on its own (ReleaseChunk.upload, .postedLast; candidates carry
their upload).
- The catalog matches a chunk to the release of its key whose articles
the range's group owns and whose posting window is within 3 h, else
starts one (key, key:2, …): releases get identity_key, group_id,
last_posted_at. Keys are locked in order (advisory locks), so
concurrent batches neither deadlock nor both start one; the merger
writes in key order too. The same upload multi-posted to other groups
is a release per group.
- The gap: 91% of uploads post within 3 h; 73% of re-uploaded copies
come later (releases:upload-gaps). releases:duplicates measures the
duplicate names.
Both implementations; check.ts compares the windows and uploads too
(identical on all captures; Rust tests mirror the TypeScript cases).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured on two neighbouring boneless ranges (merging their fragments):
- 99.8% of fragment files were single articles of a random-named
'file' of >100 parts: a random subject per article, which never
completes from headers. They now count as per-article.
- A file is kept as a fragment when its first part seen is above 1 and
comes within the start window (the idle window: uploads in busy groups
spread over minutes, and a 50k-article edge zone is ~30 s of boneless),
or when it was still open at the range's end with later parts missing.
Anything else missing parts is incomplete and only counted.
Per range: 70,308 → 130 fragment files; merged, 47 of the 48 files met
complete (57,558 segments, ~8% of the range's kept segments).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A batch is stored in one transaction and only at its range's resume
point, so a retried batch can't count twice. Garbage is judged before
per-article: a dropped tool isn't worth reading bodies for either.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
