Single-archive rules: stub archives and nonsense pieces dropped at the edge
Test and deploy / test (push) Successful in 3m0s
Test and deploy / deploy (push) Successful in 4m21s

The judge gets each file's articles seen and bytes, and whether the window
closed on the release. A settled release of one unsplit archive posted to 2+
groups is dropped as a stub archive when its articles are <= 256 KiB with the
release >= 1 MB, or it is in 4+ groups, or by a webmail poster (those two
under 100 MB); releases touching music, e-book, comics, tabs or fonts groups
are exempt. A one-article archive of 100 kB-1 MB under a release-like,
unscrambled name without spaces or an e-book format is dropped as a nonsense
piece. Both switched by the rule set (dropStubArchives, dropNonsensePieces),
mirrored in lib/rules, sampled like every drop (LOG.md (123)-(127)).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Fable 5.1 committed 2026-10-31 11:35:46 +01:00
1 parent 2f1bbf2538
commit e470e231d0
8 files changed
+397 -15

No files matched your search

+34 -2
View File
@@ -151,7 +151,9 @@ dropping garbage; 28 TB of compressed overview traffic. Throughput goal:
indexed groups in the article's Xref, the one with the lowest priority
number; see the group list) → group articles into files → files
into releases (`lib/release` rules) → drop garbage (random name +
dropped signature or tool) → set aside per-article random subjects and
dropped signature or tool), bare executables, dropped posters, and the
single-archive fakes (stub archives and nonsense pieces, below) → set
aside per-article random subjects and
obfuscated subjects (counted only; phase 2). Returns per batch: release
chunks, edge fragments, probe candidates, counters. Holds no state beyond
its lease.
@@ -776,7 +778,37 @@ its release and sets `partial` (the store: `complete = false`). A lone
article of a multi-part random file is still counted `per-article`, not
written; before the block's end a scrambled name counts as random too
(`looks_scrambled`: a random subject and poster per article, 160k of
alt.binaries.storage's releases; LOG.md (122)). Gone: fragments and the cut-off, `uploads_of` (online now),
alt.binaries.storage's releases; LOG.md (122)).
**Single-archive rules (built 2026-10-31; LOG.md (123)-(127)).** The judge
(`names::judge`, `RuleSet.judge`) gets each file's articles seen and their
bytes, and whether the release is *settled* (the window closed on it: a
par2 can still join an open one; the start edge doesn't count, since a par2
follows its rar and 7% of the catalog's releases are start edges). A settled
release of one unsplit zip, rar or 7z and nothing else is dropped as a
**stub archive** (`stub-archive`, key: the legs tripped) when posted to
2+ groups and (A) its articles are ≤ 256 KiB with the release ≥ 1 MB
(Power-Post's 250 kB default; a one-article release always has small
articles, which exempts e-books and sidecars), or (X) posted to 4+ groups,
or (W) by a webmail poster, X and W under 100 MB (the fakes stop at
~90 MB; hash-named uploads and real albums above it don't). A release
touching a music, e-book, comics, tabs or fonts group is exempt: those
communities post real single small archives the same way (karaoke zips to
a.b.sounds.karaoke + a.b.test with Power-Post's articles; 16 of 32,308
known fakes lost, LOG.md (127)). These are the
exe-stub and recycled-installer floods under film and software names:
55% of the catalog on 2026-10-30, 99.3% of the known fakes, 0 known
content in the 217k sample, verified on 1,500 decoded articles. A settled
single archive of one article and 100 kB–1 MB under a release-like,
unscrambled name without spaces and without an e-book format in it is
dropped as a **nonsense piece** (`nonsense-piece`, key: the upload tool):
the newznzb.* pieces (scene-styled or random with a "-XX" suffix), 9% of
the catalog; under 100 kB are cue/log sidecars and adverts, and the names
with spaces are books and albums. Both rules are switched by the
rule set (`dropStubArchives`, `dropNonsensePieces`), sampled like every
drop, and mirrored in `lib/rules/rule_set.ts` (the TypeScript indexer
judges as each batch completes files, so it passes settled without the
window's knowledge). Gone: fragments and the cut-off, `uploads_of` (online now),
`check.ts` and the fixture comparison (the parity with the TypeScript
indexer; CI runs `edge/bench-fixtures.sh` instead), the `incomplete` count
(incomplete releases are kept). The wire carries `finished`, `edge` and