Commit Graph
4 Commits
Author SHA1 Message Date
Daan MeijerandClaude Fable 5.1 e470e231d0 Single-archive rules: stub archives and nonsense pieces dropped at the edge
Test and deploy / test (push) Successful in 3m0s
Test and deploy / deploy (push) Successful in 4m21s
The judge gets each file's articles seen and bytes, and whether the window
closed on the release. A settled release of one unsplit archive posted to 2+
groups is dropped as a stub archive when its articles are <= 256 KiB with the
release >= 1 MB, or it is in 4+ groups, or by a webmail poster (those two
under 100 MB); releases touching music, e-book, comics, tabs or fonts groups
are exempt. A one-article archive of 100 kB-1 MB under a release-like,
unscrambled name without spaces or an e-book format is dropped as a nonsense
piece. Both switched by the rule set (dropStubArchives, dropNonsensePieces),
mirrored in lib/rules, sampled like every drop (LOG.md (123)-(127)).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 11:35:46 +01:00
Daan MeijerandClaude Opus 5.5 3d24db5565 Rules drop posters at the header pass: the CelebMovie family (rule set version 3)
Test and deploy / test (push) Successful in 50s
Test and deploy / edge (push) Successful in 1m55s
Test and deploy / deploy (push) Successful in 1m47s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 11:36:46 +02:00
Daan MeijerandClaude Opus 5.5 cd06f96f85 Rebuild: first end-to-end runs; prefetch per chunk, executable rule, wire fixes
Test and deploy / test (push) Successful in 20s
Test and deploy / deploy (push) Skipped
End to end on the workstation (coordinator:run, the API, edge:node, real
NNTP): a.b.e-book.french and a.b.cd.image.playstation2.dvdiso ranges.

- Prefetcher: every connection takes the next missing chunk (leased
  ranges first, in order); one connection per range (~26k lines/s) kept
  the worker waiting
- executable rule: releases of bare executables (malware floods named
  after films and software) are dropped; they were 92% of the catalog rows
  of the test ranges, 21% of the segments
- fragments per file (details once) and gzipped batch bodies; the JSON
  limit (1 MB by default) raised to 256 MB, empty strings kept (a rule
  count's key is '')
- release_chunks.batch_from is a bigint (boneless article numbers)
- tests get their own database; edge:bench --store times BatchStore:
  1.3 s for a boneless range (1.24 ms per release chunk)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 01:07:11 +02:00
Daan MeijerandClaude Opus 5.5 d773c91ad0 Rebuild: phase 1 modules from the current app, ordered by pipeline stage
Test and deploy / test (push) Successful in 29s
Test and deploy / deploy (push) Skipped
lib/nntp (XOVER client, Xref, GROUP), lib/headers (subjects, file keys,
segments from overview lines; the yEnc path stays behind for phase 2),
lib/release (identity, random names, now in lib), lib/files (aggregator,
volume names), lib/rules (posting tools, owner by priority, release
verdicts), lib/chunks (release chunk codec). Their unit tests came along;
new ones for codec, rules and volumes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 00:08:44 +02:00