usenet-indexer

daan/usenet-indexer

0 Fork 0

Files

T

History

Daan MeijerandClaude Opus 5.5 95f1d427df

Test and deploy / test (push) Successful in 1m25s

Details

Test and deploy / deploy (push) Successful in 2m58s

Details

Inspect fewer files: skip later volumes under random stems

Of 5,000 waiting inspection jobs, 3,212 were queued only because their
stem looked random, and 2,044 of those were later RAR volumes
(xyz.part47.rar, .r00) or par2 recovery volumes (.vol03+04.par2). Their
first segment adds nothing: the set is listed from its first volume and
named from its index .par2, as for readable names already. Each one cost a
~700 KB download; inspection pulled ~85 MB/s and fell behind the backfill.

- isCandidate() rejects later volumes (isLaterVolume) before the random-
  name rule.
- worker:file only queues files that are candidates, so the ~30% that
  inspection skips anyway no longer take queue slots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

2026-10-04 09:32:40 +02:00

workers

Inspect fewer files: skip later volumes under random stems

2026-10-04 09:32:40 +02:00

BodyQueuePrune.ts

implemented a lot of features

2026-09-29 00:32:01 +02:00

ContentBackfill.ts

Spotnet import, header backfill, staging and counter grouping, faster search

2026-10-03 19:59:23 +02:00

DbReindex.ts

Spotnet import, header backfill, staging and counter grouping, faster search