Prober: pipelined bodies, merge-first order, a probe cache across resets
Test and deploy / test (push) Successful in 56s
Test and deploy / edge (push) Successful in 2m33s
Test and deploy / deploy (push) Successful in 2m26s

- XoverClient.bodies: PROBE_PIPELINE (8) BODY commands per connection at
  once, replies read in order (one at a time it ran 3-10 articles/s on 6
  connections against a queue of 1M).
- Claims in tiers: lone files first, then par2 sets and archives of
  releases without a readable name (what the par2 merge needs), then those
  of readable releases.
- probe_cache: final answers by message-id (status, inspection with the
  16 KiB hash and par2 file list), PROBE_VERSION 2. A reset empties the
  probe queue, not the cache; recreated candidates of probed articles are
  answered from it without the news server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-12 10:23:01 +02:00
1 parent 6d1f2cc2a1
commit 0727afe968
9 files changed
+160 -34

No files matched your search

+3 -1
View File
@@ -73,7 +73,9 @@ export default {
*/
connections: env.get('PROBE_CONNECTIONS', 6),
/** Candidates claimed at a time. */
batch: 200,
batch: 400,
/** BODY commands in flight per connection (pipelined; XoverClient.bodies). */
pipeline: env.get('PROBE_PIPELINE', 8),
},
/** Stored XOVER chunks of ranges marked `capture`, for the grouping sweeps (app/services/corpus.ts). Off when unset. */
corpus: {