Complete archive listings by walking RAR headers, at low priority
Test and deploy / test (push) Successful in 1m24s
Test and deploy / deploy (push) Successful in 1m58s

Inspection lists only the entries whose headers are in the first segment
of a first volume. A walk hops from header to header, fetching one segment
per entry and volume, in its own queue (one job per collection worker) on
NNTP connections inspection isn't waiting for. content:walk-archives queues
releases inspected earlier (CONCEPT D40).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-05 23:32:52 +02:00
1 parent b7ae8267c6
commit f4ded062df
17 files changed
+597 -17

No files matched your search

+2
View File
@@ -69,6 +69,8 @@ export default await Env.create(new URL('../', import.meta.url), {
FETCH_CONCURRENCY: Env.schema.number.optional(),
BODY_CONCURRENCY: Env.schema.number.optional(),
COLLECTION_CONCURRENCY: Env.schema.number.optional(),
ARCHIVE_WALK_CONCURRENCY: Env.schema.number.optional(),
ARCHIVE_WALK_MAX_SEGMENTS: Env.schema.number.optional(),
DEFERRED_SETTLE_MS: Env.schema.number.optional(),
DEFERRED_EXPAND_MAX_PARTS: Env.schema.number.optional(),
DEFERRED_CONCURRENCY: Env.schema.number.optional(),