Complete archive listings by walking RAR headers, at low priority
Test and deploy / test (push) Successful in 1m24s
Test and deploy / deploy (push) Successful in 1m58s

Inspection lists only the entries whose headers are in the first segment
of a first volume. A walk hops from header to header, fetching one segment
per entry and volume, in its own queue (one job per collection worker) on
NNTP connections inspection isn't waiting for. content:walk-archives queues
releases inspected earlier (CONCEPT D40).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-05 23:32:52 +02:00
1 parent b7ae8267c6
commit f4ded062df
17 files changed
+597 -17

No files matched your search

+5 -1
View File
@@ -41,6 +41,10 @@ class QueueService {
get collectionQueue() {
return this.queue('collection-queue')
}
/** Complete archive listings, at low priority (ArchiveWalkService). */
get archiveWalkQueue() {
return this.queue('archive-walk-queue')
}
/** Queues no longer used by the pipeline, kept here so GC can empty them. */
get legacy(): Queue[] {
@@ -48,7 +52,7 @@ class QueueService {
}
get all(): Queue[] {
return [this.nntpFetchQueue, this.fileQueue, this.bodyQueue, this.collectionQueue]
return [this.nntpFetchQueue, this.fileQueue, this.bodyQueue, this.collectionQueue, this.archiveWalkQueue]
}
/** Closes the queues that were opened. */