Inspection backlog in Postgres; BullMQ gets a working set
Test and deploy / test (push) Successful in 2m14s
Test and deploy / deploy (push) Successful in 2m51s

5.4M waiting inspection jobs took 3.3 GB of Redis and most of its single
thread. Producers now add file ids to inspection_queue; index-scheduler
feeds up to INSPECTION_FEED_MAX (50k) into BullMQ. inspection:from-redis
moves the existing backlog over once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-07 22:12:13 +02:00
1 parent e7e338213a
commit 35d0b2da65
11 files changed
+219 -24

No files matched your search

+11
View File
@@ -371,6 +371,17 @@ export class GroupSchema extends BaseModel {
declare updatedAt: DateTime | null
}
export class InspectionQueueSchema extends BaseModel {
static $columns = ['fileId', 'priority', 'queuedAt'] as const
$columns = InspectionQueueSchema.$columns
@column({ isPrimary: true })
declare fileId: bigint | number
@column()
declare priority: number
@column.dateTime()
declare queuedAt: DateTime
}
export class PartConflictSchema extends BaseModel {
static $columns = ['articleNumber', 'binaryId', 'bytes', 'fileCrc32', 'md516K', 'messageId', 'number', 'partSize', 'pcrc32'] as const
$columns = PartConflictSchema.$columns