Daan MeijerandClaude Opus 5.5
35d0b2da65
5.4M waiting inspection jobs took 3.3 GB of Redis and most of its single thread. Producers now add file ids to inspection_queue; index-scheduler feeds up to INSPECTION_FEED_MAX (50k) into BullMQ. inspection:from-redis moves the existing backlog over once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
24 lines
925 B
TypeScript
24 lines
925 B
TypeScript
import { BaseSchema } from '@adonisjs/lucid/schema'
|
|
|
|
/**
|
|
* The content inspection backlog (LOG.md 2026-10-07 (8)): files waiting to be
|
|
* inspected, in Postgres instead of as BullMQ jobs. 5.4M waiting jobs took
|
|
* 3.3 GB of Redis memory and most of its single thread; InspectionQueue.feed
|
|
* keeps at most INSPECTION_FEED_MAX of them in BullMQ, where the workers and
|
|
* nodes take them. Lower priority first, then oldest.
|
|
*/
|
|
export default class extends BaseSchema {
|
|
async up() {
|
|
await this.db.rawQuery(`
|
|
CREATE TABLE IF NOT EXISTS inspection_queue (
|
|
file_id bigint PRIMARY KEY,
|
|
priority smallint NOT NULL DEFAULT 0,
|
|
queued_at timestamptz NOT NULL DEFAULT now())`)
|
|
await this.db.rawQuery(`CREATE INDEX IF NOT EXISTS inspection_queue_order_idx ON inspection_queue (priority, queued_at)`)
|
|
}
|
|
|
|
async down() {
|
|
await this.db.rawQuery(`DROP TABLE IF EXISTS inspection_queue`)
|
|
}
|
|
}
|