Files
usenet-indexer/database/migrations/1770000007300_probe_candidates_on_nvme.ts
T
Daan MeijerandClaude Fable 5.1 6ef7ddca92
Test and deploy / test (push) Successful in 3m0s
Test and deploy / deploy (push) Successful in 4m33s
Batch store: one lock order for the pending upsert; probe_candidates on the NVMe
writePending upserted pending_releases in chunks of 1,000 rows, each chunk
followed by its class upsert, so one transaction locked releases, classes,
releases, classes, and a later chunk's keys and classes could sort before an
earlier chunk's: the opposite of the ladder's record(), which holds all its
rows and then takes all its classes. Four deadlocks between the two class
upserts in six hours (LOG.md (136)). Now all rows are sorted once and the
classes upserted once, after the last chunk.

Migration 7300 moves probe_candidates and its indexes to the default
tablespace (nvme on homelab): the catalog claim's UPDATE from random points
took 2-3 minutes on the HDD beside the sweep and the autovacuum, with all
lanes waiting. Homelab moved it by hand (83 s, ~8 GB, services stopped); the
migration is a no-op there, and without a default tablespace.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-03 19:52:09 +01:00

35 lines
1.5 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import { BaseSchema } from '@adonisjs/lucid/schema'
/**
* probe_candidates, with its indexes, in the catalog's tablespace (default_tablespace: nvme on homelab), out of
* CHUNK_TABLESPACE where migration 6000 put it: the prober's catalog claim updates it from random points every
* minute, and on the HDD, beside the pending ladder's sweep and autovacuum, a claim took 2–3 minutes with all the
* lanes waiting (LOG.md 2026-11-03 (136)). Homelab moved it by hand (edge, node-api and the workers stopped; ~8 GB);
* a no-op where it's there already, or without a default tablespace (tests).
*/
export default class extends BaseSchema {
async up() {
this.defer(async (db) => {
const { rows: setting } = await db.rawQuery(
`SELECT current_setting('default_tablespace') AS target`
)
const target = setting[0].target
if (!target) return
const { rows } = await db.rawQuery(
`SELECT c.relname, c.relkind FROM pg_class c LEFT JOIN pg_tablespace t ON t.oid = c.reltablespace
WHERE (c.relname = 'probe_candidates' OR c.oid IN (SELECT indexrelid FROM pg_index WHERE indrelid = 'probe_candidates'::regclass))
AND c.relkind IN ('r', 'i') AND coalesce(t.spcname, 'pg_default') <> ?
ORDER BY c.relkind DESC, c.relname`,
[target]
)
for (const { relname, relkind } of rows) {
await db.rawQuery(
`ALTER ${relkind === 'i' ? 'INDEX' : 'TABLE'} ${relname} SET TABLESPACE ${target}`
)
}
})
}
async down() {}
}