Pending ladder: the sweep and the record lock their rows in key order
discardKeys updated with `release_key = ANY(subquery)`, which locks rows in the plan's scan order, and deadlocked against a batch upsert that takes its rows by key (LOG.md (137)). It now updates from a subquery that selects the rows ORDER BY release_key FOR UPDATE, so the locks are taken sorted (the plan: LockRows above Sort). record() runs the same SELECT ... FOR UPDATE over its taken rows before its UPDATE ... FROM jsonb_to_recordset, whose join order the planner chooses. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
eadae82b0c
commit
d7171a5ca4
2 files changed
+20
-5
No files matched your search
+6
-3
@@ -950,9 +950,12 @@ f. **Nothing enters the catalog without a usable name** (amended
|
|||||||
climb, each by an index (`pending_releases_signature_idx` for a
|
climb, each by an index (`pending_releases_signature_idx` for a
|
||||||
poster's rows: tested on the rows, a signature was a scan of the whole
|
poster's rows: tested on the rows, a signature was a scan of the whole
|
||||||
table; LOG.md (133), (134)). The batch store, the record, the claim
|
table; LOG.md (133), (134)). The batch store, the record, the claim
|
||||||
and the sweep lock pending rows in one order (`lockOrder`), and a
|
and the sweep lock pending rows in one order, by key and the counters
|
||||||
flush that deadlocks anyway keeps its outcomes for the next (LOG.md
|
after (`lockOrder` for the multi-row statements; the record and the
|
||||||
(134)). The namer
|
sweep lock their rows with `SELECT … ORDER BY release_key FOR UPDATE`
|
||||||
|
first, since an UPDATE's join or `= ANY(subquery)` locks in plan
|
||||||
|
order), and a flush that deadlocks anyway keeps its outcomes for the
|
||||||
|
next (LOG.md (134), (137)). The namer
|
||||||
(`app/services/pending_namer.ts`, in the prober's process): a named
|
(`app/services/pending_namer.ts`, in the prober's process): a named
|
||||||
release's article span read again from the corpus (the stored range's
|
release's article span read again from the corpus (the stored range's
|
||||||
chunks) or by XOVER on a connection of its own, a chunk of articles
|
chunks) or by XOVER on a connection of its own, a chunk of articles
|
||||||
|
|||||||
@@ -289,6 +289,12 @@ class PendingLadder {
|
|||||||
final: null,
|
final: null,
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
// The rows locked in key order first: the UPDATE's join below takes them in whatever order the plan does.
|
||||||
|
await trx.rawQuery(
|
||||||
|
`SELECT 1 FROM pending_releases WHERE release_key = ANY(SELECT decode(k, 'hex') FROM unnest(?::text[]) AS k)
|
||||||
|
ORDER BY release_key FOR UPDATE`,
|
||||||
|
[rows.map((r) => r.release_key).sort()]
|
||||||
|
)
|
||||||
await trx.rawQuery(
|
await trx.rawQuery(
|
||||||
`UPDATE pending_releases p SET status = v.status, tries = v.tries, named = coalesce(v.named, p.named),
|
`UPDATE pending_releases p SET status = v.status, tries = v.tries, named = coalesce(v.named, p.named),
|
||||||
inspections = CASE WHEN v.inspection IS NULL THEN p.inspections ELSE p.inspections || v.inspection END,
|
inspections = CASE WHEN v.inspection IS NULL THEN p.inspections ELSE p.inspections || v.inspection END,
|
||||||
@@ -470,9 +476,15 @@ class PendingLadder {
|
|||||||
.filter((key: string) => !explored(key, samplePercent))
|
.filter((key: string) => !explored(key, samplePercent))
|
||||||
.sort()
|
.sort()
|
||||||
if (keys.length === 0) return 0
|
if (keys.length === 0) return 0
|
||||||
|
// Locked in key order (the record's lock order, LOG.md (137)): a plain `key = ANY(subquery)` locks in scan
|
||||||
|
// order and deadlocked against a batch upsert; ORDER BY sorts before FOR UPDATE locks.
|
||||||
const { rows: gone } = await db.rawQuery(
|
const { rows: gone } = await db.rawQuery(
|
||||||
`UPDATE pending_releases SET status = ? WHERE status = ? AND release_key = ANY(SELECT decode(k, 'hex') FROM unnest(?::text[]) AS k)
|
`UPDATE pending_releases p SET status = ?
|
||||||
RETURNING tool, group_id, klass`,
|
FROM (SELECT release_key FROM pending_releases
|
||||||
|
WHERE status = ? AND release_key = ANY(SELECT decode(k, 'hex') FROM unnest(?::text[]) AS k)
|
||||||
|
ORDER BY release_key FOR UPDATE) k
|
||||||
|
WHERE p.release_key = k.release_key
|
||||||
|
RETURNING p.tool, p.group_id, p.klass`,
|
||||||
[PENDING.discarded, PENDING.waiting, keys]
|
[PENDING.discarded, PENDING.waiting, keys]
|
||||||
)
|
)
|
||||||
const counts = new Map<string, { tool: string; group_id: number; klass: number; n: number }>()
|
const counts = new Map<string, { tool: string; group_id: number; klass: number; n: number }>()
|
||||||
|
|||||||
Reference in new issue
Block a user