Prober: the sweep by a signature index, pending rows locked in one order, deadlocked flushes kept
Test and deploy / test (push) Successful in 2m59s
Test and deploy / deploy (push) Failing after 21m32s

The discard sweep of a dropped signature read the whole pending_releases table per signature (a parallel seq
scan over the poster test, four minutes on the HDD that starved the lanes' claims): a partial index on
(group_id, tool, poster) WHERE status = 0 makes it a range, and the sweep queries a poster's rows by it and a
tool's and group's by the ladder index.

The batch store's pending upsert, the record, and the sweep deadlocked on pending_releases and pending_classes
rows locked in different orders, and a deadlocked flush dropped its outcomes: lockOrder() sorts every
multi-row statement's set the same way, and a flush that deadlocks anyway keeps its outcomes for the next.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Fable 5.1 committed 2026-11-03 10:36:19 +01:00
1 parent 5d3b578322
commit 8e008da364
5 files changed
+110 -24

No files matched your search

+9 -4
View File
@@ -936,10 +936,15 @@ f. **Nothing enters the catalog without a usable name** (amended
`pending_signatures` for (tool, group, poster) and (tool, group), and a
signature with PENDING_MIN_PROBES (100) climbs and at most
PENDING_MAX_NAMED (0.5%) named is dropped: a sweep beside the lanes
(every five minutes, in its own loop: a signature's rows are minutes
of HDD) discards the waiting releases of each dropped signature and
class once (`swept_at`; the edge drops their new releases itself), the
rules' `samplePercent` of them left to climb (LOG.md (133)). The namer
(every five minutes, in its own loop) discards the waiting releases
of each dropped signature and class once (`swept_at`; the edge drops
their new releases itself), the rules' `samplePercent` of them left to
climb, each by an index (`pending_releases_signature_idx` for a
poster's rows: tested on the rows, a signature was a scan of the whole
table; LOG.md (133), (134)). The batch store, the record, the claim
and the sweep lock pending rows in one order (`lockOrder`), and a
flush that deadlocks anyway keeps its outcomes for the next (LOG.md
(134)). The namer
(`app/services/pending_namer.ts`, in the prober's process): a named
release's article span read again from the corpus (the stored range's
chunks) or by XOVER on a connection of its own, a chunk of articles