Prober: claim only candidates of settled releases, without the join
Test and deploy / test (push) Successful in 48s
Test and deploy / edge (push) Successful in 1m59s
Test and deploy / deploy (push) Failing after 4m2s

The settler marks a settled release's candidates ready (and unreadable when
it had no readable name). Claims took the candidates by joining releases
from random points of the queue; in its newer half nothing had settled, so
a slice scanned to the end (0.3 s each, 20 slices and 5 tiers a claim) and
the prober stood still: 2.4M waiting, 0 probed a minute. Now an index over
the claimable rows only, the slices drawn within them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-12 12:12:59 +02:00
1 parent 1866e38c25
commit cdc1cdc0cc
5 files changed
+63 -11

No files matched your search

+12 -6
View File
@@ -21,14 +21,20 @@ class Settler {
async run(): Promise<number> {
let settled = 0
while (true) {
const { rowCount } = await db.rawQuery(
`UPDATE releases SET settled_at = now(), complete = CASE WHEN file_total IS NULL THEN NULL ELSE files >= file_total END
WHERE id IN (SELECT id FROM releases WHERE settled_at IS NULL AND last_seen_at < now() - make_interval(mins => ?)
AND (last_seen_at < now() - make_interval(hours => ?) OR coalesce(last_posted_at, posted_at) < now() - make_interval(hours => ?)) LIMIT ?)`,
// Their probe candidates become claimable (Prober.claim), the unreadable ones first.
const { rows } = await db.rawQuery(
`WITH settled AS (
UPDATE releases SET settled_at = now(), complete = CASE WHEN file_total IS NULL THEN NULL ELSE files >= file_total END
WHERE id IN (SELECT id FROM releases WHERE settled_at IS NULL AND last_seen_at < now() - make_interval(mins => ?)
AND (last_seen_at < now() - make_interval(hours => ?) OR coalesce(last_posted_at, posted_at) < now() - make_interval(hours => ?)) LIMIT ?)
RETURNING id, readable),
marked AS (UPDATE probe_candidates pc SET ready = true, unreadable = NOT s.readable FROM settled s WHERE pc.release_id = s.id)
SELECT count(*)::int AS n FROM settled`,
[QUIET_HISTORY_MINUTES, QUIET_HOURS, QUIET_HOURS, BATCH]
)
settled += rowCount ?? 0
if ((rowCount ?? 0) < BATCH) return settled
const n = rows[0].n
settled += n
if (n < BATCH) return settled
}
}
}