Prober: the sweep by a signature index, pending rows locked in one order, deadlocked flushes kept

The discard sweep of a dropped signature read the whole pending_releases table per signature (a parallel seq
scan over the poster test, four minutes on the HDD that starved the lanes' claims): a partial index on
(group_id, tool, poster) WHERE status = 0 makes it a range, and the sweep queries a poster's rows by it and a
tool's and group's by the ladder index.

The batch store's pending upsert, the record, and the sweep deadlocked on pending_releases and pending_classes
rows locked in different orders, and a deadlocked flush dropped its outcomes: lockOrder() sorts every
multi-row statement's set the same way, and a flush that deadlocks anyway keeps its outcomes for the next.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

app/services/prober.ts

@@ -41,6 +41,8 @@ const TIERS: { kind: string; unreadable: boolean }[] = [
 const FLUSH_MS = 2000
 /** Between sweeps that discard the waiting releases of dropped signatures. */
 const SWEEP_MS = 5 * 60_000
+/** A flush that deadlocks (against the batch store's pending upsert) keeps its outcomes for the next, this often. */
+const DEADLOCK_RETRIES = 5
 
 /**
  * JSON for a jsonb parameter, without NUL characters: jsonb can't hold \u0000 (a name read from an article had one),
@@ -82,6 +84,8 @@ export class Prober {
   private refilling: Promise<void> | null = null
   private stopped = false
   private stats = Prober.freshStats()
+  /** Flushes deadlocked in a row, their outcomes kept. */
+  private deadlocks = 0
   /** Lanes running (each one connection, once it has work); at most the share's grant. Counted apart from the set: a lane leaving takes itself off at once. */
   private lanes = new Set<Promise<void>>()
   private running = 0
@@ -346,15 +350,34 @@ export class Prober {
     }
   }
 
-  /** The outcomes so far: candidates' status, inspections, and the releases' passworded and inner name. */
+  /**
+   * The outcomes so far: candidates' status, inspections, and the releases' passworded and inner name. A deadlock
+   * (the batch store's pending upsert locks the same rows; LOG.md (134)) keeps the outcomes for the next flush
+   * instead of losing the probes.
+   */
   private async flush() {
     const taken = this.outcomes.splice(0)
     if (taken.length === 0) return
+    try {
+      await this.store(taken)
+    } catch (error: any) {
+      if (error?.code === '40P01' && this.deadlocks < DEADLOCK_RETRIES) {
+        this.deadlocks++
+        this.outcomes.unshift(...taken)
+        logger.warn(`Probing: storing results deadlocked, kept for the next flush (${this.deadlocks}).`)
+        return
+      }
+      throw error
+    }
+    this.deadlocks = 0
     for (const o of taken) if (o.status !== 'waiting') this.stats[o.status]++
+    this.stats.pending += taken.filter((o) => o.candidate.pending && o.status !== 'waiting').length
+  }
+
+  private async store(taken: Outcome[]) {
     // The ladder's outcomes apart from the catalog's: the next target, a name for the namer, or the end of the climb.
     const pending = taken.filter((o) => o.candidate.pending)
     const outcomes = taken.filter((o) => !o.candidate.pending)
-    this.stats.pending += pending.filter((o) => o.status !== 'waiting').length
     await db.transaction(async (trx) => {
       if (outcomes.length > 0) {
         await trx.rawQuery(