Daan MeijerandClaude Fable 5.1 8e008da364 Prober: the sweep by a signature index, pending rows locked in one order, deadlocked flushes kept ...

The discard sweep of a dropped signature read the whole pending_releases table per signature (a parallel seq
scan over the poster test, four minutes on the HDD that starved the lanes' claims): a partial index on
(group_id, tool, poster) WHERE status = 0 makes it a range, and the sweep queries a poster's rows by it and a
tool's and group's by the ladder index.

The batch store's pending upsert, the record, and the sweep deadlocked on pending_releases and pending_classes
rows locked in different orders, and a deadlocked flush dropped its outcomes: lockOrder() sorts every
multi-row statement's set the same way, and a flush that deadlocks anyway keeps its outcomes for the next.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

2026-11-03 10:36:19 +01:00

..

1761885935168_create_users_table.ts

omgebouwd naar adonisjs, werkt nog niet echt goed

2026-06-03 23:41:09 +02:00

1768620764696_create_access_tokens_table.ts

omgebouwd naar adonisjs, werkt nog niet echt goed

2026-06-03 23:41:09 +02:00

1770000000100_create_groups.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000200_create_rule_sets.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000300_create_ranges.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000400_create_releases.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000500_create_release_chunks.ts

Rebuild: the merger stops when a pass over the file keys decides nothing

2026-10-09 02:49:45 +02:00

1770000000600_create_fragments.ts

Rebuild: first end-to-end runs; prefetch per chunk, executable rule, wire fixes

2026-10-09 01:07:11 +02:00

1770000000700_create_probe_candidates.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000800_create_rule_bookkeeping.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000900_create_nodes.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000001000_releases_unsettled_index.ts

Rebuild: settling quiet releases

2026-10-09 02:33:47 +02:00

1770000001100_ranges_prefetch_indexes.ts

Rebuild: index lookups for the merger's neighbours and the prefetcher's ranges

2026-10-09 02:59:30 +02:00

1770000001200_release_category.ts

Rebuild: the catalog API and Newznab, from voyager 1

2026-10-09 10:45:47 +02:00

1770000001300_groups_tail.ts

Rebuild: completeness as indexed, file totals from any counter, pausable tails, progress by articles

2026-10-09 13:06:03 +02:00

1770000001400_release_uploads.ts

Rebuild: re-uploads are releases of their own

2026-10-09 13:31:35 +02:00

1770000001500_ranges_plain_chunks.ts

Rebuild: chunks the server's compression cuts short are fetched plain

2026-10-09 13:51:25 +02:00

1770000001600_tsm_system_rows.ts

Rebuild: stats from a fixed-size sample, refreshed every 5 minutes, given up after 2

2026-10-09 17:49:45 +02:00

1770000001700_release_inspections.ts

Rebuild: phase 2's prober reads the probe queue

2026-10-11 15:32:49 +02:00

1770000001800_release_inspections_per_candidate.ts

Prober: an inspection per candidate; no inner name that is the release's name

2026-10-11 15:37:32 +02:00

1770000001900_releases_inner_name_trgm.ts

Search and the web interface use the prober's findings

2026-10-11 19:33:23 +02:00

1770000002000_releases_sort_indexes.ts

Rebuild: sort indexes that match newest-first

2026-10-11 19:39:42 +02:00

1770000002100_ranges_capture.ts

Corpus: stored XOVER chunks for the grouping sweeps

2026-10-12 09:28:22 +02:00

1770000002200_par2_merge.ts

The par2 merge: releases whose files one par2 set describes become one

2026-10-12 09:37:35 +02:00

1770000002300_probe_cache.ts

Prober: pipelined bodies, merge-first order, a probe cache across resets

2026-10-12 10:23:01 +02:00

1770000002400_probe_candidates_kind_index.ts

Probe candidates: an index on (kind, id) of the waiting ones for the prober's claims

2026-10-12 10:33:27 +02:00

1770000002500_probe_candidates_ready.ts

Prober: claim only candidates of settled releases, without the join

2026-10-12 12:12:59 +02:00

1770000002600_inner_stem_merge.ts

Merge obfuscated uploads by the random stem of their inner names

2026-10-12 23:38:31 +02:00

1770000002700_release_inspections_catalog_tablespace.ts

release_inspections in the catalog's tablespace (the NVMe)

2026-10-13 10:09:20 +02:00

1770000002800_spots.ts

Spots: unmatched ones matched again in turns; a standalone release gives way

2026-10-13 12:19:24 +02:00

1770000002900_spots_rematched.ts

Spots: unmatched ones matched again in turns; a standalone release gives way

2026-10-13 12:19:24 +02:00

1770000003000_spots_nzb_only_unmatched.ts

Spots: a matched spot's NZB isn't kept

2026-10-13 16:42:26 +02:00

1770000003100_chunk_indexes_with_chunks.ts

release_chunks' indexes with its chunks (CHUNK_TABLESPACE)

2026-10-14 16:27:58 +02:00

1770000003200_rarely_read_tables_with_chunks.ts

par2_files, fragments and probe_cache with the chunks (CHUNK_TABLESPACE)

2026-10-14 23:11:36 +02:00

1770000003300_release_verdicts.ts

The junk judge: releases judged by what their probes showed; verdicts hide them

2026-10-15 11:45:08 +02:00

1770000003400_fold_candidates.ts

Fold sidecars (.nfo, .sfv, .nzb) apart from their upload into it

2026-10-15 12:10:48 +02:00

1770000003500_fragments_back_on_nvme.ts

fragments back on the NVMe: the fragment merger reads every row, not by key

2026-10-15 12:46:14 +02:00

1770000003600_worker_cursors.ts

The merger keeps its place over a restart (worker_cursors), not from 1970 each deploy

2026-10-15 13:45:25 +02:00

1770000003700_par2_files_back_on_nvme.ts

par2_files back on the NVMe: the merger reads it per set, a random read per row on the HDD

2026-10-16 08:22:37 +02:00

1770000003800_rules_dropped_posters.ts

Rules drop posters at the header pass: the CelebMovie family (rule set version 3)

2026-10-16 11:36:46 +02:00

1770000003900_final_completeness.ts

Final completeness by time window: ranges keep their posting span, release_final() over blocks of done ranges

2026-10-16 14:19:03 +02:00

1770000004000_posters.ts

Poster search through a posters table and releases_poster_newest_idx; complete releases are final; the tail before everything (-3)

2026-10-16 17:25:30 +02:00

1770000004100_release_final_complete.ts

Poster search through a posters table and releases_poster_newest_idx; complete releases are final; the tail before everything (-3)

2026-10-16 17:25:30 +02:00

1770000004200_releases_sidecar_only.ts

Fold only sidecar-only releases (sidecar_only at insert); no queue-wide scan for gone releases

2026-10-17 20:57:59 +02:00

1770000004300_worker_cursors_key.ts

The par2 prune's cursor is (created_at to the microsecond, set_id): 20k sets of one flush share a created_at

2026-10-17 22:10:19 +02:00

1770000004400_nntp_shares.ts

NNTP shares: the account's connections shared by priority

2026-10-18 11:20:09 +02:00

1770000004500_probe_cache_pkey_on_nvme.ts

probe_cache's primary key on the catalog's tablespace

2026-10-18 12:50:35 +02:00

1770000004600_release_inspections_on_hdd.ts

release_inspections back on the NVMe

2026-10-19 13:08:35 +02:00

1770000004700_judge_indexes.ts

Judge: write only changed verdicts; scraped and locked by index

2026-10-19 11:48:54 +02:00

1770000004800_release_inspections_on_nvme.ts

release_inspections back on the NVMe

2026-10-19 13:08:35 +02:00

1770000004900_judge_dirty.ts

Judge the releases whose facts changed every 2 minutes, all daily

2026-10-19 20:10:49 +02:00

1770000004900_range_dates.ts

Posting-time windows into the corpus: ranges:dates, ranges:capture

2026-10-25 09:47:28 +01:00

1770000005000_file_hashes_hash_size_release.ts

Judge: count a chunk's file hashes once, from a covering index

2026-10-19 20:32:52 +02:00

1770000005100_late_candidates_ready.ts

par2 file lists dropped once their holders are final; late candidates claimable

2026-10-20 13:56:57 +02:00

1770000005200_release_category_trigger.ts

Categories from the inner name, spots and groups; fix the name rules

2026-10-23 12:07:12 +02:00

1770000006000_voyager3_catalog.ts

REBUILD.md: voyager 2's final numbers for group sets and albums (LOG 112)

2026-10-26 08:12:03 +01:00

1770000006100_blocks.ts

Voyager 3 schema: the slim catalog (migration 6000, the reset) and blocks (6100)

2026-10-25 13:44:54 +01:00

1770000006200_release_texts.ts

voyager 3: release_texts, the searchable strings of a release (migration 6200)

2026-10-25 20:33:28 +01:00

1770000006300_pending_releases.ts

Nothing enters the catalog without a usable name: the pending store (decision f amended)

2026-10-26 13:18:14 +01:00

1770000006400_pending_ladder.ts

The pending ladder in the prober: unnamed releases probed cheapest first, signatures learned, the named moved into the catalog

2026-10-26 13:41:30 +01:00

1770000006500_group_leases.ts

The window worker: group leases, the stream serving the corpus, the snapshot through the API (design 1 and 5)

2026-10-26 18:22:18 +01:00

1770000006600_group_lease_lanes.ts

Lease lanes for the tail, a mixed lease order, coordinator:status, the edge end to end

2026-10-27 13:47:31 +01:00

1770000006700_ancient_ranges.ts

An "ancient" range a group: articles 1 … low watermark − 1, never filled

2026-10-28 09:12:05 +01:00

1770000006800_pending_classes.ts

Pending ladder by class, twins by content, dropped classes at the edge

2026-11-02 12:22:32 +01:00

1770000006900_pending_taken_index.ts

pending:classes without table scans; an index for the prober's startup reset

2026-11-02 13:33:48 +01:00

1770000007000_probe_candidates_taken_index.ts

Tiny-archive rule; the ladder's discard by index; the namer in chunks

2026-11-02 20:23:16 +01:00

1770000007100_pending_swept.ts

The discard sweep beside the lanes, each dropped signature swept once

2026-11-02 21:12:51 +01:00

1770000007200_pending_signature_index.ts

Prober: the sweep by a signature index, pending rows locked in one order, deadlocked flushes kept

2026-11-03 10:36:19 +01:00