Daan MeijerandClaude Fable 5.1 6ef7ddca92 Batch store: one lock order for the pending upsert; probe_candidates on the NVMe ...

writePending upserted pending_releases in chunks of 1,000 rows, each chunk
followed by its class upsert, so one transaction locked releases, classes,
releases, classes, and a later chunk's keys and classes could sort before an
earlier chunk's: the opposite of the ladder's record(), which holds all its
rows and then takes all its classes. Four deadlocks between the two class
upserts in six hours (LOG.md (136)). Now all rows are sorted once and the
classes upserted once, after the last chunk.

Migration 7300 moves probe_candidates and its indexes to the default
tablespace (nvme on homelab): the catalog claim's UPDATE from random points
took 2-3 minutes on the HDD beside the sweep and the autovacuum, with all
lanes waiting. Homelab moved it by hand (83 s, ~8 GB, services stopped); the
migration is a no-op there, and without a default tablespace.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

2026-11-03 19:52:09 +01:00

..

1761885935168_create_users_table.ts

omgebouwd naar adonisjs, werkt nog niet echt goed

2026-06-03 23:41:09 +02:00

1768620764696_create_access_tokens_table.ts

omgebouwd naar adonisjs, werkt nog niet echt goed

2026-06-03 23:41:09 +02:00

1770000000100_create_groups.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000200_create_rule_sets.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000300_create_ranges.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000400_create_releases.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000500_create_release_chunks.ts

Rebuild: the merger stops when a pass over the file keys decides nothing

2026-10-09 02:49:45 +02:00

1770000000600_create_fragments.ts

Rebuild: first end-to-end runs; prefetch per chunk, executable rule, wire fixes

2026-10-09 01:07:11 +02:00

1770000000700_create_probe_candidates.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000800_create_rule_bookkeeping.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000900_create_nodes.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000001000_releases_unsettled_index.ts

Rebuild: settling quiet releases

2026-10-09 02:33:47 +02:00

1770000001100_ranges_prefetch_indexes.ts

Rebuild: index lookups for the merger's neighbours and the prefetcher's ranges

2026-10-09 02:59:30 +02:00

1770000001200_release_category.ts

Rebuild: the catalog API and Newznab, from voyager 1

2026-10-09 10:45:47 +02:00

1770000001300_groups_tail.ts

Rebuild: completeness as indexed, file totals from any counter, pausable tails, progress by articles

2026-10-09 13:06:03 +02:00

1770000001400_release_uploads.ts

Rebuild: re-uploads are releases of their own

2026-10-09 13:31:35 +02:00

1770000001500_ranges_plain_chunks.ts

Rebuild: chunks the server's compression cuts short are fetched plain

2026-10-09 13:51:25 +02:00

1770000001600_tsm_system_rows.ts

Rebuild: stats from a fixed-size sample, refreshed every 5 minutes, given up after 2

2026-10-09 17:49:45 +02:00

1770000001700_release_inspections.ts

Rebuild: phase 2's prober reads the probe queue

2026-10-11 15:32:49 +02:00

1770000001800_release_inspections_per_candidate.ts

Prober: an inspection per candidate; no inner name that is the release's name

2026-10-11 15:37:32 +02:00

1770000001900_releases_inner_name_trgm.ts

Search and the web interface use the prober's findings

2026-10-11 19:33:23 +02:00

1770000002000_releases_sort_indexes.ts

Rebuild: sort indexes that match newest-first

2026-10-11 19:39:42 +02:00

1770000002100_ranges_capture.ts

Corpus: stored XOVER chunks for the grouping sweeps

2026-10-12 09:28:22 +02:00

1770000002200_par2_merge.ts

The par2 merge: releases whose files one par2 set describes become one

2026-10-12 09:37:35 +02:00

1770000002300_probe_cache.ts

Prober: pipelined bodies, merge-first order, a probe cache across resets

2026-10-12 10:23:01 +02:00

1770000002400_probe_candidates_kind_index.ts

Probe candidates: an index on (kind, id) of the waiting ones for the prober's claims

2026-10-12 10:33:27 +02:00

1770000002500_probe_candidates_ready.ts

Prober: claim only candidates of settled releases, without the join

2026-10-12 12:12:59 +02:00

1770000002600_inner_stem_merge.ts

Merge obfuscated uploads by the random stem of their inner names

2026-10-12 23:38:31 +02:00

1770000002700_release_inspections_catalog_tablespace.ts

release_inspections in the catalog's tablespace (the NVMe)

2026-10-13 10:09:20 +02:00

1770000002800_spots.ts

Spots: unmatched ones matched again in turns; a standalone release gives way

2026-10-13 12:19:24 +02:00

1770000002900_spots_rematched.ts

Spots: unmatched ones matched again in turns; a standalone release gives way

2026-10-13 12:19:24 +02:00

1770000003000_spots_nzb_only_unmatched.ts

Spots: a matched spot's NZB isn't kept

2026-10-13 16:42:26 +02:00

1770000003100_chunk_indexes_with_chunks.ts

release_chunks' indexes with its chunks (CHUNK_TABLESPACE)

2026-10-14 16:27:58 +02:00

1770000003200_rarely_read_tables_with_chunks.ts

par2_files, fragments and probe_cache with the chunks (CHUNK_TABLESPACE)

2026-10-14 23:11:36 +02:00

1770000003300_release_verdicts.ts

The junk judge: releases judged by what their probes showed; verdicts hide them

2026-10-15 11:45:08 +02:00

1770000003400_fold_candidates.ts

Fold sidecars (.nfo, .sfv, .nzb) apart from their upload into it

2026-10-15 12:10:48 +02:00

1770000003500_fragments_back_on_nvme.ts

fragments back on the NVMe: the fragment merger reads every row, not by key

2026-10-15 12:46:14 +02:00

1770000003600_worker_cursors.ts

The merger keeps its place over a restart (worker_cursors), not from 1970 each deploy

2026-10-15 13:45:25 +02:00

1770000003700_par2_files_back_on_nvme.ts

par2_files back on the NVMe: the merger reads it per set, a random read per row on the HDD

2026-10-16 08:22:37 +02:00

1770000003800_rules_dropped_posters.ts

Rules drop posters at the header pass: the CelebMovie family (rule set version 3)

2026-10-16 11:36:46 +02:00

1770000003900_final_completeness.ts

Final completeness by time window: ranges keep their posting span, release_final() over blocks of done ranges

2026-10-16 14:19:03 +02:00

1770000004000_posters.ts

Poster search through a posters table and releases_poster_newest_idx; complete releases are final; the tail before everything (-3)

2026-10-16 17:25:30 +02:00

1770000004100_release_final_complete.ts

Poster search through a posters table and releases_poster_newest_idx; complete releases are final; the tail before everything (-3)

2026-10-16 17:25:30 +02:00

1770000004200_releases_sidecar_only.ts

Fold only sidecar-only releases (sidecar_only at insert); no queue-wide scan for gone releases

2026-10-17 20:57:59 +02:00

1770000004300_worker_cursors_key.ts

The par2 prune's cursor is (created_at to the microsecond, set_id): 20k sets of one flush share a created_at

2026-10-17 22:10:19 +02:00

1770000004400_nntp_shares.ts

NNTP shares: the account's connections shared by priority

2026-10-18 11:20:09 +02:00

1770000004500_probe_cache_pkey_on_nvme.ts

probe_cache's primary key on the catalog's tablespace

2026-10-18 12:50:35 +02:00

1770000004600_release_inspections_on_hdd.ts

release_inspections back on the NVMe

2026-10-19 13:08:35 +02:00

1770000004700_judge_indexes.ts

Judge: write only changed verdicts; scraped and locked by index

2026-10-19 11:48:54 +02:00

1770000004800_release_inspections_on_nvme.ts

release_inspections back on the NVMe

2026-10-19 13:08:35 +02:00

1770000004900_judge_dirty.ts

Judge the releases whose facts changed every 2 minutes, all daily

2026-10-19 20:10:49 +02:00

1770000004900_range_dates.ts

Posting-time windows into the corpus: ranges:dates, ranges:capture

2026-10-25 09:47:28 +01:00

1770000005000_file_hashes_hash_size_release.ts

Judge: count a chunk's file hashes once, from a covering index

2026-10-19 20:32:52 +02:00

1770000005100_late_candidates_ready.ts

par2 file lists dropped once their holders are final; late candidates claimable

2026-10-20 13:56:57 +02:00

1770000005200_release_category_trigger.ts

Categories from the inner name, spots and groups; fix the name rules

2026-10-23 12:07:12 +02:00

1770000006000_voyager3_catalog.ts

REBUILD.md: voyager 2's final numbers for group sets and albums (LOG 112)

2026-10-26 08:12:03 +01:00

1770000006100_blocks.ts

Voyager 3 schema: the slim catalog (migration 6000, the reset) and blocks (6100)

2026-10-25 13:44:54 +01:00

1770000006200_release_texts.ts

voyager 3: release_texts, the searchable strings of a release (migration 6200)

2026-10-25 20:33:28 +01:00

1770000006300_pending_releases.ts

Nothing enters the catalog without a usable name: the pending store (decision f amended)

2026-10-26 13:18:14 +01:00

1770000006400_pending_ladder.ts

The pending ladder in the prober: unnamed releases probed cheapest first, signatures learned, the named moved into the catalog

2026-10-26 13:41:30 +01:00

1770000006500_group_leases.ts

The window worker: group leases, the stream serving the corpus, the snapshot through the API (design 1 and 5)

2026-10-26 18:22:18 +01:00

1770000006600_group_lease_lanes.ts

Lease lanes for the tail, a mixed lease order, coordinator:status, the edge end to end

2026-10-27 13:47:31 +01:00

1770000006700_ancient_ranges.ts

An "ancient" range a group: articles 1 … low watermark − 1, never filled

2026-10-28 09:12:05 +01:00

1770000006800_pending_classes.ts

Pending ladder by class, twins by content, dropped classes at the edge

2026-11-02 12:22:32 +01:00

1770000006900_pending_taken_index.ts

pending:classes without table scans; an index for the prober's startup reset

2026-11-02 13:33:48 +01:00

1770000007000_probe_candidates_taken_index.ts

Tiny-archive rule; the ladder's discard by index; the namer in chunks

2026-11-02 20:23:16 +01:00

1770000007100_pending_swept.ts

The discard sweep beside the lanes, each dropped signature swept once

2026-11-02 21:12:51 +01:00

1770000007200_pending_signature_index.ts

Prober: the signature index by the poster's md5

2026-11-03 12:08:53 +01:00

1770000007300_probe_candidates_on_nvme.ts

Batch store: one lock order for the pending upsert; probe_candidates on the NVMe

2026-11-03 19:52:09 +01:00