Daan MeijerandClaude Opus 5.5 8ad5ea162c par2 file lists dropped once their holders are final; late candidates claimable ...

Most par2 sets describe readable uploads whose files are never hashed, so they
were never complete and kept forever (19 GB). Candidates added to a release
after it settled were never marked ready (93,642 since 2026-10-13); the batch
store and merge mark them now, migration 5100 repairs the waiting ones
(LOG.md 2026-10-20 (87)).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

2026-10-20 13:56:57 +02:00

..

1761885935168_create_users_table.ts

omgebouwd naar adonisjs, werkt nog niet echt goed

2026-06-03 23:41:09 +02:00

1768620764696_create_access_tokens_table.ts

omgebouwd naar adonisjs, werkt nog niet echt goed

2026-06-03 23:41:09 +02:00

1770000000100_create_groups.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000200_create_rule_sets.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000300_create_ranges.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000400_create_releases.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000500_create_release_chunks.ts

Rebuild: the merger stops when a pass over the file keys decides nothing

2026-10-09 02:49:45 +02:00

1770000000600_create_fragments.ts

Rebuild: first end-to-end runs; prefetch per chunk, executable rule, wire fixes

2026-10-09 01:07:11 +02:00

1770000000700_create_probe_candidates.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000800_create_rule_bookkeeping.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000000900_create_nodes.ts

Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes

2026-10-09 00:19:00 +02:00

1770000001000_releases_unsettled_index.ts

Rebuild: settling quiet releases

2026-10-09 02:33:47 +02:00

1770000001100_ranges_prefetch_indexes.ts

Rebuild: index lookups for the merger's neighbours and the prefetcher's ranges

2026-10-09 02:59:30 +02:00

1770000001200_release_category.ts

Rebuild: the catalog API and Newznab, from voyager 1

2026-10-09 10:45:47 +02:00

1770000001300_groups_tail.ts

Rebuild: completeness as indexed, file totals from any counter, pausable tails, progress by articles

2026-10-09 13:06:03 +02:00

1770000001400_release_uploads.ts

Rebuild: re-uploads are releases of their own

2026-10-09 13:31:35 +02:00

1770000001500_ranges_plain_chunks.ts

Rebuild: chunks the server's compression cuts short are fetched plain

2026-10-09 13:51:25 +02:00

1770000001600_tsm_system_rows.ts

Rebuild: stats from a fixed-size sample, refreshed every 5 minutes, given up after 2

2026-10-09 17:49:45 +02:00

1770000001700_release_inspections.ts

Rebuild: phase 2's prober reads the probe queue

2026-10-11 15:32:49 +02:00

1770000001800_release_inspections_per_candidate.ts

Prober: an inspection per candidate; no inner name that is the release's name

2026-10-11 15:37:32 +02:00

1770000001900_releases_inner_name_trgm.ts

Search and the web interface use the prober's findings

2026-10-11 19:33:23 +02:00

1770000002000_releases_sort_indexes.ts

Rebuild: sort indexes that match newest-first

2026-10-11 19:39:42 +02:00

1770000002100_ranges_capture.ts

Corpus: stored XOVER chunks for the grouping sweeps

2026-10-12 09:28:22 +02:00

1770000002200_par2_merge.ts

The par2 merge: releases whose files one par2 set describes become one

2026-10-12 09:37:35 +02:00

1770000002300_probe_cache.ts

Prober: pipelined bodies, merge-first order, a probe cache across resets

2026-10-12 10:23:01 +02:00

1770000002400_probe_candidates_kind_index.ts

Probe candidates: an index on (kind, id) of the waiting ones for the prober's claims

2026-10-12 10:33:27 +02:00

1770000002500_probe_candidates_ready.ts

Prober: claim only candidates of settled releases, without the join

2026-10-12 12:12:59 +02:00

1770000002600_inner_stem_merge.ts

Merge obfuscated uploads by the random stem of their inner names

2026-10-12 23:38:31 +02:00

1770000002700_release_inspections_catalog_tablespace.ts

release_inspections in the catalog's tablespace (the NVMe)

2026-10-13 10:09:20 +02:00

1770000002800_spots.ts

Spots: unmatched ones matched again in turns; a standalone release gives way

2026-10-13 12:19:24 +02:00

1770000002900_spots_rematched.ts

Spots: unmatched ones matched again in turns; a standalone release gives way

2026-10-13 12:19:24 +02:00

1770000003000_spots_nzb_only_unmatched.ts

Spots: a matched spot's NZB isn't kept

2026-10-13 16:42:26 +02:00

1770000003100_chunk_indexes_with_chunks.ts

release_chunks' indexes with its chunks (CHUNK_TABLESPACE)

2026-10-14 16:27:58 +02:00

1770000003200_rarely_read_tables_with_chunks.ts

par2_files, fragments and probe_cache with the chunks (CHUNK_TABLESPACE)

2026-10-14 23:11:36 +02:00

1770000003300_release_verdicts.ts

The junk judge: releases judged by what their probes showed; verdicts hide them

2026-10-15 11:45:08 +02:00

1770000003400_fold_candidates.ts

Fold sidecars (.nfo, .sfv, .nzb) apart from their upload into it

2026-10-15 12:10:48 +02:00

1770000003500_fragments_back_on_nvme.ts

fragments back on the NVMe: the fragment merger reads every row, not by key

2026-10-15 12:46:14 +02:00

1770000003600_worker_cursors.ts

The merger keeps its place over a restart (worker_cursors), not from 1970 each deploy

2026-10-15 13:45:25 +02:00

1770000003700_par2_files_back_on_nvme.ts

par2_files back on the NVMe: the merger reads it per set, a random read per row on the HDD

2026-10-16 08:22:37 +02:00

1770000003800_rules_dropped_posters.ts

Rules drop posters at the header pass: the CelebMovie family (rule set version 3)

2026-10-16 11:36:46 +02:00

1770000003900_final_completeness.ts

Final completeness by time window: ranges keep their posting span, release_final() over blocks of done ranges

2026-10-16 14:19:03 +02:00

1770000004000_posters.ts

Poster search through a posters table and releases_poster_newest_idx; complete releases are final; the tail before everything (-3)

2026-10-16 17:25:30 +02:00

1770000004100_release_final_complete.ts

Poster search through a posters table and releases_poster_newest_idx; complete releases are final; the tail before everything (-3)

2026-10-16 17:25:30 +02:00

1770000004200_releases_sidecar_only.ts

Fold only sidecar-only releases (sidecar_only at insert); no queue-wide scan for gone releases

2026-10-17 20:57:59 +02:00

1770000004300_worker_cursors_key.ts

The par2 prune's cursor is (created_at to the microsecond, set_id): 20k sets of one flush share a created_at

2026-10-17 22:10:19 +02:00

1770000004400_nntp_shares.ts

NNTP shares: the account's connections shared by priority

2026-10-18 11:20:09 +02:00

1770000004500_probe_cache_pkey_on_nvme.ts

probe_cache's primary key on the catalog's tablespace

2026-10-18 12:50:35 +02:00

1770000004600_release_inspections_on_hdd.ts

release_inspections back on the NVMe

2026-10-19 13:08:35 +02:00

1770000004700_judge_indexes.ts

Judge: write only changed verdicts; scraped and locked by index

2026-10-19 11:48:54 +02:00

1770000004800_release_inspections_on_nvme.ts

release_inspections back on the NVMe

2026-10-19 13:08:35 +02:00

1770000004900_judge_dirty.ts

Judge the releases whose facts changed every 2 minutes, all daily

2026-10-19 20:10:49 +02:00

1770000005000_file_hashes_hash_size_release.ts

Judge: count a chunk's file hashes once, from a covering index

2026-10-19 20:32:52 +02:00

1770000005100_late_candidates_ready.ts

par2 file lists dropped once their holders are final; late candidates claimable

2026-10-20 13:56:57 +02:00