Daan MeijerandClaude Opus 5.5 2178847513 Stop refilling parts of binaries that are already a file ...

Crossposts, reposts and backfill ranges keep bringing articles of files
that worker:file already created. Ingest inserted their header parts again,
and GC's copied-parts pass (which searched all binaries for parts on every
batch) removed only 150k-450k per sweep: parts grew to 68M rows, 56M of
them belonging to finished files, and the 23 GB table paused the backfill.

- Ingest skips header parts (no per-part CRC) for binaries with a file_id.
- GC drops copied parts with a cursor over binaries, 2000 per statement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

2026-10-04 02:53:11 +02:00

..

workers

Spotnet import, header backfill, staging and counter grouping, faster search

2026-10-03 19:59:23 +02:00

BodyQueuePrune.ts

implemented a lot of features

2026-09-29 00:32:01 +02:00

ContentBackfill.ts

Spotnet import, header backfill, staging and counter grouping, faster search

2026-10-03 19:59:23 +02:00

DbReindex.ts

Spotnet import, header backfill, staging and counter grouping, faster search

2026-10-03 19:59:23 +02:00

DeferredExpandRanges.ts

implemented a lot of features

2026-09-29 00:32:01 +02:00

FilesCleanNames.ts

Spotnet import, header backfill, staging and counter grouping, faster search

2026-10-03 19:59:23 +02:00

GcScheduler.ts

Stop refilling parts of binaries that are already a file

2026-10-04 02:53:11 +02:00

GroupAdd.ts

Spotnet import, header backfill, staging and counter grouping, faster search

2026-10-03 19:59:23 +02:00

GroupBackfill.ts

Spotnet import, header backfill, staging and counter grouping, faster search

2026-10-03 19:59:23 +02:00

GroupsTop.ts

implemented a lot of features

2026-09-29 00:32:01 +02:00

IndexScheduler.ts

Spotnet import, header backfill, staging and counter grouping, faster search

2026-10-03 19:59:23 +02:00

ReleaseMergeSplit.ts

Spotnet import, header backfill, staging and counter grouping, faster search

2026-10-03 19:59:23 +02:00

ReleaseRebuild.ts

implemented a lot of features

2026-09-29 00:32:01 +02:00

ReleaseRename.ts

implemented a lot of features

2026-09-29 00:32:01 +02:00

ReleaseStats.ts

implemented a lot of features

2026-09-29 00:32:01 +02:00

SpotPasswords.ts

Spotnet import, header backfill, staging and counter grouping, faster search

2026-10-03 19:59:23 +02:00

SpotReimport.ts

Spotnet import, header backfill, staging and counter grouping, faster search

2026-10-03 19:59:23 +02:00