Files
usenet-indexer/database/migrations/1768620765210_binary_part_count.ts
T
Daan MeijerandClaude Opus 5.5 693e6c3244
Test and deploy / test (push) Successful in 1m27s
Test and deploy / deploy (push) Failing after 33m7s
Seed plans fill every uncovered gap, with priorities; cheaper ingest and file storing
- seed:plan plans every part of a group's history no range covers (it
  started below the lowest known range and left a.b.ath's 13 most recent
  months out); seed_ranges.priority orders leases (seed:plan --priority).
- binaries.part_count, kept by the parts insert in the same statement:
  claimCompleted compares it instead of counting parts every batch.
- file_keys: 64-bit hashes of all file keys; FileStore.insert asks it
  before probing every files partition.
- worker:file stores 100 completed binaries per job with set-based queries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 14:18:01 +02:00

22 lines
859 B
TypeScript

import { BaseSchema } from '@adonisjs/lucid/schema'
/**
* Stored parts per binary, kept by IngestService as it inserts parts
* (LOG.md 2026-10-06 (7)). claimCompleted counted every binary's parts in
* every batch (15% of Postgres' active time); a binary of thousands of
* parts that gets a few per batch was counted thousands of times. NULL:
* not counted yet (binaries from before), counted exactly on the next new
* part. Only parts of binaries without a file are deleted, so it stays exact
* until a binary completes.
*/
export default class extends BaseSchema {
async up() {
await this.db.rawQuery(`SET LOCAL lock_timeout = '2min'`)
await this.db.rawQuery(`ALTER TABLE binaries ADD COLUMN IF NOT EXISTS part_count integer`)
}
async down() {
await this.db.rawQuery(`ALTER TABLE binaries DROP COLUMN IF EXISTS part_count`)
}
}