Seed plans fill every uncovered gap, with priorities; cheaper ingest and file storing
- seed:plan plans every part of a group's history no range covers (it started below the lowest known range and left a.b.ath's 13 most recent months out); seed_ranges.priority orders leases (seed:plan --priority). - binaries.part_count, kept by the parts insert in the same statement: claimCompleted compares it instead of counting parts every batch. - file_keys: 64-bit hashes of all file keys; FileStore.insert asks it before probing every files partition. - worker:file stores 100 completed binaries per job with set-based queries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
61b683a71f
commit
693e6c3244
14 files changed
+473
-132
No files matched your search
@@ -0,0 +1,21 @@
|
||||
import { BaseSchema } from '@adonisjs/lucid/schema'
|
||||
|
||||
/**
|
||||
* Stored parts per binary, kept by IngestService as it inserts parts
|
||||
* (LOG.md 2026-10-06 (7)). claimCompleted counted every binary's parts in
|
||||
* every batch (15% of Postgres' active time); a binary of thousands of
|
||||
* parts that gets a few per batch was counted thousands of times. NULL:
|
||||
* not counted yet (binaries from before), counted exactly on the next new
|
||||
* part. Only parts of binaries without a file are deleted, so it stays exact
|
||||
* until a binary completes.
|
||||
*/
|
||||
export default class extends BaseSchema {
|
||||
async up() {
|
||||
await this.db.rawQuery(`SET LOCAL lock_timeout = '2min'`)
|
||||
await this.db.rawQuery(`ALTER TABLE binaries ADD COLUMN IF NOT EXISTS part_count integer`)
|
||||
}
|
||||
|
||||
async down() {
|
||||
await this.db.rawQuery(`ALTER TABLE binaries DROP COLUMN IF EXISTS part_count`)
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user