Daan MeijerandClaude Opus 5.5
0613984de9
A poster re-uploading under the same name made one release of all uploads: in a sample, 24% of releases with 2+ files had a file name more than once (4.3% of files extra copies), 70% of those copies posted a day or more apart; their NZBs listed each copy. - File assembly keeps a re-upload's copy apart: a part already in every open copy, posted more than 3 h from them, starts another copy; a missing part fills the nearest copy (a re-send of what was missing), a re-sent part near a copy that has it is ignored as before. - A key's files in a batch split into uploads where posting pauses for more than 3 h, counted from the latest article posted so far (a file that takes hours to post doesn't split it); each upload is judged and chunked on its own (ReleaseChunk.upload, .postedLast; candidates carry their upload). - The catalog matches a chunk to the release of its key whose articles the range's group owns and whose posting window is within 3 h, else starts one (key, key:2, …): releases get identity_key, group_id, last_posted_at. Keys are locked in order (advisory locks), so concurrent batches neither deadlock nor both start one; the merger writes in key order too. The same upload multi-posted to other groups is a release per group. - The gap: 91% of uploads post within 3 h; 73% of re-uploaded copies come later (releases:upload-gaps). releases:duplicates measures the duplicate names. Both implementations; check.ts compares the windows and uploads too (identical on all captures; Rust tests mirror the TypeScript cases). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
27 lines
1.4 KiB
TypeScript
27 lines
1.4 KiB
TypeScript
import { BaseSchema } from '@adonisjs/lucid/schema'
|
|
|
|
/**
|
|
* Re-uploads as releases of their own (REBUILD.md, decision d): releases
|
|
* keep the identity key their files gave (identity_key; release_key is that
|
|
* for the first upload, then key:2, key:3, …), the group that owns their
|
|
* articles, and their posting window (posted_at … last_posted_at). A batch's
|
|
* chunk joins a release with its key and group whose window is within
|
|
* UPLOAD_GAP_MS, else starts one.
|
|
*/
|
|
export default class extends BaseSchema {
|
|
async up() {
|
|
this.schema.raw(`ALTER TABLE releases ADD COLUMN IF NOT EXISTS identity_key varchar(40)`)
|
|
this.schema.raw(`ALTER TABLE releases ADD COLUMN IF NOT EXISTS group_id integer`)
|
|
this.schema.raw(`ALTER TABLE releases ADD COLUMN IF NOT EXISTS last_posted_at timestamptz`)
|
|
this.schema.raw(`UPDATE releases SET identity_key = release_key, last_posted_at = posted_at WHERE identity_key IS NULL`)
|
|
this.schema.raw(`ALTER TABLE releases ALTER COLUMN identity_key SET NOT NULL`)
|
|
this.schema.raw(`ALTER TABLE releases ALTER COLUMN release_key TYPE varchar(48)`)
|
|
this.schema.raw(`CREATE INDEX IF NOT EXISTS releases_identity_idx ON releases (identity_key, group_id)`)
|
|
}
|
|
|
|
async down() {
|
|
this.schema.raw(`DROP INDEX IF EXISTS releases_identity_idx`)
|
|
this.schema.raw(`ALTER TABLE releases DROP COLUMN IF EXISTS identity_key, DROP COLUMN IF EXISTS group_id, DROP COLUMN IF EXISTS last_posted_at`)
|
|
}
|
|
}
|