The pending ladder in the prober: unnamed releases probed cheapest first, signatures learned, the named moved into the catalog
pending_ladder.ts claims the oldest waiting pending releases' next target (par2, archive, lone, 7z-tail in that order) as at least PENDING_SHARE of each claim; a usable name (the largest named file of a par2 set or a listing, or a lone file's yEnc name, neither packaging nor random) hands the release to the namer, no name the next target or the end of the climb; climbs count on pending_signatures per (tool, group, poster) and (tool, group), and a signature with 100 climbs and 0.5% named or fewer is dropped: its waiting releases are discarded without a probe, the rules' sample excepted. pending_namer.ts reads a named release's articles again from the corpus or by XOVER and writes it to the catalog under the pending key (source weak-stem, the name as inner name) with its chunk and candidates. Migration 6400; config PENDING_*. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
9cf85e09d8
commit
e135076fcd
14 files changed
+1045
-50
No files matched your search
@@ -0,0 +1,49 @@
|
||||
import { BaseSchema } from '@adonisjs/lucid/schema'
|
||||
|
||||
/**
|
||||
* The pending ladder (REBUILD.md decision f, 2026-10-26): what the prober needs to work pending_releases cheapest
|
||||
* first and to learn which signatures never name anything.
|
||||
* - pending_releases: `file_total` (the counter's, for the catalog row once named), `inspections` (what each probe
|
||||
* showed, by kind), `named` (the usable name a probe found; status 4 until the namer moved it into the catalog).
|
||||
* Status: 0 waiting, 1 named and in the catalog, 2 discarded (its signature's verdict), 3 exhausted (every
|
||||
* target probed, no name), 4 named, awaiting the move, 9 taken by a running prober.
|
||||
* - pending_signatures: per (tool, group, poster) how many pending releases were probed to the end and how many
|
||||
* got a name; `dropped` once probed ≥ PENDING_MIN_PROBES and named ≤ PENDING_MAX_NAMED of them (voyager 2's
|
||||
* garbage learner, LOG.md 2026-10-08 (5)). The poster '' row is the tool's and group's together.
|
||||
*/
|
||||
export default class extends BaseSchema {
|
||||
async up() {
|
||||
this.schema.raw(`ALTER TABLE pending_releases ADD COLUMN IF NOT EXISTS file_total integer`)
|
||||
this.schema.raw(
|
||||
`ALTER TABLE pending_releases ADD COLUMN IF NOT EXISTS inspections jsonb NOT NULL DEFAULT '[]'`
|
||||
)
|
||||
this.schema.raw(`ALTER TABLE pending_releases ADD COLUMN IF NOT EXISTS named varchar(255)`)
|
||||
// The ladder's claim: the oldest waiting ones; the namer's: the named ones.
|
||||
this.schema.raw(
|
||||
`CREATE INDEX IF NOT EXISTS pending_releases_waiting_idx ON pending_releases (created_at) WHERE status = 0`
|
||||
)
|
||||
this.schema.raw(
|
||||
`CREATE INDEX IF NOT EXISTS pending_releases_naming_idx ON pending_releases (probed_at) WHERE status = 4`
|
||||
)
|
||||
this.schema.raw(`
|
||||
CREATE TABLE pending_signatures (
|
||||
tool varchar(16) NOT NULL,
|
||||
group_id integer NOT NULL,
|
||||
poster text NOT NULL DEFAULT '',
|
||||
probed integer NOT NULL DEFAULT 0,
|
||||
named integer NOT NULL DEFAULT 0,
|
||||
dropped boolean NOT NULL DEFAULT false,
|
||||
updated_at timestamptz NOT NULL DEFAULT now(),
|
||||
PRIMARY KEY (tool, group_id, poster)
|
||||
)`)
|
||||
}
|
||||
|
||||
async down() {
|
||||
this.schema.raw(`DROP TABLE IF EXISTS pending_signatures`)
|
||||
this.schema.raw(`DROP INDEX IF EXISTS pending_releases_naming_idx`)
|
||||
this.schema.raw(`DROP INDEX IF EXISTS pending_releases_waiting_idx`)
|
||||
this.schema.raw(`ALTER TABLE pending_releases DROP COLUMN IF EXISTS named`)
|
||||
this.schema.raw(`ALTER TABLE pending_releases DROP COLUMN IF EXISTS inspections`)
|
||||
this.schema.raw(`ALTER TABLE pending_releases DROP COLUMN IF EXISTS file_total`)
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user