Nothing enters the catalog without a usable name: the pending store (decision f amended)
An unnamed release (no readable file name) is written to pending_releases (migration 6300, the HDD) with its keys, counts, article span, upload tool and probe targets, instead of releases, release_texts, posters and a chunk. The edge sends it with an empty payload and its tool; the batch store splits a batch by readable. The pending ladder (probes cheapest first, signature verdicts, the move into the catalog) comes next. The replay report shows the pending releases. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
6860e6ccaf
commit
9cf85e09d8
8 files changed
+286
-15
No files matched your search
@@ -0,0 +1,60 @@
|
||||
import { BaseSchema } from '@adonisjs/lucid/schema'
|
||||
import env from '#start/env'
|
||||
|
||||
/**
|
||||
* pending_releases (REBUILD.md decision f, 2026-10-26): nothing enters the catalog without a usable name. A release
|
||||
* the window closed on whose files all have random names is written here instead of releases: its keys (as the
|
||||
* catalog would store them), its counts and article span, the upload tool from its message-id (with the group its
|
||||
* signature), and its probe targets by effort (the edge's candidates: a par2, an archive start, a lone file, a 7z
|
||||
* tail), no chunk. The pending ladder probes them cheapest first and moves a named one into the catalog, reading its
|
||||
* segments again from the corpus or the server; a signature that names nothing is discarded. On CHUNK_TABLESPACE
|
||||
* (the HDD): written once, read by the ladder, never by search.
|
||||
*/
|
||||
export default class extends BaseSchema {
|
||||
async up() {
|
||||
this.schema.raw(`
|
||||
CREATE TABLE pending_releases (
|
||||
-- lib/release/keys.ts: the identity, the group and the first article; the identity alone.
|
||||
release_key bytea PRIMARY KEY,
|
||||
identity_key bytea NOT NULL,
|
||||
group_id integer NOT NULL,
|
||||
-- The stem or post the edge keyed it by (a random name: no release name, but the inner-stem merge's handle).
|
||||
name varchar(255) NOT NULL,
|
||||
poster text,
|
||||
posted_at timestamptz,
|
||||
last_posted_at timestamptz,
|
||||
first_article bigint NOT NULL,
|
||||
last_article bigint NOT NULL,
|
||||
files integer NOT NULL DEFAULT 0,
|
||||
segments bigint NOT NULL DEFAULT 0,
|
||||
bytes bigint NOT NULL DEFAULT 0,
|
||||
-- A file misses parts.
|
||||
partial boolean NOT NULL DEFAULT false,
|
||||
-- False: a tail release still open; a later chunk of its key adds its counts up.
|
||||
finished boolean NOT NULL DEFAULT true,
|
||||
-- lib/rules uploadTool: ngPost, nyuu, hex32, random, other, …
|
||||
tool varchar(16) NOT NULL,
|
||||
-- [{kind, messageId, file}] by effort (lib/edge ProbeCandidate); empty: nothing to read, only a signature.
|
||||
candidates jsonb NOT NULL DEFAULT '[]',
|
||||
-- 0 waiting, 1 named (in the catalog), 2 discarded (its signature's verdict, or every probe gave nothing).
|
||||
status smallint NOT NULL DEFAULT 0,
|
||||
tries smallint NOT NULL DEFAULT 0,
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
probed_at timestamptz
|
||||
)`)
|
||||
// The ladder's order: a group's and tool's waiting releases, oldest first.
|
||||
this.schema.raw(
|
||||
`CREATE INDEX pending_releases_ladder_idx ON pending_releases (group_id, tool, status, created_at)`
|
||||
)
|
||||
const hdd = env.get('CHUNK_TABLESPACE')
|
||||
if (hdd) {
|
||||
this.schema.raw(`ALTER TABLE pending_releases SET TABLESPACE ${hdd}`)
|
||||
this.schema.raw(`ALTER INDEX pending_releases_pkey SET TABLESPACE ${hdd}`)
|
||||
this.schema.raw(`ALTER INDEX pending_releases_ladder_idx SET TABLESPACE ${hdd}`)
|
||||
}
|
||||
}
|
||||
|
||||
async down() {
|
||||
this.schema.raw(`DROP TABLE IF EXISTS pending_releases`)
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user