Rebuild: phase 2's prober reads the probe queue
probe:run (compose service prober, PROBE_CONNECTIONS, default 6) claims waiting candidates of settled releases (lone, then par2, then archive), fetches each article with BODY over connections of its own (no compression) and inspects it: the yEnc name and size, the content type from the first bytes, a par2 set's files, a RAR volume's entries and whether they're encrypted. Results per release and kind in release_inspections (CHUNK_TABLESPACE, the HDD); on the release passworded, inner_name (the largest content name) and inspected_at. 430 = missing, not yEnc = error, both final; connection trouble puts the candidate back. Fetch and parse in one process: an article takes ~0.1 s per connection, parsing a few ms (no queue in between). Parsers from voyager 1 (lib/yenc, lib/rar, lib/par2, lib/content sniff) with their tests; XoverClient.body; yencode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
d72fe2f13f
commit
8bd6d6aa5f
30 files changed
+1666
-7
No files matched your search
@@ -0,0 +1,38 @@
|
||||
import { BaseSchema } from '@adonisjs/lucid/schema'
|
||||
import env from '#start/env'
|
||||
|
||||
/**
|
||||
* Phase 2's probe results (REBUILD.md, part 6): per release and probe kind
|
||||
* what its article showed (yEnc name, content type, a few names inside,
|
||||
* encryption), in CHUNK_TABLESPACE (the HDD) when set: they're read per
|
||||
* release, not searched. On the catalog only what search needs: whether the
|
||||
* release is passworded and its best inner name.
|
||||
*/
|
||||
export default class extends BaseSchema {
|
||||
async up() {
|
||||
const tablespace = env.get('CHUNK_TABLESPACE')
|
||||
const where = tablespace ? ` TABLESPACE ${tablespace}` : ''
|
||||
this.schema.raw(`
|
||||
CREATE TABLE release_inspections (
|
||||
release_id bigint NOT NULL,
|
||||
kind varchar(8) NOT NULL,
|
||||
result jsonb NOT NULL,
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
PRIMARY KEY (release_id, kind)${tablespace ? ` USING INDEX TABLESPACE ${tablespace}` : ''}
|
||||
)${where}`)
|
||||
this.schema.alterTable('releases', (table) => {
|
||||
table.boolean('passworded').nullable()
|
||||
table.string('inner_name', 255).nullable()
|
||||
table.timestamp('inspected_at', { useTz: true }).nullable()
|
||||
})
|
||||
}
|
||||
|
||||
async down() {
|
||||
this.schema.alterTable('releases', (table) => {
|
||||
table.dropColumn('passworded')
|
||||
table.dropColumn('inner_name')
|
||||
table.dropColumn('inspected_at')
|
||||
})
|
||||
this.schema.dropTable('release_inspections')
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user