Files
usenet-indexer/database/migrations/1770000002300_probe_cache.ts
Daan MeijerandClaude Opus 5.5 0727afe968
Test and deploy / test (push) Successful in 56s
Test and deploy / edge (push) Successful in 2m33s
Test and deploy / deploy (push) Successful in 2m26s
Prober: pipelined bodies, merge-first order, a probe cache across resets
- XoverClient.bodies: PROBE_PIPELINE (8) BODY commands per connection at
  once, replies read in order (one at a time it ran 3-10 articles/s on 6
  connections against a queue of 1M).
- Claims in tiers: lone files first, then par2 sets and archives of
  releases without a readable name (what the par2 merge needs), then those
  of readable releases.
- probe_cache: final answers by message-id (status, inspection with the
  16 KiB hash and par2 file list), PROBE_VERSION 2. A reset empties the
  probe queue, not the cache; recreated candidates of probed articles are
  answered from it without the news server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 10:23:01 +02:00

26 lines
819 B
TypeScript

import { BaseSchema } from '@adonisjs/lucid/schema'
/**
* What a probed article showed, by message-id (app/services/prober.ts): an
* article's inspection doesn't depend on the grouping rules, so a catalog
* reset (which empties the probe queue) needn't read it from the news server
* again. `version` is PROBE_VERSION when it was inspected: a change to the
* inspection makes older rows stale. Not truncated by a reset.
*/
export default class extends BaseSchema {
async up() {
this.schema.raw(`
CREATE TABLE probe_cache (
message_id text PRIMARY KEY,
status varchar(8) NOT NULL,
result jsonb,
version integer NOT NULL,
created_at timestamptz NOT NULL DEFAULT now()
)`)
}
async down() {
this.schema.raw(`DROP TABLE IF EXISTS probe_cache`)
}
}