- seed_nodes (hashed bearer tokens) and seed_ranges (planned ranges with
lease, resume point and counts); migration …140.
- SeedOrchestrator: lease (oldest plan first, FOR UPDATE SKIP LOCKED; a
lease without a batch for 15 min passes to the next node and resumes at
next_id), batch (stores files and fragments, moves next_id), finish,
release, plan (from below live indexing / the backfill / earlier plans
down to N days back; takes over the group's backfill), status.
- Seed API under /api/v1/seed (lease, ranges/:id/batches|done|release,
status) behind a node-token middleware; JSON bodies up to 64 MB after
gzip.
- seed:node runs a node: no database or Redis, outgoing HTTPS only, N
connections, gzip batches, retries for network errors and 5xx, drops a
range on 409, gives its ranges back on SIGTERM. seed-node/ has a compose
file that builds the image from this repository.
- seed:token, seed:plan, seed:status for the operator. Header parsing moved
to lib/ingest and the range loop to lib/seed/runner, shared by seed:range
and seed:node.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A seeding node fetches a range over one NNTP connection, parses headers
with the live rules, groups them into files in memory (lib/aggregate) and
stores per complete file its release, the file and a completed binary
(SeedService), instead of one parts row per article. What doesn't complete
within the window goes through the regular ingest path as fragments.
`--dry-run` only fetches and aggregates, to measure a node.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
