Daan MeijerandClaude Opus 5.5
8c36744088
- seed_nodes (hashed bearer tokens) and seed_ranges (planned ranges with lease, resume point and counts); migration …140. - SeedOrchestrator: lease (oldest plan first, FOR UPDATE SKIP LOCKED; a lease without a batch for 15 min passes to the next node and resumes at next_id), batch (stores files and fragments, moves next_id), finish, release, plan (from below live indexing / the backfill / earlier plans down to N days back; takes over the group's backfill), status. - Seed API under /api/v1/seed (lease, ranges/:id/batches|done|release, status) behind a node-token middleware; JSON bodies up to 64 MB after gzip. - seed:node runs a node: no database or Redis, outgoing HTTPS only, N connections, gzip batches, retries for network errors and 5xx, drops a range on 409, gives its ranges back on SIGTERM. seed-node/ has a compose file that builds the image from this repository. - seed:token, seed:plan, seed:status for the operator. Header parsing moved to lib/ingest and the range loop to lib/seed/runner, shared by seed:range and seed:node. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
19 lines
733 B
YAML
19 lines
733 B
YAML
# A seeding node (CONCEPT.md D38): leases article ranges from the seed API over
|
|
# HTTPS, indexes their headers over its own NNTP connections, and posts the
|
|
# results back. Outgoing connections only. Builds the indexer image straight
|
|
# from the git repository.
|
|
#
|
|
# cp node.env.example node.env # fill in SEED_TOKEN and the NNTP account
|
|
# docker compose up -d --build
|
|
name: usenet-seed-node
|
|
|
|
services:
|
|
seed-node:
|
|
build: https://gitea.exp.sforum.nl/daan/usenet-indexer.git#master
|
|
image: usenet-seed-node:latest
|
|
command: ["node", "ace", "seed:node"]
|
|
env_file: node.env
|
|
restart: unless-stopped
|
|
# On SIGTERM the node gives its ranges back (resumed from the last batch elsewhere).
|
|
stop_grace_period: 60s
|