Content inspection and archive walks on seeding nodes (relay)
Test and deploy / test (push) Successful in 1m49s
Test and deploy / deploy (push) Successful in 2m15s

Inspection is split into prepare (database), probe (lib/content/probe:
downloads, decoding and parsing, no database) and apply (database).
worker:collection runs all three; seed:node with SEED_INSPECTORS leases
jobs from the same queues through the seed API (BullMQ manual leases),
fetches articles through the seed API and posts the probes back.

A node now needs only SEED_URL and SEED_TOKEN: the settings the app
requires but a node never uses get placeholders for seed:node.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-06 11:17:02 +02:00
1 parent 51975fe603
commit 2dbca9b152
22 files changed
+1198 -540

No files matched your search

+13 -29
View File
@@ -1,34 +1,18 @@
# --- Seed API: the orchestrator, and this node's token (`node ace seed:token <name>` on the indexer).
# The seed API (the orchestrator) and this node's token (`node ace seed:token <name>` on the indexer).
SEED_URL=https://seed.exp.sforum.nl
SEED_TOKEN=sn_replace_me
# Ranges worked on at once (about 0.4 CPU core each).
SEED_CONNECTIONS=4
# relay: headers come through the seed API, no news server account needed (the default).
# direct: the node fetches with its own NNTP account; then fill in the NNTP_* lines below.
SEED_FETCH=relay
# --- Only for SEED_FETCH=direct: the NNTP account (counts against its connection and IP limits).
NNTP_HOST=localhost
NNTP_PORT=563
NNTP_SECURE=true
NNTP_USER=unused
NNTP_PASSWORD=unused
# Ranges seeded at once (about 0.4 CPU core each). 0: no seeding.
SEED_CONNECTIONS=4
# Content inspections and archive walks at once (CPU for decoding and parsing; articles come through the seed API).
SEED_INSPECTORS=0
# Optional: SEED_FETCH=direct seeds with this node's own NNTP account instead of through the seed API.
# SEED_FETCH=direct
# NNTP_HOST=news.eweka.nl
# NNTP_PORT=563
# NNTP_SECURE=true
# NNTP_USER=
# NNTP_PASSWORD=
# Through a tunnel or proxy: the hostname the server's TLS certificate is for.
# NNTP_TLS_SERVERNAME=news.eweka.nl
# --- Required by the app's configuration, but unused on a seeding node (no database, Redis or web server).
NODE_ENV=production
TZ=UTC
PORT=3333
HOST=0.0.0.0
LOG_LEVEL=info
APP_KEY=unused-on-seed-nodes-unused-on-seed-nodes
APP_URL=http://localhost:3333
SESSION_DRIVER=memory
DB_CONNECTION=pg
DB_HOST=localhost
DB_PORT=5432
DB_USER=unused
DB_DATABASE=unused
REDIS_HOST=localhost
REDIS_PORT=6379