Seed relay: nodes get their headers through the seed API
The seed API keeps a pool of NNTP connections (SEED_NNTP_CONNECTIONS) and serves GET /api/v1/seed/ranges/:id/chunk?from=&to= for leased ranges: the gzip stream the news server sent, passed on unopened. seed:node fetches that way by default (SEED_FETCH=relay), so a node needs only a token and SEED_CONNECTIONS; SEED_FETCH=direct keeps its own NNTP connections. All fetching then comes from the indexer's IP (the account allows two). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
3d5cfb2527
commit
41859636d3
13 files changed
+289
-33
No files matched your search
+11
-3
@@ -1,10 +1,18 @@
|
||||
# Seeding node
|
||||
|
||||
A seeding node indexes old article ranges for the indexer (CONCEPT.md D37,
|
||||
D38). It asks the seed API for a range over HTTPS, fetches that range's
|
||||
headers over its own NNTP connections, groups them into files, and posts the
|
||||
D38). It asks the seed API for a range over HTTPS, gets that range's headers
|
||||
(by default through the seed API, which fetches them from the news server and
|
||||
passes them on gzip-compressed), groups them into files, and posts the
|
||||
results back. It only makes outgoing connections: no port forwarding,
|
||||
database or Redis.
|
||||
database, Redis or news server account. Only a token and
|
||||
`SEED_CONNECTIONS` need setting.
|
||||
|
||||
Headers through the seed API (`SEED_FETCH=relay`, the default) use the
|
||||
indexer's news server account: `SEED_NNTP_CONNECTIONS` in the indexer's
|
||||
`.env` sets how many connections it fetches with for all nodes together.
|
||||
With `SEED_FETCH=direct` a node uses its own NNTP account instead (fill in
|
||||
the NNTP lines in `node.env`).
|
||||
|
||||
## What a node needs
|
||||
|
||||
|
||||
@@ -1,17 +1,19 @@
|
||||
# --- Seed API: the orchestrator, and this node's token (`node ace seed:token <name>` on the indexer).
|
||||
SEED_URL=https://seed.exp.sforum.nl
|
||||
SEED_TOKEN=sn_replace_me
|
||||
# Ranges worked on at once; each uses one NNTP connection of the account below.
|
||||
SEED_CONNECTIONS=2
|
||||
# Ranges worked on at once (about 0.4 CPU core each).
|
||||
SEED_CONNECTIONS=4
|
||||
# relay: headers come through the seed API, no news server account needed (the default).
|
||||
# direct: the node fetches with its own NNTP account; then fill in the NNTP_* lines below.
|
||||
SEED_FETCH=relay
|
||||
|
||||
# --- NNTP account this node fetches headers with (mind its connection limit across all nodes).
|
||||
NNTP_HOST=news.example.com
|
||||
# --- Only for SEED_FETCH=direct: the NNTP account (counts against its connection and IP limits).
|
||||
NNTP_HOST=localhost
|
||||
NNTP_PORT=563
|
||||
NNTP_SECURE=true
|
||||
NNTP_USER=replace_me
|
||||
NNTP_PASSWORD=replace_me
|
||||
# Only when NNTP_HOST is a tunnel or proxy (e.g. localhost:1563 forwarded to news.eweka.nl:563):
|
||||
# the hostname the server's TLS certificate is for.
|
||||
NNTP_USER=unused
|
||||
NNTP_PASSWORD=unused
|
||||
# Through a tunnel or proxy: the hostname the server's TLS certificate is for.
|
||||
# NNTP_TLS_SERVERNAME=news.eweka.nl
|
||||
|
||||
# --- Required by the app's configuration, but unused on a seeding node (no database, Redis or web server).
|
||||
|
||||
Reference in new issue
Block a user