seed-node: step-by-step guide for running a node on a remote server

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

seed-node/README.md

@@ -3,24 +3,93 @@
 A seeding node indexes old article ranges for the indexer (CONCEPT.md D37,
 D38). It asks the seed API for a range over HTTPS, fetches that range's
 headers over its own NNTP connections, groups them into files, and posts the
-result back. It only makes outgoing connections: no port forwarding, database
-or Redis.
+results back. It only makes outgoing connections: no port forwarding,
+database or Redis.
 
-On the indexer (homelab):
+## What a node needs
 
-    docker compose exec web node ace seed:token my-server         # prints the node's token once
-    docker compose exec web node ace seed:plan alt.binaries.ath --days 365
-    docker compose exec web node ace seed:status
+- Docker with the compose plugin (`docker compose version` works).
+- Outgoing HTTPS to `seed.exp.sforum.nl` (the seed API) and
+  `gitea.exp.sforum.nl` (to build the image), and outgoing NNTP to the news
+  server (port 563 for TLS).
+- Per connection: roughly 0.4 CPU core and 1–3 MB/s download (headers come
+  gzip-compressed); a few hundred MB of RAM per node. Disk: about 1.5 GB for
+  the image.
+- An NNTP account. Every connection a node opens counts against that
+  account's connection limit, next to the ~40 the indexer itself uses if it's
+  the same account. Too many gives `481`/`502` errors at login.
 
-On the node (any machine with Docker):
+## 1. On the indexer (homelab): a token for the node
+
+    cd ~/dev/usenet-indexer
+    docker compose exec web node ace seed:token my-server
+
+This prints a token starting with `sn_` once. Treat it like a password: it
+lets the holder lease ranges and submit results. Running the command again
+for the same name replaces the token; `--disable` revokes it.
+
+## 2. On the server: get the files
+
+    mkdir -p ~/usenet-seed-node && cd ~/usenet-seed-node
+    curl -fsSLO https://gitea.exp.sforum.nl/daan/usenet-indexer/raw/branch/master/seed-node/compose.yaml
+    curl -fsSL -o node.env https://gitea.exp.sforum.nl/daan/usenet-indexer/raw/branch/master/seed-node/node.env.example
+    chmod 600 node.env
+
+## 3. Configure `node.env`
+
+| Setting | Value |
+|---|---|
+| `SEED_URL` | `https://seed.exp.sforum.nl` (leave as is) |
+| `SEED_TOKEN` | the token from step 1 |
+| `SEED_CONNECTIONS` | ranges worked on at once, one NNTP connection each. Start with 2–4. |
+| `NNTP_HOST`, `NNTP_PORT`, `NNTP_SECURE` | the news server; over the internet use TLS: port `563`, `NNTP_SECURE=true` (for Eweka: `news.eweka.nl`) |
+| `NNTP_USER`, `NNTP_PASSWORD` | the NNTP account |
+
+Leave the block below "Required by the app's configuration, but unused" as
+it is: the app checks these settings at startup, but a node never uses them.
+
+## 4. Start it
 
-    cp node.env.example node.env     # SEED_TOKEN, NNTP account, SEED_CONNECTIONS
     docker compose up -d --build
     docker compose logs -f
 
-`docker compose stop` gives the node's ranges back; a node that disappears
-loses its ranges to other nodes after 15 minutes without a batch. Updating:
-`docker compose up -d --build` again.
+The first build takes a few minutes. A working node logs:
 
-Mind the NNTP account's connection limit: every node's SEED_CONNECTIONS
-count against it, next to the indexer's own connections.
+    [ info ] Seeding node for https://seed.exp.sforum.nl with 2 connection(s).
+    [ info ] [1] alt.binaries.ath 49834000000–49835999999
+    [ info ] [1] done: 2,000,000 headers in 237 s (8,450/s)
+
+With nothing planned it waits and asks again every minute (no log lines).
+
+## 5. Plan work and follow it (on homelab)
+
+    docker compose exec web node ace seed:plan alt.binaries.ath alt.binaries.mom --days 365
+    docker compose exec web node ace seed:status
+
+`seed:plan` queues ranges of 2 million articles from below what is already
+indexed down to `--days` back (or everything the server has, without
+`--days`), newest first and interleaved across groups. It takes over the
+rest of those groups' backfill. `seed:status` shows ranges per status,
+articles reached, and per node its leases and headers.
+
+## Day to day
+
+- **Update** after a new version of the indexer:
+  `docker compose build --pull && docker compose up -d`
+- **Stop**: `docker compose stop` gives the node's ranges back; another node
+  resumes them after the last batch that was stored.
+- **A node that disappears** (crash, network) loses its ranges to other
+  nodes after 15 minutes without a batch. A range that fails 5 times is set
+  aside as `failed` (see `seed:status`).
+- **Remove a node**: `docker compose down` on the server, then
+  `docker compose exec web node ace seed:token my-server --disable` on homelab.
+
+## Troubleshooting
+
+| Log line | Meaning |
+|---|---|
+| `The seed API rejected this node token` | wrong or revoked `SEED_TOKEN`; the node stops |
+| `Seed API …: 404 …; retrying` | the seed API is restarting (a deploy on homelab); the node retries |
+| `NNTP: unexpected reply "481 …"` or `"502 …"` | NNTP login refused: wrong account, or too many connections on it; lower `SEED_CONNECTIONS` |
+| `Range … failed: …` | the range is given back and retried later, possibly by another node |
+| `Range … passed to another node; dropping it.` | this node was too slow to report (15 minutes); harmless |