Spots: the spotter and the spot inspector, passwords kept apart

Two processes, one NNTP connection each (the coordinator 40 -> 38):
- spots:read (service spotter): free.pt's overviews after the last article
  seen, each spot's full header verified (lib/spotnet from voyager 1, as
  Spotweb), its description, website and NZB articles; the password its
  title or text names; delete/dispose messages applied.
- spots:inspect (service spot-inspector): each spot's compressed NZB read,
  its files listed (spot_files), each matched to the header pass's release
  by the same release identity in one of its groups within 48 h
  (release_spots); unmatched spots matched again hourly, after 24 h a
  release of their own (source spot) with the spot's NZB.
- release_passwords: per release and password, its source (nzb, spot-title,
  spot-text) and spot; the catalog keeps only passworded.
XoverClient.head(); PROBE_CONNECTIONS default 10.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

deploy/README.md

@@ -20,6 +20,8 @@ homelab with the deploy key, whose forced command
 | `coordinator` | `usenet-rebuild` | prefetching (all NNTP connections), planning, the tail, merging, settling |
 | `prober` | `usenet-rebuild` | phase 2: reads the probe queue's articles over `PROBE_CONNECTIONS` (6) connections, stores what they show |
 | `merger` | `usenet-rebuild` | the par2 merge: releases whose files one par2 set describes become one (no NNTP) |
+| `spotter` | `usenet-rebuild` | Spotnet: free.pt's spots into `spots`, signatures checked, the passwords they name kept; 1 connection |
+| `spot-inspector` | `usenet-rebuild` | each spot's NZB: its files in `spot_files`, linked to the releases they match (`release_spots`), passwords in `release_passwords`; 1 connection |
 | `edge` | `usenet-rebuild-edge` (target `edge`) | the Rust worker, `--threads $EDGE_THREADS` (8) |
 | `edge-ts` | `usenet-rebuild` | the TypeScript worker, profile `edge-ts`, not started |
 | `ace` | `usenet-rebuild` | one-off ace commands, profile `tools` |

deploy/compose.yaml

@@ -101,17 +101,18 @@ services:
     command: ['node', 'ace', 'coordinator:run']
     environment:
       <<: *app-env
-      NNTP_CONNECTIONS: ${NNTP_CONNECTIONS:-40}
+      NNTP_CONNECTIONS: ${NNTP_CONNECTIONS:-38}
 
   # Phase 2's prober: reads the probe queue's articles over PROBE_CONNECTIONS
-  # connections of its own (next to the coordinator's 40 + 1; the account has 50).
+  # connections of its own. The account has 50: the coordinator's 38 (+ its
+  # plain one), the prober's 10, the spotter's and the spot inspector's 1 each.
   prober:
     <<: *app
     command: ['node', 'ace', 'probe:run']
     volumes: []
     environment:
       <<: *app-env
-      PROBE_CONNECTIONS: ${PROBE_CONNECTIONS:-6}
+      PROBE_CONNECTIONS: ${PROBE_CONNECTIONS:-10}
 
   # The par2 merge: releases whose files one par2 set describes (what the prober read) become one. No NNTP.
   merger:
@@ -119,6 +120,19 @@ services:
     command: ['node', 'ace', 'releases:merge']
     volumes: []
 
+  # Spotnet: the spotter reads free.pt's spots (verified, with the passwords
+  # they name) into the spots table; the spot inspector reads each spot's NZB,
+  # lists its files and links the releases they match. One connection each.
+  spotter:
+    <<: *app
+    command: ['node', 'ace', 'spots:read']
+    volumes: []
+
+  spot-inspector:
+    <<: *app
+    command: ['node', 'ace', 'spots:inspect']
+    volumes: []
+
   # The Rust worker; it has no NNTP account and talks to node-api only.
   # Stopped before node-api (a worker holding a stream keeps the server from
   # exiting).

deploy/deploy.sh

@@ -45,10 +45,10 @@ docker compose run --rm --no-deps migrate
 # stream keeps the node API from exiting.
 echo "== recreating app services"
 docker compose stop edge
-docker compose up -d --no-deps node-api web coordinator prober merger
+docker compose up -d --no-deps node-api web coordinator prober merger spotter spot-inspector
 docker compose up -d --no-deps edge
 echo "$SHA $(date -Is)" >> "$ROOT/deployed.log"
 # The images this replaced (321 MB each; the root filesystem is the NVMe, with a 10 GB floor).
 docker image prune -f --filter label=com.docker.compose.project=usenet-rebuild >/dev/null
-docker compose ps --format '{{.Service}} {{.Status}}' node-api web coordinator prober merger edge
+docker compose ps --format '{{.Service}} {{.Status}}' node-api web coordinator prober merger spotter spot-inspector edge
 echo "== deployed $SHA"