Rebuild: the edge workers' API in its own service
The node API (leases, hour-long streams, batches of up to 256 MB JSON parsed in-process) shared the web process with the catalog API, Newznab and the web interface; voyager 1 had to make the same split on 2026-10-06. HTTP_ROUTES picks a process's routes (node, public, all by default): compose service node-api (same image, port 3335 on the compose network) serves the workers, web the rest. The workers' COORDINATOR_URL points at node-api; deploy.sh stops edge before recreating node-api. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
34fea342f5
commit
48f4e982f9
5 files changed
+87
-52
No files matched your search
+22
-8
@@ -66,14 +66,27 @@ services:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
|
||||
# Node API for the edge workers (leases, streams, batches), catalog API,
|
||||
# Newznab and the web interface.
|
||||
# Catalog API, Newznab and the web interface.
|
||||
web:
|
||||
<<: *app
|
||||
command: ['node', 'bin/server.js']
|
||||
environment:
|
||||
<<: *app-env
|
||||
HTTP_ROUTES: public
|
||||
ports:
|
||||
- '3334:3334'
|
||||
|
||||
# The edge workers' API (leases, streams, batches), apart from the web
|
||||
# interface: streams hold a connection up to an hour, batches of up to
|
||||
# 256 MB are parsed in-process. Only the compose network reaches it.
|
||||
node-api:
|
||||
<<: *app
|
||||
command: ['node', 'bin/server.js']
|
||||
environment:
|
||||
<<: *app-env
|
||||
HTTP_ROUTES: node
|
||||
PORT: 3335
|
||||
|
||||
# Prefetching (all NNTP connections), planning, the tail, merging, settling.
|
||||
coordinator:
|
||||
<<: *app
|
||||
@@ -92,8 +105,9 @@ services:
|
||||
<<: *app-env
|
||||
PROBE_CONNECTIONS: ${PROBE_CONNECTIONS:-6}
|
||||
|
||||
# The Rust worker; it has no NNTP account and talks to web only. Stopped
|
||||
# before web (a worker holding a stream keeps the server from exiting).
|
||||
# The Rust worker; it has no NNTP account and talks to node-api only.
|
||||
# Stopped before node-api (a worker holding a stream keeps the server from
|
||||
# exiting).
|
||||
edge:
|
||||
image: usenet-rebuild-edge:latest
|
||||
build:
|
||||
@@ -102,11 +116,11 @@ services:
|
||||
restart: unless-stopped
|
||||
command: ['--threads', '${EDGE_THREADS:-8}']
|
||||
environment:
|
||||
COORDINATOR_URL: http://web:3334
|
||||
COORDINATOR_URL: http://node-api:3335
|
||||
NODE_TOKEN: ${NODE_TOKEN}
|
||||
stop_grace_period: 60s
|
||||
depends_on:
|
||||
web:
|
||||
node-api:
|
||||
condition: service_started
|
||||
|
||||
# The TypeScript worker, for comparing: docker compose --profile edge-ts up -d edge-ts
|
||||
@@ -116,9 +130,9 @@ services:
|
||||
command: ['node', 'ace', 'edge:node', '--processes', '${EDGE_THREADS:-8}']
|
||||
environment:
|
||||
<<: *app-env
|
||||
COORDINATOR_URL: http://web:3334
|
||||
COORDINATOR_URL: http://node-api:3335
|
||||
depends_on:
|
||||
web:
|
||||
node-api:
|
||||
condition: service_started
|
||||
|
||||
# One-off ace commands: docker compose run --rm ace releases:duplicates
|
||||
|
||||
Reference in new issue
Block a user