Daan MeijerandClaude Opus 5.5
84d3af5511
Dockerfile with two targets: app (compiled AdonisJS, production dependencies, Node 24.16 as in CI) and edge (the Rust worker, cargo caches as build mounts). deploy/compose.yaml, project usenet-rebuild: postgres (data on the HDD, catalog tablespace on the NVMe), migrate, web, coordinator, edge; edge-ts and ace behind profiles. deploy/deploy.sh (~/bin/deploy-usenet-rebuild on homelab): fetches the commit (must be on rebuild), builds, migrates, recreates web, coordinator and edge (the worker stopped first), never postgres. CI deploys a push to rebuild that passes: the deploy key's forced command passes "deploy-rebuild <sha>" on. A push to master no longer deploys from this branch's workflow. deploy/README.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
68 lines
3.0 KiB
Markdown
68 lines
3.0 KiB
Markdown
# Deploying voyager 2
|
|
|
|
Voyager 2 (branch `rebuild`) runs on homelab as the compose project
|
|
`usenet-rebuild`, separate from voyager 1's (`usenet-indexer`, branch
|
|
`master`): its own Postgres, network, images and directory.
|
|
|
|
A push to `rebuild` that passes CI is deployed: the deploy job SSHes to
|
|
homelab with the deploy key, whose forced command
|
|
(`~/bin/deploy-usenet-indexer`) passes `deploy-rebuild <sha>` on to
|
|
`~/bin/deploy-usenet-rebuild` (this directory's `deploy.sh`).
|
|
|
|
## What runs
|
|
|
|
| Service | Image | Does |
|
|
|---|---|---|
|
|
| `postgres` | postgres:17 | the catalog (tablespace `nvme`) and the chunks (HDD) |
|
|
| `migrate` | `usenet-rebuild` (target `app`) | `migration:run`, before every deploy |
|
|
| `web` | `usenet-rebuild` | node API, catalog API, Newznab, web interface; port 3334 |
|
|
| `coordinator` | `usenet-rebuild` | prefetching (all NNTP connections), planning, the tail, merging, settling |
|
|
| `edge` | `usenet-rebuild-edge` (target `edge`) | the Rust worker, `--threads $EDGE_THREADS` (8) |
|
|
| `edge-ts` | `usenet-rebuild` | the TypeScript worker, profile `edge-ts`, not started |
|
|
| `ace` | `usenet-rebuild` | one-off ace commands, profile `tools` |
|
|
|
|
## On homelab
|
|
|
|
`/mnt/bulk/usenet-rebuild` (`/home` is nearly full):
|
|
|
|
| Path | |
|
|
|---|---|
|
|
| `compose.yaml` | copied from `src/deploy/compose.yaml` on every deploy |
|
|
| `.env` | credentials and tuning, mode 600; not in git (below) |
|
|
| `src/` | the code at the deployed commit (the build context) |
|
|
| `pg/` | Postgres data (HDD) |
|
|
| `spool/` | prefetched XOVER chunks |
|
|
| `deployed.log` | one line per deploy: sha, time (local) |
|
|
| `.cache/repo.git` | bare clone the deploys fetch into |
|
|
| `app/` | an older copy of the code with dev dependencies, for `tools/*.ts` (`r.sh`); not what runs |
|
|
|
|
The catalog's tablespace is `/var/lib/usenet-rebuild-catalog` on the NVMe
|
|
(`PG_NVME_DIR`). Keep 10 GB free on the root filesystem.
|
|
|
|
`.env` holds `APP_KEY`, `DB_USER`, `DB_PASSWORD`, `DB_DATABASE`, the `NNTP_*`
|
|
account, `NODE_TOKEN` (the edge worker's token, from `nodes:create`),
|
|
`CHUNK_TABLESPACE=pg_default`, `TZ`, `LOG_LEVEL`, `SESSION_DRIVER` and
|
|
optional tuning (`NNTP_CONNECTIONS`, `EDGE_THREADS`, `PG_SHARED_BUFFERS`,
|
|
anything in `start/env.ts`). Hosts, ports and paths are set in compose.yaml.
|
|
|
|
## By hand
|
|
|
|
```sh
|
|
cd /mnt/bulk/usenet-rebuild
|
|
~/bin/deploy-usenet-rebuild <sha> # what CI runs
|
|
docker compose ps
|
|
docker compose logs -f --since 5m coordinator
|
|
docker compose run --rm ace releases:duplicates
|
|
docker compose stop edge coordinator # pause indexing (the worker first)
|
|
docker compose up -d --no-deps edge coordinator
|
|
```
|
|
|
|
- Recreate app services only, with `--no-deps`; Postgres only on purpose,
|
|
with the worker and coordinator stopped first.
|
|
- Stop `edge` before `web`: a worker holding a stream keeps the server from
|
|
exiting.
|
|
- The NNTP account is shared with voyager 1 and downloads: `NNTP_CONNECTIONS`
|
|
(40) plus one plain connection, out of 50.
|
|
- `deploy.sh` doesn't update itself: after changing it, install it again
|
|
(`install -m 755 src/deploy/deploy.sh ~/bin/deploy-usenet-rebuild`).
|