Daan MeijerandClaude Opus 5.5
8bd6d6aa5f
probe:run (compose service prober, PROBE_CONNECTIONS, default 6) claims waiting candidates of settled releases (lone, then par2, then archive), fetches each article with BODY over connections of its own (no compression) and inspects it: the yEnc name and size, the content type from the first bytes, a par2 set's files, a RAR volume's entries and whether they're encrypted. Results per release and kind in release_inspections (CHUNK_TABLESPACE, the HDD); on the release passworded, inner_name (the largest content name) and inspected_at. 430 = missing, not yEnc = error, both final; connection trouble puts the candidate back. Fetch and parse in one process: an article takes ~0.1 s per connection, parsing a few ms (no queue in between). Parsers from voyager 1 (lib/yenc, lib/rar, lib/par2, lib/content sniff) with their tests; XoverClient.body; yencode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
55 lines
2.1 KiB
Bash
Executable File
55 lines
2.1 KiB
Bash
Executable File
#!/bin/sh
|
|
# Deploys voyager 2 (branch rebuild) on homelab at one commit. Installed as
|
|
# ~/bin/deploy-usenet-rebuild (by hand: it doesn't update itself). Gitea CI
|
|
# runs it through the forced SSH command of the deploy key, as
|
|
# "deploy-rebuild <sha>" (~/bin/deploy-usenet-indexer passes that on); by
|
|
# hand: deploy-usenet-rebuild <sha>. The code comes from the Gitea repo, not
|
|
# from the CI job.
|
|
set -eu
|
|
|
|
REPO=https://gitea.exp.sforum.nl/daan/usenet-indexer.git
|
|
BRANCH=rebuild
|
|
ROOT=/mnt/bulk/usenet-rebuild
|
|
CACHE="$ROOT/.cache"
|
|
GIT="$CACHE/repo.git"
|
|
|
|
SHA="${1:-}"
|
|
case "$SHA" in
|
|
*[!0-9a-f]* | "") echo "usage: deploy-usenet-rebuild <40-character commit sha>" >&2; exit 2 ;;
|
|
esac
|
|
[ "${#SHA}" -eq 40 ] || { echo "usage: deploy-usenet-rebuild <40-character commit sha>" >&2; exit 2; }
|
|
|
|
mkdir -p "$CACHE"
|
|
# One deploy at a time; a second one waits.
|
|
exec 9>"$CACHE/deploy.lock"
|
|
flock 9
|
|
|
|
echo "== fetching $SHA"
|
|
[ -d "$GIT" ] || git clone --quiet --bare "$REPO" "$GIT"
|
|
git -C "$GIT" fetch --quiet --prune "$REPO" '+refs/heads/*:refs/heads/*'
|
|
git -C "$GIT" cat-file -e "$SHA^{commit}"
|
|
git -C "$GIT" merge-base --is-ancestor "$SHA" "refs/heads/$BRANCH" || { echo "$SHA is not on $BRANCH" >&2; exit 2; }
|
|
|
|
STAGE=$(mktemp -d)
|
|
trap 'rm -rf "$STAGE"' EXIT
|
|
git -C "$GIT" archive "$SHA" | tar -x -C "$STAGE"
|
|
rsync -a --delete "$STAGE"/ "$ROOT/src/"
|
|
install -m 644 "$ROOT/src/deploy/compose.yaml" "$ROOT/compose.yaml"
|
|
|
|
cd "$ROOT"
|
|
echo "== building images"
|
|
docker compose build migrate edge
|
|
echo "== running migrations"
|
|
docker compose run --rm --no-deps migrate
|
|
# Only the app services, never postgres. The worker first: one holding a
|
|
# stream keeps the web server from exiting.
|
|
echo "== recreating app services"
|
|
docker compose stop edge
|
|
docker compose up -d --no-deps web coordinator prober
|
|
docker compose up -d --no-deps edge
|
|
echo "$SHA $(date -Is)" >> "$ROOT/deployed.log"
|
|
# The images this replaced (321 MB each; the root filesystem is the NVMe, with a 10 GB floor).
|
|
docker image prune -f --filter label=com.docker.compose.project=usenet-rebuild >/dev/null
|
|
docker compose ps --format '{{.Service}} {{.Status}}' web coordinator prober edge
|
|
echo "== deployed $SHA"
|