The par2 merge: releases whose files one par2 set describes become one
The prober stores, per probed article that starts a file, the MD5 of its
first 16 KiB (file_hashes), and per par2 body the set's whole file list
(par2_files, par2_sets): FileDesc holds the same hash per file, whatever
the file was renamed to. releases:merge (the merger service, no NNTP)
merges every minute the releases of one group, posted within 48 hours of
a release holding the set's par2, whose files the set describes, into the
one with the most files: chunks, probe candidates, inspections, hashes and
counts move; the par2's shared name when none was readable; release_merges
keeps what went where. Obfuscated uploads whose files were each a release
of their own ("[9/20] - "14f5999a…"", hep: 13,232) become one.
Also: corpus_sweep.py counts the marked ranges without psql's command tag.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
9ea2669b2a
commit
3d0dc6ca1f
13 files changed
+502
-9
No files matched your search
+2
-2
@@ -45,10 +45,10 @@ docker compose run --rm --no-deps migrate
|
||||
# stream keeps the node API from exiting.
|
||||
echo "== recreating app services"
|
||||
docker compose stop edge
|
||||
docker compose up -d --no-deps node-api web coordinator prober
|
||||
docker compose up -d --no-deps node-api web coordinator prober merger
|
||||
docker compose up -d --no-deps edge
|
||||
echo "$SHA $(date -Is)" >> "$ROOT/deployed.log"
|
||||
# The images this replaced (321 MB each; the root filesystem is the NVMe, with a 10 GB floor).
|
||||
docker image prune -f --filter label=com.docker.compose.project=usenet-rebuild >/dev/null
|
||||
docker compose ps --format '{{.Service}} {{.Status}}' node-api web coordinator prober edge
|
||||
docker compose ps --format '{{.Service}} {{.Status}}' node-api web coordinator prober merger edge
|
||||
echo "== deployed $SHA"
|
||||
Reference in new issue
Block a user