Rebuild: phase 2's prober reads the probe queue
probe:run (compose service prober, PROBE_CONNECTIONS, default 6) claims waiting candidates of settled releases (lone, then par2, then archive), fetches each article with BODY over connections of its own (no compression) and inspects it: the yEnc name and size, the content type from the first bytes, a par2 set's files, a RAR volume's entries and whether they're encrypted. Results per release and kind in release_inspections (CHUNK_TABLESPACE, the HDD); on the release passworded, inner_name (the largest content name) and inspected_at. 430 = missing, not yEnc = error, both final; connection trouble puts the candidate back. Fetch and parse in one process: an article takes ~0.1 s per connection, parsing a few ms (no queue in between). Parsers from voyager 1 (lib/yenc, lib/rar, lib/par2, lib/content sniff) with their tests; XoverClient.body; yencode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
d72fe2f13f
commit
8bd6d6aa5f
30 files changed
+1666
-7
No files matched your search
@@ -82,6 +82,16 @@ services:
|
||||
<<: *app-env
|
||||
NNTP_CONNECTIONS: ${NNTP_CONNECTIONS:-40}
|
||||
|
||||
# Phase 2's prober: reads the probe queue's articles over PROBE_CONNECTIONS
|
||||
# connections of its own (next to the coordinator's 40 + 1; the account has 50).
|
||||
prober:
|
||||
<<: *app
|
||||
command: ['node', 'ace', 'probe:run']
|
||||
volumes: []
|
||||
environment:
|
||||
<<: *app-env
|
||||
PROBE_CONNECTIONS: ${PROBE_CONNECTIONS:-6}
|
||||
|
||||
# The Rust worker; it has no NNTP account and talks to web only. Stopped
|
||||
# before web (a worker holding a stream keeps the server from exiting).
|
||||
edge:
|
||||
|
||||
Reference in new issue
Block a user