The pending ladder in the prober: unnamed releases probed cheapest first, signatures learned, the named moved into the catalog
pending_ladder.ts claims the oldest waiting pending releases' next target (par2, archive, lone, 7z-tail in that order) as at least PENDING_SHARE of each claim; a usable name (the largest named file of a par2 set or a listing, or a lone file's yEnc name, neither packaging nor random) hands the release to the namer, no name the next target or the end of the climb; climbs count on pending_signatures per (tool, group, poster) and (tool, group), and a signature with 100 climbs and 0.5% named or fewer is dropped: its waiting releases are discarded without a probe, the rules' sample excepted. pending_namer.ts reads a named release's articles again from the corpus or by XOVER and writes it to the catalog under the pending key (source weak-stem, the name as inner name) with its chunk and candidates. Migration 6400; config PENDING_*. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
9cf85e09d8
commit
e135076fcd
14 files changed
+1045
-50
No files matched your search
@@ -103,6 +103,14 @@ export default {
|
||||
batch: 400,
|
||||
/** BODY commands in flight per connection (pipelined; XoverClient.bodies). */
|
||||
pipeline: env.get('PROBE_PIPELINE', 8),
|
||||
/**
|
||||
* The pending ladder (REBUILD.md decision f; app/services/pending_ladder.ts): the share of each claim taken
|
||||
* from pending_releases at least (the catalog's own candidates come first, the rest of the claim is theirs).
|
||||
*/
|
||||
pendingShare: env.get('PENDING_SHARE', 0.25),
|
||||
/** A signature (tool, group, poster) with this many climbs and at most pendingMaxNamed of them named is dropped. */
|
||||
pendingMinProbes: env.get('PENDING_MIN_PROBES', 100),
|
||||
pendingMaxNamed: env.get('PENDING_MAX_NAMED', 0.005),
|
||||
},
|
||||
/**
|
||||
* The catalog's tablespace may grow to this many GB (app/services/catalog_space.ts); above it only the tail and
|
||||
|
||||
Reference in new issue
Block a user