Idle connections fetch capture ranges into the corpus
The spool only fetches what workers lease soon, in priority order, and the tail never ends: after a reset the 1,400 capture ranges waited behind it while 34 of 40 connections idled. Up to CORPUS_AHEAD (8) planned capture ranges whose chunks aren't all stored are now targets after everything else, fetched into the corpus only (no spool); a later lease reads them from disk. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
0fafe71b21
commit
7b436f3f77
5 files changed
+106
-3
No files matched your search
@@ -80,6 +80,12 @@ export default {
|
||||
/** Stored XOVER chunks of ranges marked `capture`, for the grouping sweeps (app/services/corpus.ts). Off when unset. */
|
||||
corpus: {
|
||||
dir: env.get('CORPUS_DIR'),
|
||||
/**
|
||||
* Capture ranges fetched into the corpus by connections the spool leaves idle, this many at a time: the
|
||||
* spool only fetches what workers lease soon, and the tail comes first, so after a reset the capture
|
||||
* ranges waited behind it for hours (1,400 marked, none stored, 34 of 40 connections idle; LOG.md 2026-10-12 (27)).
|
||||
*/
|
||||
ahead: env.get('CORPUS_AHEAD', 8),
|
||||
},
|
||||
spool: {
|
||||
dir: env.get('SPOOL_DIR', app.tmpPath('spool')),
|
||||
|
||||
Reference in new issue
Block a user