Idle connections fetch capture ranges into the corpus
The spool only fetches what workers lease soon, in priority order, and the tail never ends: after a reset the 1,400 capture ranges waited behind it while 34 of 40 connections idled. Up to CORPUS_AHEAD (8) planned capture ranges whose chunks aren't all stored are now targets after everything else, fetched into the corpus only (no spool); a later lease reads them from disk. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
0fafe71b21
commit
7b436f3f77
5 files changed
+106
-3
No files matched your search
@@ -61,6 +61,7 @@ export default await Env.create(new URL('../', import.meta.url), {
|
||||
LEASE_TTL_SECONDS: Env.schema.number.optional(),
|
||||
SPOOL_DIR: Env.schema.string.optional(),
|
||||
CORPUS_DIR: Env.schema.string.optional(),
|
||||
CORPUS_AHEAD: Env.schema.number.optional(),
|
||||
PROBE_PIPELINE: Env.schema.number.optional(),
|
||||
SPOOL_AHEAD_PER_LEASE: Env.schema.number.optional(),
|
||||
SPOOL_MIN_AHEAD: Env.schema.number.optional(),
|
||||
|
||||
Reference in new issue
Block a user