Idle connections fetch capture ranges into the corpus
Test and deploy / test (push) Successful in 42s
Test and deploy / edge (push) Successful in 1m48s
Test and deploy / deploy (push) Successful in 1m6s

The spool only fetches what workers lease soon, in priority order, and the
tail never ends: after a reset the 1,400 capture ranges waited behind it
while 34 of 40 connections idled. Up to CORPUS_AHEAD (8) planned capture
ranges whose chunks aren't all stored are now targets after everything else,
fetched into the corpus only (no spool); a later lease reads them from disk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-12 20:11:57 +02:00
1 parent 0fafe71b21
commit 7b436f3f77
5 files changed
+106 -3

No files matched your search

+1
View File
@@ -61,6 +61,7 @@ export default await Env.create(new URL('../', import.meta.url), {
LEASE_TTL_SECONDS: Env.schema.number.optional(),
SPOOL_DIR: Env.schema.string.optional(),
CORPUS_DIR: Env.schema.string.optional(),
CORPUS_AHEAD: Env.schema.number.optional(),
PROBE_PIPELINE: Env.schema.number.optional(),
SPOOL_AHEAD_PER_LEASE: Env.schema.number.optional(),
SPOOL_MIN_AHEAD: Env.schema.number.optional(),