Rebuild: hedged chunk fetches and connection max age (both off by default)
Test and deploy / test (push) Successful in 24s
Test and deploy / deploy (push) Skipped

A chunk not in after NNTP_HEDGE_AFTER_SECONDS (10) is asked again on one
of NNTP_HEDGE_CONNECTIONS spare connections; the first answer wins and
the other connection is closed (lib/nntp/hedge.ts, tested). Connections
can be retired after NNTP_CONNECTION_MAX_MINUTES (±25%). The prefetch
report counts hedges and slow first requests by connection age.

Homelab, 15 min each: hedging on (40 + 6) 280k articles/s, off 273k.
The second request won about half of its races, but ~25-35% of first
requests were slow (>10 s) at every connection age, so neither helps:
Eweka is slow under our load, not on particular connections. Both
off by default; the age statistics stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-09 22:13:44 +02:00
1 parent fc8ec9ff48
commit 8475969145
7 files changed
+234 -11

No files matched your search

+18
View File
@@ -37,6 +37,24 @@ export default {
* the prefetcher nearly stopped. Data arrives within seconds otherwise.
*/
nntpTimeoutMs: env.get('NNTP_TIMEOUT_MS', 2 * 60 * 1000),
/**
* A chunk not fetched after this long is asked again on another
* connection; the first answer wins and the other connection is closed.
* Eweka stalls single requests: the same chunk took 27–52 s on one
* connection and 1–9 s on another at the same moment; median ~3 s
* (LOG.md 2026-10-09 (15)). Measured in production (40 + 6 connections vs
* 40, 15 min each): 280k vs 273k articles/s, within the noise; the second
* request won half the time. Off by default.
*/
nntpHedgeAfterMs: env.get('NNTP_HEDGE_AFTER_SECONDS', 10) * 1000,
/** Of nntpConnections, kept for those second requests (the prefetcher's loops are the rest). 0: no hedging. */
nntpHedgeConnections: env.get('NNTP_HEDGE_CONNECTIONS', 0),
/**
* A connection is closed after this long (when it's free), and a fresh one
* opened. 0: never. Requests were as often slow (>10 s) on connections
* under 2 minutes old as on ones over 10 (~25–35%; LOG.md 2026-10-09 (16)).
*/
nntpConnectionMaxMs: env.get('NNTP_CONNECTION_MAX_MINUTES', 0) * 60 * 1000,
spool: {
dir: env.get('SPOOL_DIR', app.tmpPath('spool')),
/** Ranges kept fetched ahead, per active lease. */