Rebuild: hedged chunk fetches and connection max age (both off by default)
A chunk not in after NNTP_HEDGE_AFTER_SECONDS (10) is asked again on one of NNTP_HEDGE_CONNECTIONS spare connections; the first answer wins and the other connection is closed (lib/nntp/hedge.ts, tested). Connections can be retired after NNTP_CONNECTION_MAX_MINUTES (±25%). The prefetch report counts hedges and slow first requests by connection age. Homelab, 15 min each: hedging on (40 + 6) 280k articles/s, off 273k. The second request won about half of its races, but ~25-35% of first requests were slow (>10 s) at every connection age, so neither helps: Eweka is slow under our load, not on particular connections. Both off by default; the age statistics stay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
fc8ec9ff48
commit
8475969145
7 files changed
+234
-11
No files matched your search
@@ -37,6 +37,24 @@ export default {
|
||||
* the prefetcher nearly stopped. Data arrives within seconds otherwise.
|
||||
*/
|
||||
nntpTimeoutMs: env.get('NNTP_TIMEOUT_MS', 2 * 60 * 1000),
|
||||
/**
|
||||
* A chunk not fetched after this long is asked again on another
|
||||
* connection; the first answer wins and the other connection is closed.
|
||||
* Eweka stalls single requests: the same chunk took 27–52 s on one
|
||||
* connection and 1–9 s on another at the same moment; median ~3 s
|
||||
* (LOG.md 2026-10-09 (15)). Measured in production (40 + 6 connections vs
|
||||
* 40, 15 min each): 280k vs 273k articles/s, within the noise; the second
|
||||
* request won half the time. Off by default.
|
||||
*/
|
||||
nntpHedgeAfterMs: env.get('NNTP_HEDGE_AFTER_SECONDS', 10) * 1000,
|
||||
/** Of nntpConnections, kept for those second requests (the prefetcher's loops are the rest). 0: no hedging. */
|
||||
nntpHedgeConnections: env.get('NNTP_HEDGE_CONNECTIONS', 0),
|
||||
/**
|
||||
* A connection is closed after this long (when it's free), and a fresh one
|
||||
* opened. 0: never. Requests were as often slow (>10 s) on connections
|
||||
* under 2 minutes old as on ones over 10 (~25–35%; LOG.md 2026-10-09 (16)).
|
||||
*/
|
||||
nntpConnectionMaxMs: env.get('NNTP_CONNECTION_MAX_MINUTES', 0) * 60 * 1000,
|
||||
spool: {
|
||||
dir: env.get('SPOOL_DIR', app.tmpPath('spool')),
|
||||
/** Ranges kept fetched ahead, per active lease. */
|
||||
|
||||
Reference in new issue
Block a user