Seed relay: compressed overviews by default again
Test and deploy / test (push) Successful in 1m3s
Test and deploy / deploy (push) Successful in 1m51s

Measured with 20 relay connections, plain and gzip both gave ~1.5 chunks/s
for 2026 a.b.ath articles; plain used many times the CPU and traffic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-06 16:14:47 +02:00
1 parent 3e6b5bed56
commit 3cc69a617f
3 files changed
+7 -7

No files matched your search

+1 -2
View File
@@ -31,8 +31,7 @@ class SeedFetchPool {
}
/**
* Pooled connections fetch compressed only with SEED_NNTP_COMPRESS (plain is
* faster for recent articles). A failure is retried once without compression, on a connection of its
* Pooled connections fetch compressed unless SEED_NNTP_COMPRESS=false. A failure is retried once without compression, on a connection of its
* own: either the pooled connection was dead (dropped while idle), or the
* server stalled mid-reply in compressed mode. Eweka does that for some
* chunks: it stops after most of the stream and never sends the rest
+5 -4
View File
@@ -73,11 +73,12 @@ export default {
*/
nntpTimeoutMs: env.get('SEED_NNTP_TIMEOUT_MS', 90 * 1000),
/**
* Ask the news server for gzip-compressed overviews. Off: for 2026 articles Eweka's
* compressed replies have a long tail (p90 12.4 s per 20k, against 3.9 s plain; 6.1k
* against 10.5k headers/s, LOG.md 2026-10-06 (9)). Nodes get gzip from the relay either way.
* Ask the news server for gzip-compressed overviews (SEED_NNTP_COMPRESS=false: plain).
* On one connection plain had a shorter tail for 2026 articles, but with 20 relay
* connections both gave ~1.5 chunks/s, plain at many times the relay's CPU and
* more traffic (LOG.md 2026-10-06 (9)).
*/
nntpCompress: env.get('SEED_NNTP_COMPRESS', false),
nntpCompress: env.get('SEED_NNTP_COMPRESS', true),
},
backfill: {
+1 -1
View File
@@ -27,7 +27,7 @@ test.group('SeedFetchPool', () => {
})
await new Promise<void>((resolve) => server.listen(0, '127.0.0.1', resolve))
const saved = { host: nntpConfig.host, port: nntpConfig.port, secure: nntpConfig.secure, connections: indexerConfig.seed.nntpConnections, compress: indexerConfig.seed.nntpCompress }
// The fallback from a compressed pooled connection (SEED_NNTP_COMPRESS on).
// The fallback from a compressed pooled connection (the default).
indexerConfig.seed.nntpCompress = true
Object.assign(nntpConfig, { host: '127.0.0.1', port: (server.address() as net.AddressInfo).port, secure: false })
indexerConfig.seed.nntpConnections = 2