Seed relay: compressed overviews by default again
Measured with 20 relay connections, plain and gzip both gave ~1.5 chunks/s for 2026 a.b.ath articles; plain used many times the CPU and traffic. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
3e6b5bed56
commit
3cc69a617f
3 files changed
+7
-7
No files matched your search
@@ -31,8 +31,7 @@ class SeedFetchPool {
|
||||
}
|
||||
|
||||
/**
|
||||
* Pooled connections fetch compressed only with SEED_NNTP_COMPRESS (plain is
|
||||
* faster for recent articles). A failure is retried once without compression, on a connection of its
|
||||
* Pooled connections fetch compressed unless SEED_NNTP_COMPRESS=false. A failure is retried once without compression, on a connection of its
|
||||
* own: either the pooled connection was dead (dropped while idle), or the
|
||||
* server stalled mid-reply in compressed mode. Eweka does that for some
|
||||
* chunks: it stops after most of the stream and never sends the rest
|
||||
|
||||
+5
-4
@@ -73,11 +73,12 @@ export default {
|
||||
*/
|
||||
nntpTimeoutMs: env.get('SEED_NNTP_TIMEOUT_MS', 90 * 1000),
|
||||
/**
|
||||
* Ask the news server for gzip-compressed overviews. Off: for 2026 articles Eweka's
|
||||
* compressed replies have a long tail (p90 12.4 s per 20k, against 3.9 s plain; 6.1k
|
||||
* against 10.5k headers/s, LOG.md 2026-10-06 (9)). Nodes get gzip from the relay either way.
|
||||
* Ask the news server for gzip-compressed overviews (SEED_NNTP_COMPRESS=false: plain).
|
||||
* On one connection plain had a shorter tail for 2026 articles, but with 20 relay
|
||||
* connections both gave ~1.5 chunks/s, plain at many times the relay's CPU and
|
||||
* more traffic (LOG.md 2026-10-06 (9)).
|
||||
*/
|
||||
nntpCompress: env.get('SEED_NNTP_COMPRESS', false),
|
||||
nntpCompress: env.get('SEED_NNTP_COMPRESS', true),
|
||||
},
|
||||
|
||||
backfill: {
|
||||
|
||||
@@ -27,7 +27,7 @@ test.group('SeedFetchPool', () => {
|
||||
})
|
||||
await new Promise<void>((resolve) => server.listen(0, '127.0.0.1', resolve))
|
||||
const saved = { host: nntpConfig.host, port: nntpConfig.port, secure: nntpConfig.secure, connections: indexerConfig.seed.nntpConnections, compress: indexerConfig.seed.nntpCompress }
|
||||
// The fallback from a compressed pooled connection (SEED_NNTP_COMPRESS on).
|
||||
// The fallback from a compressed pooled connection (the default).
|
||||
indexerConfig.seed.nntpCompress = true
|
||||
Object.assign(nntpConfig, { host: '127.0.0.1', port: (server.address() as net.AddressInfo).port, secure: false })
|
||||
indexerConfig.seed.nntpConnections = 2
|
||||
|
||||
Reference in new issue
Block a user