Rebuild spec: measurements after the second performance round
Test and deploy / test (push) Successful in 23s
Test and deploy / deploy (push) Skipped

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-09 03:37:04 +02:00
1 parent 69f1cc63b2
commit 53bfea8d15
1 file changed
+20 -10
+20 -10
View File
@@ -14,17 +14,27 @@ prefetch spool, leases, rule sets, fragment merger, settling), the node API
(`edge:node`, `--processes`), the catalog and chunk store, NZBs and a name
search. Measured on the workstation (4 cores, 100 Mbit, Eweka):
- Edge worker, one core: ~190–215 k headers/s on recent a.b.boneless,
~330 k on a 2013 a.b.ath range; ~0.66 GB peak per process. (Start:
73–103 k/s, 1.0–1.2 GB.)
- NNTP: ~22–30 k overview lines/s per connection in chunks of 100 k
(Eweka takes ~1.5–2 s per command); connections don't slow each other.
- End to end (coordinator with 10 connections, web server, Postgres and 3
worker processes on one 4-core machine): 244 k headers/s; the server side
used 10% (web) + 22% (coordinator) + 32% (Postgres) of a core.
- Edge worker, one core: ~210–215 k headers/s on recent a.b.boneless,
~330–340 k on a 2013 a.b.ath range; ~0.7–0.8 GB peak per process.
(Start: 73–103 k/s, 1.0–1.2 GB.) What's left is spread out (the
per-file key ~12%, GC ~10%, zlib ~10%, the scan ~6%).
- NNTP: Eweka takes ~1.5–2 s per XOVER command; ~22–30 k lines/s per
connection in chunks of 100 k when the line allows; connections don't
slow each other.
- Bandwidth: compressed XOVER is ~69 B per article on recent boneless
(100 k-article chunks of ~7 MB). The workstation's 100 Mbit line carries
~140–150 k articles/s, which is where its end-to-end runs now stop (10
connections 90–99% busy, ~89 Mbit/s).
- End to end at that rate (one 4-core machine): workers about half busy
each; server side ~17% (coordinator) + ~8% (web) + ~3% (Postgres) of a
core. (Before this round's fixes Postgres was at 32%: the merger's
neighbour query read every range per file.)
- Storage: chunks 17.9–20.3 B per segment (format v2); a release ~1.1 KB.
- For the whole target (528 B overview lines): ~4–6 days at 50 connections,
~3 worker cores.
- For the whole target (528 B overview lines): ~36 TB of compressed XOVER
to download. At 50 connections (~1–1.25 M lines/s, ~0.6–0.7 Gbit/s)
~5–6 days, ~5–6 worker cores, two coordinator shards (one process
handles ~60 MB/s), under a core for web and Postgres. The line is the
limit below ~0.7 Gbit/s: at 1 Gbit, ~3.5 days at best.
Not built yet: re-running ranges for a retracted rule (the counts and rule
versions are recorded), deployment (Dockerfile, compose), phase 2.