Rebuild spec: measurements after the second performance round
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
69f1cc63b2
commit
53bfea8d15
1 file changed
+20
-10
+20
-10
@@ -14,17 +14,27 @@ prefetch spool, leases, rule sets, fragment merger, settling), the node API
|
||||
(`edge:node`, `--processes`), the catalog and chunk store, NZBs and a name
|
||||
search. Measured on the workstation (4 cores, 100 Mbit, Eweka):
|
||||
|
||||
- Edge worker, one core: ~190–215 k headers/s on recent a.b.boneless,
|
||||
~330 k on a 2013 a.b.ath range; ~0.66 GB peak per process. (Start:
|
||||
73–103 k/s, 1.0–1.2 GB.)
|
||||
- NNTP: ~22–30 k overview lines/s per connection in chunks of 100 k
|
||||
(Eweka takes ~1.5–2 s per command); connections don't slow each other.
|
||||
- End to end (coordinator with 10 connections, web server, Postgres and 3
|
||||
worker processes on one 4-core machine): 244 k headers/s; the server side
|
||||
used 10% (web) + 22% (coordinator) + 32% (Postgres) of a core.
|
||||
- Edge worker, one core: ~210–215 k headers/s on recent a.b.boneless,
|
||||
~330–340 k on a 2013 a.b.ath range; ~0.7–0.8 GB peak per process.
|
||||
(Start: 73–103 k/s, 1.0–1.2 GB.) What's left is spread out (the
|
||||
per-file key ~12%, GC ~10%, zlib ~10%, the scan ~6%).
|
||||
- NNTP: Eweka takes ~1.5–2 s per XOVER command; ~22–30 k lines/s per
|
||||
connection in chunks of 100 k when the line allows; connections don't
|
||||
slow each other.
|
||||
- Bandwidth: compressed XOVER is ~69 B per article on recent boneless
|
||||
(100 k-article chunks of ~7 MB). The workstation's 100 Mbit line carries
|
||||
~140–150 k articles/s, which is where its end-to-end runs now stop (10
|
||||
connections 90–99% busy, ~89 Mbit/s).
|
||||
- End to end at that rate (one 4-core machine): workers about half busy
|
||||
each; server side ~17% (coordinator) + ~8% (web) + ~3% (Postgres) of a
|
||||
core. (Before this round's fixes Postgres was at 32%: the merger's
|
||||
neighbour query read every range per file.)
|
||||
- Storage: chunks 17.9–20.3 B per segment (format v2); a release ~1.1 KB.
|
||||
- For the whole target (528 B overview lines): ~4–6 days at 50 connections,
|
||||
~3 worker cores.
|
||||
- For the whole target (528 B overview lines): ~36 TB of compressed XOVER
|
||||
to download. At 50 connections (~1–1.25 M lines/s, ~0.6–0.7 Gbit/s)
|
||||
~5–6 days, ~5–6 worker cores, two coordinator shards (one process
|
||||
handles ~60 MB/s), under a core for web and Postgres. The line is the
|
||||
limit below ~0.7 Gbit/s: at 1 Gbit, ~3.5 days at best.
|
||||
|
||||
Not built yet: re-running ranges for a retracted rule (the counts and rule
|
||||
versions are recorded), deployment (Dockerfile, compose), phase 2.
|
||||
|
||||
Reference in new issue
Block a user