Rebuild spec: status and measurements so far
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
821de668d0
commit
583ea1ded6
1 file changed
+24
+24
@@ -5,6 +5,30 @@ Everything learned from articles (names from yEnc headers, par2, archive
|
|||||||
listings, encryption, spots) is phase 2 and adds to what phase 1 stored; it
|
listings, encryption, spots) is phase 2 and adds to what phase 1 stored; it
|
||||||
never rewrites it. Sources: CONCEPT.md, LOG.md 2026-10-08 (13)–(14).
|
never rewrites it. Sources: CONCEPT.md, LOG.md 2026-10-08 (13)–(14).
|
||||||
|
|
||||||
|
## Status (2026-10-09)
|
||||||
|
|
||||||
|
Built on this branch (a fresh AdonisJS 7 app; phase 1 code imported from
|
||||||
|
the current app by pipeline stage): coordinator (planner, NNTP pool,
|
||||||
|
prefetch spool, leases, rule sets, fragment merger, settling), the node API
|
||||||
|
(lease, framed stream, batches, finish/release/heartbeat), the edge worker
|
||||||
|
(`edge:node`, `--processes`), the catalog and chunk store, NZBs and a name
|
||||||
|
search. Measured on the workstation (4 cores, 100 Mbit, Eweka):
|
||||||
|
|
||||||
|
- Edge worker, one core: ~190–215 k headers/s on recent a.b.boneless,
|
||||||
|
~330 k on a 2013 a.b.ath range; ~0.66 GB peak per process. (Start:
|
||||||
|
73–103 k/s, 1.0–1.2 GB.)
|
||||||
|
- NNTP: ~22–30 k overview lines/s per connection in chunks of 100 k
|
||||||
|
(Eweka takes ~1.5–2 s per command); connections don't slow each other.
|
||||||
|
- End to end (coordinator with 10 connections, web server, Postgres and 3
|
||||||
|
worker processes on one 4-core machine): 244 k headers/s; the server side
|
||||||
|
used 10% (web) + 22% (coordinator) + 32% (Postgres) of a core.
|
||||||
|
- Storage: chunks 17.9–20.3 B per segment (format v2); a release ~1.1 KB.
|
||||||
|
- For the whole target (528 B overview lines): ~4–6 days at 50 connections,
|
||||||
|
~3 worker cores.
|
||||||
|
|
||||||
|
Not built yet: re-running ranges for a retracted rule (the counts and rule
|
||||||
|
versions are recorded), deployment (Dockerfile, compose), phase 2.
|
||||||
|
|
||||||
## Target
|
## Target
|
||||||
|
|
||||||
7,000 days (all of Eweka's retention, back to 2008-08) of the 1000 largest
|
7,000 days (all of Eweka's retention, back to 2008-08) of the 1000 largest
|
||||||
|
|||||||
Reference in new issue
Block a user