Rebuild spec: status and measurements so far
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
821de668d0
commit
583ea1ded6
1 file changed
+24
+24
@@ -5,6 +5,30 @@ Everything learned from articles (names from yEnc headers, par2, archive
|
||||
listings, encryption, spots) is phase 2 and adds to what phase 1 stored; it
|
||||
never rewrites it. Sources: CONCEPT.md, LOG.md 2026-10-08 (13)–(14).
|
||||
|
||||
## Status (2026-10-09)
|
||||
|
||||
Built on this branch (a fresh AdonisJS 7 app; phase 1 code imported from
|
||||
the current app by pipeline stage): coordinator (planner, NNTP pool,
|
||||
prefetch spool, leases, rule sets, fragment merger, settling), the node API
|
||||
(lease, framed stream, batches, finish/release/heartbeat), the edge worker
|
||||
(`edge:node`, `--processes`), the catalog and chunk store, NZBs and a name
|
||||
search. Measured on the workstation (4 cores, 100 Mbit, Eweka):
|
||||
|
||||
- Edge worker, one core: ~190–215 k headers/s on recent a.b.boneless,
|
||||
~330 k on a 2013 a.b.ath range; ~0.66 GB peak per process. (Start:
|
||||
73–103 k/s, 1.0–1.2 GB.)
|
||||
- NNTP: ~22–30 k overview lines/s per connection in chunks of 100 k
|
||||
(Eweka takes ~1.5–2 s per command); connections don't slow each other.
|
||||
- End to end (coordinator with 10 connections, web server, Postgres and 3
|
||||
worker processes on one 4-core machine): 244 k headers/s; the server side
|
||||
used 10% (web) + 22% (coordinator) + 32% (Postgres) of a core.
|
||||
- Storage: chunks 17.9–20.3 B per segment (format v2); a release ~1.1 KB.
|
||||
- For the whole target (528 B overview lines): ~4–6 days at 50 connections,
|
||||
~3 worker cores.
|
||||
|
||||
Not built yet: re-running ranges for a retracted rule (the counts and rule
|
||||
versions are recorded), deployment (Dockerfile, compose), phase 2.
|
||||
|
||||
## Target
|
||||
|
||||
7,000 days (all of Eweka's retention, back to 2008-08) of the 1000 largest
|
||||
|
||||
Reference in new issue
Block a user