Rebuild spec: status and measurements so far
Test and deploy / test (push) Successful in 19s
Test and deploy / deploy (push) Skipped

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-09 02:53:36 +02:00
1 parent 821de668d0
commit 583ea1ded6
1 file changed
+24
+24
View File
@@ -5,6 +5,30 @@ Everything learned from articles (names from yEnc headers, par2, archive
listings, encryption, spots) is phase 2 and adds to what phase 1 stored; it
never rewrites it. Sources: CONCEPT.md, LOG.md 2026-10-08 (13)–(14).
## Status (2026-10-09)
Built on this branch (a fresh AdonisJS 7 app; phase 1 code imported from
the current app by pipeline stage): coordinator (planner, NNTP pool,
prefetch spool, leases, rule sets, fragment merger, settling), the node API
(lease, framed stream, batches, finish/release/heartbeat), the edge worker
(`edge:node`, `--processes`), the catalog and chunk store, NZBs and a name
search. Measured on the workstation (4 cores, 100 Mbit, Eweka):
- Edge worker, one core: ~190–215 k headers/s on recent a.b.boneless,
~330 k on a 2013 a.b.ath range; ~0.66 GB peak per process. (Start:
73–103 k/s, 1.0–1.2 GB.)
- NNTP: ~22–30 k overview lines/s per connection in chunks of 100 k
(Eweka takes ~1.5–2 s per command); connections don't slow each other.
- End to end (coordinator with 10 connections, web server, Postgres and 3
worker processes on one 4-core machine): 244 k headers/s; the server side
used 10% (web) + 22% (coordinator) + 32% (Postgres) of a core.
- Storage: chunks 17.9–20.3 B per segment (format v2); a release ~1.1 KB.
- For the whole target (528 B overview lines): ~4–6 days at 50 connections,
~3 worker cores.
Not built yet: re-running ranges for a retracted rule (the counts and rule
versions are recorded), deployment (Dockerfile, compose), phase 2.
## Target
7,000 days (all of Eweka's retention, back to 2008-08) of the 1000 largest