4 Commits

Include renames

Author	SHA1	Message	Date
Daan MeijerandClaude Opus 5.5	8442c4fc44	Rebuild: 2-minute NNTP reply timeout; the Rust worker runs on homelab ... Eweka stopped answering on all 40 of the coordinator's long-lived connections (quiet sockets; a fresh connection fetched the same chunk in 20 s): with a 10-minute timeout the prefetcher fetched one chunk a minute. The timeout (silence, not duration) is 2 minutes now: 365-411k articles/s after the restart. tools/probe_chunk.ts times a chunk compressed and plain. README and REBUILD.md: edge-node, the Rust worker, is the one running (8x less CPU per header, 20x less memory than edge:node in production). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-09 21:05:37 +02:00
Daan MeijerandClaude Opus 5.5	fd0f2bde75	Rebuild: the live tail ... coordinator:run plans the articles after each group's last range every TAIL_EVERY_SECONDS (120): one LIST ACTIVE per name prefix gives every group's newest article, and new articles become ranges at priority -1, leased and prefetched before any history. A group gets a tail range once it has TAIL_MIN_ARTICLES (100k, a chunk) new or its last planning is TAIL_MAX_DELAY_MINUTES (15) ago, so quiet groups don't make a range every round. History (planAll, hourly) is now only for groups not planned yet. ranges:plan --tail plans one round. Found on the way: with XFEATURE COMPRESS GZIP on, Eweka compresses LIST ACTIVE's reply too ("215 NewsGroups Follow [COMPRESS=GZIP]"), and listActive waited for a plain end that never came (groups:add --top too). It reads the compressed reply now; the fake server in the test does both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-09 10:41:16 +02:00
Daan MeijerandClaude Opus 5.5	5ba93f558c	Rebuild: XOVER chunks of 100k lines; batches by articles ... Eweka takes ~1.5–2 s per XOVER command whatever its size. One connection on recent a.b.boneless: 9–15k lines/s in chunks of 25k, ~20k in 50k, ~30k in 100k–200k. Four connections at once: 15k each at 50k, 22k each at 100k (the same as one connection alone at that moment): connections don't slow each other. CHUNK_ARTICLES defaults to 100k; edge:node posts a batch every --batch-articles (500k) instead of every N chunks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-09 02:09:04 +02:00
Daan MeijerandClaude Opus 5.5	d5f168d8d2	Rebuild: coordinator (planner, NNTP pool, spool, prefetcher, leases, rule sets), node API and the edge:node worker ... - groups:add / ranges:plan / nodes:create / coordinator:run commands - the node API: lease, the range's chunks as one framed stream from the spool, batches, heartbeat, finish, release (node guard, token per node) - edge:node: needs COORDINATOR_URL and NODE_TOKEN only (placeholders for the rest), posts a batch every 10 chunks, gives its range back on SIGTERM - releases are gathered per batch: 1.04 chunk rows per release at 10 chunks per batch (1.35 when built per XOVER chunk) - partitions are checked per batch instead of remembered Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-09 00:31:06 +02:00