Commit Graph
11 Commits
Author SHA1 Message Date
Daan MeijerandClaude Opus 5.5 53bfea8d15 Rebuild spec: measurements after the second performance round
Test and deploy / test (push) Successful in 23s
Test and deploy / deploy (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 03:37:04 +02:00
Daan MeijerandClaude Opus 5.5 69f1cc63b2 Rebuild: coordinator:run --shard=i/n splits prefetching over processes
The coordinator's event loop was 16.5% busy at ~10 MB/s of compressed
XOVER (socket reads, buffers, GC, spool writes; nothing stands out), so
one process tops out around 60 MB/s: less than 50 connections bring
(~1M articles/s, ~70-86 MB/s). A shard prefetches the ranges with
id % n = i - 1 with its own NNTP_CONNECTIONS; only shard 1 plans, merges
and settles. Two shards of 5 connections fetched what one of 10 did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 03:36:24 +02:00
Daan MeijerandClaude Opus 5.5 583ea1ded6 Rebuild spec: status and measurements so far
Test and deploy / test (push) Successful in 19s
Test and deploy / deploy (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 02:53:36 +02:00
Daan MeijerandClaude Opus 5.5 20910647fa Rebuild: the batch store (catalog upserts, write-once chunks, candidates, rule counts, fragments, resume point)
Test and deploy / test (push) Successful in 20s
Test and deploy / deploy (push) Skipped
A batch is stored in one transaction and only at its range's resume
point, so a retried batch can't count twice. Garbage is judged before
per-article: a dropped tool isn't worth reading bodies for either.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 00:21:17 +02:00
Daan MeijerandClaude Opus 5.5 a18ead666b Rebuild: edge:bench, and its result in the spec (TypeScript is fast enough)
Test and deploy / test (push) Successful in 19s
Test and deploy / deploy (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 00:17:26 +02:00
Daan MeijerandClaude Opus 5.5 699395c51f Rebuild spec: a worker streams its whole range in one request
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 00:01:03 +02:00
Daan MeijerandClaude Opus 5.5 06f5d81fbb Rebuild spec: d and e decided; coordinator prefetches ranges into a spool
Test and deploy / test (push) Successful in 1m11s
Test and deploy / deploy (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 23:57:17 +02:00
Daan MeijerandClaude Opus 5.5 76ecef81af Rebuild spec: a, b, c decided; group list with priority-based crosspost owners
Test and deploy / test (push) Successful in 1m9s
Test and deploy / deploy (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 23:54:20 +02:00
Daan MeijerandClaude Opus 5.5 1931c7f66c Rebuild spec: rules and recovery (counts per range and rule, rule versions, a sample)
Test and deploy / test (push) Successful in 1m3s
Test and deploy / deploy (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 23:39:36 +02:00
Daan MeijerandClaude Opus 5.5 2d4f3f48cf Rebuild spec: workers return probe candidates per release; a probe queue
Test and deploy / test (push) Successful in 1m4s
Test and deploy / deploy (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 23:31:18 +02:00
Daan MeijerandClaude Opus 5.5 16e504a8fb Rebuild: spec draft (header-first indexer)
Test and deploy / test (push) Successful in 1m21s
Test and deploy / deploy (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 19:37:26 +02:00