Commit Graph
2 Commits
Author SHA1 Message Date
Daan MeijerandClaude Opus 5.5 31493a98be Rebuild: merger rounds with one neighbour query, a cursor and a time budget; planning skips failing groups
Test and deploy / test (push) Successful in 22s
Test and deploy / deploy (push) Skipped
The neighbour check was three queries per file (~2,000 files/minute); now
one per round of 20k files, with a cursor so waiting files don't take
every round's places, and rounds until nothing is left or 30 s. On the
dev backlog (old-style fragments): ~18k files/minute in the 30 s budget;
at full throughput the new fragment rule leaves ~1,200 files/minute.
A group that fails GROUP (gone from the server) no longer stops planning
the others; edge:bench's groups are inactive.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 02:25:45 +02:00
Daan MeijerandClaude Opus 5.5 d5f168d8d2 Rebuild: coordinator (planner, NNTP pool, spool, prefetcher, leases, rule sets), node API and the edge:node worker
Test and deploy / test (push) Successful in 21s
Test and deploy / deploy (push) Skipped
- groups:add / ranges:plan / nodes:create / coordinator:run commands
- the node API: lease, the range's chunks as one framed stream from the
  spool, batches, heartbeat, finish, release (node guard, token per node)
- edge:node: needs COORDINATOR_URL and NODE_TOKEN only (placeholders for
  the rest), posts a batch every 10 chunks, gives its range back on SIGTERM
- releases are gathered per batch: 1.04 chunk rows per release at 10
  chunks per batch (1.35 when built per XOVER chunk)
- partitions are checked per batch instead of remembered

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 00:31:06 +02:00