Commit Graph
2 Commits
Author SHA1 Message Date
Daan MeijerandClaude Opus 5.5 530a752cca Seed: finished ranges marked done; relay XOVER timeout 10 minutes
Test and deploy / test (push) Successful in 1m37s
Test and deploy / deploy (push) Successful in 2m10s
- A range reported to its end but not finished (its node went away at
  the very end) was leased again with start past end; nodes asked for
  "from 49208000000 to 49207999999", got 400, and the range failed after
  five leases. The orchestrator now marks such ranges done before
  leasing, and runRange reports them done without fetching.
- Some old ranges come so slowly that one XOVER of 20k headers takes over
  two minutes: five ranges failed on "NNTP reply timed out". The relay
  pool's timeout is SEED_NNTP_TIMEOUT_MS (default 10 minutes, the node's
  HTTP timeout).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:21:30 +02:00
Daan MeijerandClaude Opus 5.5 68eb9a44fd Edge aggregation: seed:range indexes history one row per file
Test and deploy / test (push) Successful in 1m36s
Test and deploy / deploy (push) Successful in 3m37s
A seeding node fetches a range over one NNTP connection, parses headers
with the live rules, groups them into files in memory (lib/aggregate) and
stores per complete file its release, the file and a completed binary
(SeedService), instead of one parts row per article. What doesn't complete
within the window goes through the regular ingest path as fragments.
`--dry-run` only fetches and aggregates, to measure a node.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 10:41:18 +02:00