Commit Graph
4 Commits
Author SHA1 Message Date
Daan MeijerandClaude Opus 5.5 b7ae8267c6 Seed relay: a stalled compressed XOVER is fetched again uncompressed
Test and deploy / test (push) Successful in 1m9s
Test and deploy / deploy (push) Successful in 1m41s
The four ranges kept failing on fresh connections too: in gzip mode Eweka
stops some chunks partway (a.b.ath 49701720000: 19,672 of 20,000
overviews, the last line cut, then silence), while the same chunk without
compression comes in 0.3 s. A failed relay chunk is now retried once
without compression on a connection of its own (dead pooled connections are
covered by the same retry).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 21:16:39 +02:00
Daan MeijerandClaude Opus 5.5 028dd90a6f Seed relay: dead NNTP connections found in 90 s and retried
Test and deploy / test (push) Successful in 1m14s
Test and deploy / deploy (push) Successful in 1m46s
Five ranges "never answered" in 10 minutes, yet their chunk came back in
0.3 s on a fresh connection: the news server drops idle connections
without telling, and the pool's next request on one waited for the
10-minute inactivity timeout. Now: SEED_NNTP_TIMEOUT_MS 90 s (inactivity;
slow ranges keep streaming), one retry on a fresh connection, and pooled
connections idle over 60 s are replaced before use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 20:51:19 +02:00
Daan MeijerandClaude Opus 5.5 530a752cca Seed: finished ranges marked done; relay XOVER timeout 10 minutes
Test and deploy / test (push) Successful in 1m37s
Test and deploy / deploy (push) Successful in 2m10s
- A range reported to its end but not finished (its node went away at
  the very end) was leased again with start past end; nodes asked for
  "from 49208000000 to 49207999999", got 400, and the range failed after
  five leases. The orchestrator now marks such ranges done before
  leasing, and runRange reports them done without fetching.
- Some old ranges come so slowly that one XOVER of 20k headers takes over
  two minutes: five ranges failed on "NNTP reply timed out". The relay
  pool's timeout is SEED_NNTP_TIMEOUT_MS (default 10 minutes, the node's
  HTTP timeout).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:21:30 +02:00
Daan MeijerandClaude Opus 5.5 41859636d3 Seed relay: nodes get their headers through the seed API
Test and deploy / test (push) Successful in 1m28s
Test and deploy / deploy (push) Successful in 2m5s
The seed API keeps a pool of NNTP connections (SEED_NNTP_CONNECTIONS) and
serves GET /api/v1/seed/ranges/:id/chunk?from=&to= for leased ranges:
the gzip stream the news server sent, passed on unopened. seed:node fetches
that way by default (SEED_FETCH=relay), so a node needs only a token and
SEED_CONNECTIONS; SEED_FETCH=direct keeps its own NNTP connections. All
fetching then comes from the indexer's IP (the account allows two).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 15:24:39 +02:00