daan/usenet-indexer
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
38 Commits 5 Branches 0 Tags
2dbca9b15240e116f0a1fbe1fdff1e1dd054ee3b
Commit Graph
5 Commits
Author SHA1 Message Date
Daan MeijerandClaude Opus 5.5 028dd90a6f Seed relay: dead NNTP connections found in 90 s and retried
Test and deploy / test (push) Successful in 1m14s
Details
Test and deploy / deploy (push) Successful in 1m46s
Details 
Five ranges "never answered" in 10 minutes, yet their chunk came back in
0.3 s on a fresh connection: the news server drops idle connections
without telling, and the pool's next request on one waited for the
10-minute inactivity timeout. Now: SEED_NNTP_TIMEOUT_MS 90 s (inactivity;
slow ranges keep streaming), one retry on a fresh connection, and pooled
connections idle over 60 s are replaced before use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 20:51:19 +02:00
Daan MeijerandClaude Opus 5.5 41859636d3 Seed relay: nodes get their headers through the seed API
Test and deploy / test (push) Successful in 1m28s
Details
Test and deploy / deploy (push) Successful in 2m5s
Details 
The seed API keeps a pool of NNTP connections (SEED_NNTP_CONNECTIONS) and
serves GET /api/v1/seed/ranges/:id/chunk?from=&to= for leased ranges:
the gzip stream the news server sent, passed on unopened. seed:node fetches
that way by default (SEED_FETCH=relay), so a node needs only a token and
SEED_CONNECTIONS; SEED_FETCH=direct keeps its own NNTP connections. All
fetching then comes from the indexer's IP (the account allows two).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 15:24:39 +02:00
Daan MeijerandClaude Opus 5.5 70036847cf Seeding nodes log in before leasing; TLS through tunnels
Test and deploy / test (push) Successful in 1m29s
Details
Test and deploy / deploy (push) Successful in 1m59s
Details 
- seed:node logs in to NNTP before taking a range and keeps the connection
  across ranges (replaced after 2 minutes idle or a failed range). A node
  that can't reach its news server takes no range: two ranges were set
  aside as failed while an office node's tunnel was being set up
  (ECONNREFUSED 127.0.0.1:1563, five leases each). After an IP-address
  limit refusal it waits 5 minutes.
- NNTP_TLS_SERVERNAME: the hostname the TLS certificate is checked
  against when NNTP_HOST is a tunnel ("Hostname/IP does not match
  certificate's altnames: Host: localhost").
- seed-node/README: the IP-address limit and tunnelling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 14:14:15 +02:00
Daan MeijerandClaude Opus 5.5 04f945055f XoverClient times replies only, not idle time between commands
Test and deploy / test (push) Successful in 1m12s
Details
Test and deploy / deploy (push) Successful in 1m37s
Details 
A seeding node that waited over two minutes for the seed API (restarting)
had its idle NNTP connection closed by its own timeout, and gave its range
back. Now the timeout runs only while a reply is awaited.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 13:40:15 +02:00
Daan MeijerandClaude Opus 5.5 97bdf8e982 Seeding node: gzip overviews, batched SFV lookups, fragments beside fetching
Test and deploy / test (push) Successful in 1m0s
Details
Test and deploy / deploy (push) Successful in 1m43s
Details 
- lib/nntp/xover_client: a minimal one-connection client for seed:range
  with XFEATURE COMPRESS GZIP (a zlib stream after "224 … [COMPRESS=GZIP]",
  then a "." line; 420 for an empty range). About 6x less header traffic.
  Overviews come out with the field names nntp-js uses.
- SfvService.claimAll: one sfv_entries lookup per stored batch.
- Fragments go through the regular ingest path beside fetching (at most
  two batches in flight); a failure fails the run at the end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 11:11:41 +02:00