Commit Graph
2 Commits
Author SHA1 Message Date
Daan MeijerandClaude Opus 5.5 97bdf8e982 Seeding node: gzip overviews, batched SFV lookups, fragments beside fetching
Test and deploy / test (push) Successful in 1m0s
Test and deploy / deploy (push) Successful in 1m43s
- lib/nntp/xover_client: a minimal one-connection client for seed:range
  with XFEATURE COMPRESS GZIP (a zlib stream after "224 … [COMPRESS=GZIP]",
  then a "." line; 420 for an empty range). About 6x less header traffic.
  Overviews come out with the field names nntp-js uses.
- SfvService.claimAll: one sfv_entries lookup per stored batch.
- Fragments go through the regular ingest path beside fetching (at most
  two batches in flight); a failure fails the run at the end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 11:11:41 +02:00
Daan MeijerandClaude Opus 5.5 68eb9a44fd Edge aggregation: seed:range indexes history one row per file
Test and deploy / test (push) Successful in 1m36s
Test and deploy / deploy (push) Successful in 3m37s
A seeding node fetches a range over one NNTP connection, parses headers
with the live rules, groups them into files in memory (lib/aggregate) and
stores per complete file its release, the file and a completed binary
(SeedService), instead of one parts row per article. What doesn't complete
within the window goes through the regular ingest path as fragments.
`--dry-run` only fetches and aggregates, to measure a node.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 10:41:18 +02:00