Seed relay: nodes get their headers through the seed API
Test and deploy / test (push) Successful in 1m28s
Test and deploy / deploy (push) Successful in 2m5s

The seed API keeps a pool of NNTP connections (SEED_NNTP_CONNECTIONS) and
serves GET /api/v1/seed/ranges/:id/chunk?from=&to= for leased ranges:
the gzip stream the news server sent, passed on unopened. seed:node fetches
that way by default (SEED_FETCH=relay), so a node needs only a token and
SEED_CONNECTIONS; SEED_FETCH=direct keeps its own NNTP connections. All
fetching then comes from the indexer's IP (the account allows two).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-05 15:24:39 +02:00
1 parent 3d5cfb2527
commit 41859636d3
13 files changed
+289 -33

No files matched your search

+14
View File
@@ -3,6 +3,7 @@ import db from '@adonisjs/lucid/services/db'
import NntpService from '#services/NntpService'
import SeedService from '#services/SeedService'
import BackfillService from '#services/BackfillService'
import SeedFetchPool from '#services/SeedFetchPool'
import { articleAtDate } from '#lib/nntp/date_search'
import { normalizeSeedFile, type SeedBatch } from '#lib/seed/wire'
@@ -92,6 +93,19 @@ class SeedOrchestrator {
return stored
}
/**
* Relay: overviews of `from`..`to` of a leased range, fetched here and handed
* on as the gzip stream the news server sent. Keeps the lease alive.
*/
async chunk(nodeId: number, rangeId: number, from: number, to: number): Promise<Buffer> {
const range = await this.leased(nodeId, rangeId)
const start = Math.max(from, Number(range.start_id))
const end = Math.min(to, Number(range.end_id), start + 49_999)
if (end < start) throw new RangeError(`Articles ${from}–${to} are outside range ${rangeId}`)
await db.from('seed_ranges').where('id', rangeId).update({ heartbeat_at: new Date() })
return SeedFetchPool.fetch(range.group_name, start, end)
}
async finish(nodeId: number, rangeId: number) {
await this.leased(nodeId, rangeId)
await db.from('seed_ranges').where('id', rangeId).update({ status: 'done', done_at: new Date(), heartbeat_at: new Date(), error: null })