Seed orchestrator: remote nodes lease ranges and post results over HTTPS
Test and deploy / test (push) Successful in 1m4s
Test and deploy / deploy (push) Successful in 1m37s

- seed_nodes (hashed bearer tokens) and seed_ranges (planned ranges with
  lease, resume point and counts); migration …140.
- SeedOrchestrator: lease (oldest plan first, FOR UPDATE SKIP LOCKED; a
  lease without a batch for 15 min passes to the next node and resumes at
  next_id), batch (stores files and fragments, moves next_id), finish,
  release, plan (from below live indexing / the backfill / earlier plans
  down to N days back; takes over the group's backfill), status.
- Seed API under /api/v1/seed (lease, ranges/:id/batches|done|release,
  status) behind a node-token middleware; JSON bodies up to 64 MB after
  gzip.
- seed:node runs a node: no database or Redis, outgoing HTTPS only, N
  connections, gzip batches, retries for network errors and 5xx, drops a
  range on 409, gives its ranges back on SIGTERM. seed-node/ has a compose
  file that builds the image from this repository.
- seed:token, seed:plan, seed:status for the operator. Header parsing moved
  to lib/ingest and the range loop to lib/seed/runner, shared by seed:range
  and seed:node.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-05 12:02:01 +02:00
1 parent 97bdf8e982
commit 8c36744088
24 files changed
+1198 -246

No files matched your search

+2 -1
View File
@@ -5,6 +5,7 @@ import { segmentFromHeader, type IngestSegment } from '#services/IngestService'
import SeedService from '#services/SeedService'
import QueueService from '#services/QueueService'
import { FileAggregator } from '#lib/aggregate/index'
import { toSeedFile } from '#lib/seed/wire'
import { segmentsOf } from '#services/FileStore'
function header(subject: string, n: number) {
@@ -25,7 +26,7 @@ test.group('SeedService', (group) => {
const aggregator = new FileAggregator<IngestSegment>()
const subjects = [1, 2].map((i) => `Seed.Test.S01E01.1080p - [1/2] - "Seed.Test.S01E01.1080p.part${i}.mkv" yEnc (%d/3)`)
const segments = subjects.flatMap((s) => [1, 2, 3].map((n) => segmentFromHeader(header(s.replace('%d', String(n)), n), 'alt.binaries.test', n)!))
const files = aggregator.add(segments)
const files = aggregator.add(segments).map(toSeedFile)
assert.lengthOf(files, 2)
const result = await SeedService.store(files)