A cap on the catalog's NVMe size: above it only the tail and the stored ranges
Test and deploy / test (push) Successful in 54s
Test and deploy / edge (push) Successful in 2m3s
Test and deploy / deploy (push) Successful in 1m20s

After reset 24 the catalog grew ~3.7 GB an hour (77 GB, root 31 GB free, 600k
history ranges to go). CATALOG_MAX_NVME_GB (as voyager 1's
BACKFILL_MAX_NVME_GB): once the default tablespace is over it, leases and the
prefetcher take priority < 0 only (the tail, the stored ranges); measured once
a minute. Off when unset; 90 on homelab.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-14 16:10:11 +02:00
1 parent eddbe2863f
commit 256fb24c46
6 files changed
+74 -2

No files matched your search

+4 -1
View File
@@ -4,6 +4,7 @@ import coordinatorConfig from '#config/coordinator'
import NntpPool from '#services/nntp_pool'
import Spool, { type ChunkSpan } from '#services/spool'
import Corpus from '#services/corpus'
import CatalogSpace from '#services/catalog_space'
import { hedge, type Request, type XoverClient } from '#lib/nntp/index'
interface Target {
@@ -386,8 +387,10 @@ export class Prefetcher {
const { perRange } = coordinatorConfig.spool
const short = perRange > 0 ? Math.ceil(this.loops / perRange) + 8 - (leased.length + started.length) : 0
const want = Math.max(ahead - Number(ready), short)
// The catalog at its cap on the NVMe: no history fetched ahead (CatalogSpace).
const full = await CatalogSpace.full()
const fresh = room && want > 0
? await rows(db.query()).where('r.status', 'planned').where('r.spooled_chunks', 0).whereRaw('r.next_id <= r.end_id').orderBy([{ column: 'r.priority' }, { column: 'r.id' }]).limit(want)
? await rows(db.query()).where('r.status', 'planned').where('r.spooled_chunks', 0).whereRaw('r.next_id <= r.end_id').where((q: any) => (full ? q.where('r.priority', '<', 0) : q)).orderBy([{ column: 'r.priority' }, { column: 'r.id' }]).limit(want)
: []
const targets: Target[] = []
for (const row of [...leased, ...(room ? started : []), ...fresh]) {