A cap on the catalog's NVMe size: above it only the tail and the stored ranges
Test and deploy / test (push) Successful in 54s
Test and deploy / edge (push) Successful in 2m3s
Test and deploy / deploy (push) Successful in 1m20s

After reset 24 the catalog grew ~3.7 GB an hour (77 GB, root 31 GB free, 600k
history ranges to go). CATALOG_MAX_NVME_GB (as voyager 1's
BACKFILL_MAX_NVME_GB): once the default tablespace is over it, leases and the
prefetcher take priority < 0 only (the tail, the stored ranges); measured once
a minute. Off when unset; 90 on homelab.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-14 16:10:11 +02:00
1 parent eddbe2863f
commit 256fb24c46
6 files changed
+74 -2

No files matched your search

+4 -1
View File
@@ -3,6 +3,7 @@ import coordinatorConfig from '#config/coordinator'
import RuleSets from '#services/rule_sets'
import Spool from '#services/spool'
import Corpus from '#services/corpus'
import CatalogSpace from '#services/catalog_space'
import type { Lease } from '#lib/edge/index'
export class LeaseLost extends Error {}
@@ -19,12 +20,14 @@ export class LeaseLost extends Error {}
class Leases {
async lease(nodeId: number): Promise<Lease | null> {
const rules = await RuleSets.current()
// The catalog at its cap on the NVMe: the tail and the stored ranges only (CatalogSpace).
const history = (await CatalogSpace.full()) ? 'AND priority < 0' : ''
const take = (spooled: boolean) =>
db.rawQuery(
`UPDATE ranges r SET status = 'leased', node_id = ?, leased_at = now(), heartbeat_at = now(), attempts = attempts + 1, rule_version = ?
WHERE r.id = (
SELECT id FROM ranges
WHERE ${spooled ? 'spooled_chunks > 0' : 'true'} AND next_id <= end_id
WHERE ${spooled ? 'spooled_chunks > 0' : 'true'} AND next_id <= end_id ${history}
AND (status = 'planned' OR (status = 'leased' AND heartbeat_at < now() - make_interval(secs => ?)))
ORDER BY priority, id LIMIT 1 FOR UPDATE SKIP LOCKED)
RETURNING r.id, r.group_id, r.start_id, r.end_id, r.next_id`,