Lease lanes for the tail, a mixed lease order, coordinator:status, the edge end to end
Test and deploy / test (push) Successful in 3m14s
Test and deploy / deploy (push) Successful in 2m30s

A group now has two lease lanes (migration 6600): `history` works the ranges the bounds hand out, `tail` the tail
planner's, each with its own holder, window, snapshot and takeover redo, so a thread on the tail never wades
through a group's history and a held group's tail isn't stuck behind it. The edge puts `--tail-threads`
(EDGE_TAIL_THREADS, 1 of 2+ by default) of its threads on the tail lane; the group routes take `?lane=`. The
lease order within a lane alternates the group with the most planned articles and the longest unheld one. The
takeover redo puts back only the lane's ranges the window did after the snapshot, and a group whose last range
is held by a holder gone quiet can be taken over.

`coordinator:status` prints the catalog's room, the ranges by band, the last hour's rates, the held groups with
their snapshots and the phase-2 backlogs. `tests/e2e` runs the real edge-node binary against the node API: the
corpus-fill race, a failed stream and the restart that restores the snapshot, a restart without the snapshot
file, a takeover (STREAM_CHUNK_WAIT_MINUTES is a setting now). CI builds the binary before the tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Fable 5.1 committed 2026-10-27 13:47:31 +01:00
1 parent db96642a72
commit 680751e26a
21 files changed
+1362 -180

No files matched your search

+1 -1
View File
@@ -1,7 +1,7 @@
export { RangeIndexer, postedSpan, chunkOf, candidatesOf, releaseKeyOf, uploadsOf, UPLOAD_GAP_MS, type EdgeBatch, type ReleaseChunk, type DroppedChunk, type ProbeCandidate, type CandidateKind, type RuleCount, type Fragment, type RangeIndexerOptions } from './range_indexer.js'
export { encodeFrame, decodeFrames, isKeepalive, KEEPALIVE, type Frame } from './stream.js'
export { toWire, fromWire, type WireBatch } from './wire.js'
export type { Lease, GroupLease, NextRange } from './lease.js'
export type { Lease, GroupLease, Lane, NextRange } from './lease.js'
export type { EdgeSegment, FileMeta } from './scan.js'
export { throttle } from './throttle.js'
export { filterOverviews, undelta, type ChunkDrops } from './corpus_filter.js'
+11 -2
View File
@@ -7,11 +7,20 @@ export interface Lease {
}
/**
* A group lease (REBUILD.md, "Voyager 3: window indexing", 1): the group a worker thread holds, the rules, and the
* snapshot of the group's window when there is one (its next article; the body comes from GET groups/:id/snapshot).
* A group's two cursors (REBUILD.md, "Voyager 3: window indexing", 1): `history` works the ranges the lease bounds
* hand out (the windows, the history), `tail` the tail planner's ranges. Each lane has its own holder, window and
* snapshot; the group routes take `?lane=` (history by default).
*/
export type Lane = 'history' | 'tail'
/**
* A group lease (REBUILD.md, "Voyager 3: window indexing", 1): the group a worker thread holds, the lane, the rules,
* and the snapshot of the lane's window when there is one (its next article; the body comes from GET
* groups/:id/snapshot).
*/
export interface GroupLease {
group: { id: number; name: string; priority: number }
lane: Lane
rules: Rules
snapshot: { next: number } | null
}