Lease lanes for the tail, a mixed lease order, coordinator:status, the edge end to end
Test and deploy / test (push) Successful in 3m14s
Test and deploy / deploy (push) Successful in 2m30s

A group now has two lease lanes (migration 6600): `history` works the ranges the bounds hand out, `tail` the tail
planner's, each with its own holder, window, snapshot and takeover redo, so a thread on the tail never wades
through a group's history and a held group's tail isn't stuck behind it. The edge puts `--tail-threads`
(EDGE_TAIL_THREADS, 1 of 2+ by default) of its threads on the tail lane; the group routes take `?lane=`. The
lease order within a lane alternates the group with the most planned articles and the longest unheld one. The
takeover redo puts back only the lane's ranges the window did after the snapshot, and a group whose last range
is held by a holder gone quiet can be taken over.

`coordinator:status` prints the catalog's room, the ranges by band, the last hour's rates, the held groups with
their snapshots and the phase-2 backlogs. `tests/e2e` runs the real edge-node binary against the node API: the
corpus-fill race, a failed stream and the restart that restores the snapshot, a restart without the snapshot
file, a takeover (STREAM_CHUNK_WAIT_MINUTES is a setting now). CI builds the binary before the tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Fable 5.1 committed 2026-10-27 13:47:31 +01:00
1 parent db96642a72
commit 680751e26a
21 files changed
+1362 -180

No files matched your search

@@ -0,0 +1,27 @@
import { BaseSchema } from '@adonisjs/lucid/schema'
/**
* Lease lanes (REBUILD.md, "Voyager 3: window indexing", 1, amended 2026-10-27): a group has two cursors, each with
* its own holder, window and snapshot. The `history` lane works the ranges the lease bounds hand out (the windows,
* the history), the `tail` lane the tail planner's (priority at or below the tail's): a thread on the tail never
* waits behind a group's history, and a held history group's tail isn't stuck behind its days of ranges.
*/
export default class extends BaseSchema {
async up() {
this.schema.raw(
`ALTER TABLE group_leases ADD COLUMN lane varchar(8) NOT NULL DEFAULT 'history'`
)
this.schema.raw(`ALTER TABLE group_leases DROP CONSTRAINT group_leases_pkey`)
this.schema.raw(`ALTER TABLE group_leases ADD PRIMARY KEY (group_id, lane)`)
// Counts the leases handed out: odd and even ones start from the other half of the order's mix.
this.schema.raw(`CREATE SEQUENCE group_lease_turns`)
}
async down() {
this.schema.raw(`DROP SEQUENCE IF EXISTS group_lease_turns`)
this.schema.raw(`DELETE FROM group_leases WHERE lane <> 'history'`)
this.schema.raw(`ALTER TABLE group_leases DROP CONSTRAINT group_leases_pkey`)
this.schema.raw(`ALTER TABLE group_leases DROP COLUMN lane`)
this.schema.raw(`ALTER TABLE group_leases ADD PRIMARY KEY (group_id)`)
}
}