voyager 3: the window in the Rust edge
Test and deploy / test (push) Successful in 47s
Test and deploy / edge (push) Successful in 1m47s
Test and deploy / deploy (push) Skipped

The indexer finishes files and releases by posting time instead of the
range path: the frontier is the median Date of the last 1,024 plausible
dates (within 7 days of it; a run of 1,024 agreeing dates moves it), and
a release is finished, judged and written once the frontier is 24 h past
its last part, flagged edge when posted within a day of the block's start.
At the block's end the rest is written provisionally as edges for the
block merger. Files short of parts stay in their release (partial: the
store writes complete = false). Gone: fragments and the cut-off,
uploads_of (online now), check.ts and the fixture comparison; CI replays
the fixtures through edge-bench instead. The wire carries finished, edge
and partial.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Fable 5.1 committed 2026-10-25 21:28:35 +01:00
1 parent 2580f20a43
commit f8cb17903b
13 files changed
+457 -413

No files matched your search

+12 -9
View File
@@ -6,18 +6,21 @@ from the node API, reads their prefetched XOVER chunks, finds releases and
posts batches. `edge-bench` (`src/main.rs`) replays a captured range through
the same indexing (`src/lib.rs`) for timing and for comparing.
The TypeScript `RangeIndexer` (`lib/edge`, `node ace edge:node`) is the
reference: `check.ts` runs both on a capture and compares every output field
by field. An identity or rule change goes into both and has to come out
identical. CI (`.gitea/workflows/deploy.yaml`, job `edge`) runs `cargo test`
and `check-fixtures.sh`, which compares both on the committed captures in
`fixtures/` (25k overview lines each of a.b.models, a.b.ath and
a.b.boneless, in 5 chunks); a push to `rebuild` deploys only when they
match.
Since voyager 3 (2026-10-25) the Rust indexer is the one implementation of
the window (REBUILD.md, "Voyager 3: window indexing"; `src/indexer.rs`):
files and releases are finished when the frontier (the median Date of the
last 1,024 plausibly dated articles) is 24 h past their last part, or at
the block's end (provisionally, as edges), and written once. The
TypeScript `RangeIndexer` (`lib/edge`) keeps the range path for the corpus
scanner and the tests; the field-by-field comparison (`check.ts`) went
with the parity. CI (`.gitea/workflows/deploy.yaml`, job `edge`) runs
`cargo test` and `bench-fixtures.sh`, which replays the committed captures
in `fixtures/` (25k overview lines each of a.b.ath, a.b.boneless, a.b.fta
and a.b.models, in 5 chunks).
```sh
cargo test --release --manifest-path edge/Cargo.toml
edge/check-fixtures.sh # from the repository root
edge/bench-fixtures.sh # from the repository root
```
In production since 2026-10-09: 122M headers in 10 minutes with 0.37 cores