voyager 3: the window in the Rust edge
Test and deploy / test (push) Successful in 47s
Test and deploy / edge (push) Successful in 1m47s
Test and deploy / deploy (push) Skipped

The indexer finishes files and releases by posting time instead of the
range path: the frontier is the median Date of the last 1,024 plausible
dates (within 7 days of it; a run of 1,024 agreeing dates moves it), and
a release is finished, judged and written once the frontier is 24 h past
its last part, flagged edge when posted within a day of the block's start.
At the block's end the rest is written provisionally as edges for the
block merger. Files short of parts stay in their release (partial: the
store writes complete = false). Gone: fragments and the cut-off,
uploads_of (online now), check.ts and the fixture comparison; CI replays
the fixtures through edge-bench instead. The wire carries finished, edge
and partial.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Fable 5.1 committed 2026-10-25 21:28:35 +01:00
1 parent 2580f20a43
commit f8cb17903b
13 files changed
+457 -413

No files matched your search

+7 -4
View File
@@ -31,8 +31,10 @@ export interface ReleaseChunk {
* later chunk of the same release key (the totals then are the release's; the payload only the files not sent yet).
*/
finished?: boolean
/** Still open at its block's end: the block merger joins it with the neighbour block's part. */
/** Still open at its block's end, or posted within the window of its start: the block merger joins it with the neighbour block's part. */
edge?: boolean
/** A file of the release misses parts (the window closed on it): not complete, whatever the counter says. */
partial?: boolean
payload: Buffer
}
@@ -95,7 +97,8 @@ export interface EdgeBatch {
samples: DroppedChunk[]
candidates: ProbeCandidate[]
counts: RuleCount[]
fragments: Fragment[]
/** The range path's cut-off files (the TypeScript indexer); the Rust edge's window has none, the store ignores them. */
fragments?: Fragment[]
}
export interface RangeIndexerOptions {
@@ -210,7 +213,7 @@ export class RangeIndexer {
/** Files completed and fragments cut since the last take: how big the next batch is. */
get pending(): number {
let files = this.batch.fragments.length
let files = this.batch.fragments!.length
for (const list of this.completed.values()) files += list.length
return files
}
@@ -294,7 +297,7 @@ export class RangeIndexer {
fragment.segments.push([s.number, s.messageId, s.bytes, s.articleNumber])
if (Number.isFinite(s.postedAt) && (fragment.postedAt === null || s.postedAt < fragment.postedAt)) fragment.postedAt = s.postedAt
}
this.batch.fragments.push(fragment)
this.batch.fragments!.push(fragment)
}
}