Posting-time windows into the corpus: ranges:dates, ranges:capture
Test and deploy / test (push) Successful in 55s
Test and deploy / edge (push) Successful in 2m2s
Test and deploy / deploy (push) Successful in 2m12s

ranges:dates samples each active group's posting dates from the news
server (group_dates) and estimates every range's date (ranges.est_date).
ranges:capture --from --to marks a window's ranges of every group for the
corpus. The corpus fill takes capture ranges in priority order, done ones
too, and remembers complete ones (ranges.stored_at).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Fable 5.1 committed 2026-10-25 09:47:28 +01:00
1 parent 0469fcd975
commit 137e931e9e
8 files changed
+683 -40

No files matched your search

+14
View File
@@ -244,6 +244,20 @@ The worker makes the same releases of a filtered chunk; rule_counts lack
those two rules' counts. `corpus:filter` filters chunks stored raw. A
corpus filtered this way no longer serves a sweep of those two rules.
Windows by posting time (2026-10-25; voyager 3's test data, LOG.md (104)):
`ranges:dates` samples when each active group's articles were posted (one
XOVER of 32 articles every 500M articles of the group, 16 … 1000 samples,
the median Date of each slice, kept monotone; `group_dates`) and estimates
every range's posting date from the curve (`ranges.est_date`; the edges of
a window are off by up to one sample interval, three weeks in boneless).
`ranges:capture --from 2025-01-01 --to 2026-01-01` marks every active
group's ranges posted in the window `capture`: planned ones not started at
`--priority` (0), done ones as they are, both fetched into the corpus by the
fill, which takes capture ranges in priority order, done ones too (they're
fetched again, corpus only), and skips those complete (`ranges.stored_at`,
set when it sees the range.json). A year of every group is the data to
replay voyager 3's 30-day blocks on.
## Phase 2
Built: the prober (2026-10-11; `probe:run`, LOG.md 2026-10-11 (2)). It