Posting-time windows into the corpus: ranges:dates, ranges:capture
ranges:dates samples each active group's posting dates from the news server (group_dates) and estimates every range's date (ranges.est_date). ranges:capture --from --to marks a window's ranges of every group for the corpus. The corpus fill takes capture ranges in priority order, done ones too, and remembers complete ones (ranges.stored_at). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
0469fcd975
commit
137e931e9e
8 files changed
+683
-40
No files matched your search
+14
@@ -244,6 +244,20 @@ The worker makes the same releases of a filtered chunk; rule_counts lack
|
||||
those two rules' counts. `corpus:filter` filters chunks stored raw. A
|
||||
corpus filtered this way no longer serves a sweep of those two rules.
|
||||
|
||||
Windows by posting time (2026-10-25; voyager 3's test data, LOG.md (104)):
|
||||
`ranges:dates` samples when each active group's articles were posted (one
|
||||
XOVER of 32 articles every 500M articles of the group, 16 … 1000 samples,
|
||||
the median Date of each slice, kept monotone; `group_dates`) and estimates
|
||||
every range's posting date from the curve (`ranges.est_date`; the edges of
|
||||
a window are off by up to one sample interval, three weeks in boneless).
|
||||
`ranges:capture --from 2025-01-01 --to 2026-01-01` marks every active
|
||||
group's ranges posted in the window `capture`: planned ones not started at
|
||||
`--priority` (0), done ones as they are, both fetched into the corpus by the
|
||||
fill, which takes capture ranges in priority order, done ones too (they're
|
||||
fetched again, corpus only), and skips those complete (`ranges.stored_at`,
|
||||
set when it sees the range.json). A year of every group is the data to
|
||||
replay voyager 3's 30-day blocks on.
|
||||
|
||||
## Phase 2
|
||||
|
||||
Built: the prober (2026-10-11; `probe:run`, LOG.md 2026-10-11 (2)). It
|
||||
|
||||
Reference in new issue
Block a user