Posting-time windows into the corpus: ranges:dates, ranges:capture
Test and deploy / test (push) Successful in 55s
Test and deploy / edge (push) Successful in 2m2s
Test and deploy / deploy (push) Successful in 2m12s

ranges:dates samples each active group's posting dates from the news
server (group_dates) and estimates every range's date (ranges.est_date).
ranges:capture --from --to marks a window's ranges of every group for the
corpus. The corpus fill takes capture ranges in priority order, done ones
too, and remembers complete ones (ranges.stored_at).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Fable 5.1 committed 2026-10-25 09:47:28 +01:00
1 parent 0469fcd975
commit 137e931e9e
8 files changed
+683 -40

No files matched your search

@@ -0,0 +1,27 @@
import { BaseSchema } from '@adonisjs/lucid/schema'
/**
* When a group's articles were posted, sampled from the news server (ranges:dates; app/services/group_dates.ts), and
* from those each range's estimated posting date, so ranges can be chosen by era: ranges:capture --from --to marks a
* posting-time window of every group for the corpus (voyager 3's test data; REBUILD.md, corpus). `stored_at`: the
* range is complete in the corpus (the prefetcher saw its range.json), so the corpus fill skips it without looking.
*/
export default class extends BaseSchema {
async up() {
this.schema.raw(`
CREATE TABLE IF NOT EXISTS group_dates (
group_id integer NOT NULL REFERENCES groups (id) ON DELETE CASCADE,
article bigint NOT NULL,
posted_at timestamptz NOT NULL,
PRIMARY KEY (group_id, article)
)`)
this.schema.raw(`ALTER TABLE ranges ADD COLUMN IF NOT EXISTS est_date timestamptz`)
this.schema.raw(`ALTER TABLE ranges ADD COLUMN IF NOT EXISTS stored_at timestamptz`)
}
async down() {
this.schema.raw(`ALTER TABLE ranges DROP COLUMN IF EXISTS stored_at`)
this.schema.raw(`ALTER TABLE ranges DROP COLUMN IF EXISTS est_date`)
this.schema.raw(`DROP TABLE IF EXISTS group_dates`)
}
}