Rebuild: settling quiet releases
Hourly in coordinator:run: a release unchanged for 6 hours is settled, complete when every file its [i/N] counter announced was found, unknown without a counter (phase 2 reads par2 sets for those). Partial index on unsettled releases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
fde1e6cc90
commit
80c920f678
4 files changed
+64
-2
No files matched your search
@@ -4,6 +4,7 @@ import { setTimeout as sleep } from 'node:timers/promises'
|
||||
import { Prefetcher } from '#services/prefetcher'
|
||||
import Planner from '#services/planner'
|
||||
import FragmentMerger from '#services/fragment_merger'
|
||||
import Settler from '#services/settler'
|
||||
import NntpPool from '#services/nntp_pool'
|
||||
|
||||
const TICK_MS = 2000
|
||||
@@ -13,8 +14,8 @@ const MERGE_BUDGET_MS = 30 * 1000
|
||||
|
||||
/**
|
||||
* The coordinator's background work (REBUILD.md, part 1): the prefetcher,
|
||||
* merging fragments of finished ranges every minute, and planning ranges
|
||||
* for new articles every hour. The API (leases, stream,
|
||||
* merging fragments of finished ranges every minute, and every hour
|
||||
* planning ranges for new articles and settling quiet releases. The API (leases, stream,
|
||||
* batches) runs in the web server.
|
||||
*/
|
||||
export default class CoordinatorRun extends BaseCommand {
|
||||
@@ -42,6 +43,8 @@ export default class CoordinatorRun extends BaseCommand {
|
||||
plannedAt = Date.now()
|
||||
const { groups, ranges } = await Planner.planAll()
|
||||
if (ranges > 0) this.logger.info(`Planned ${ranges} ranges over ${groups} groups.`)
|
||||
const settled = await Settler.run()
|
||||
if (settled > 0) this.logger.info(`Settled ${settled} releases.`)
|
||||
}
|
||||
if (Date.now() - mergedAt >= MERGE_EVERY_MS) {
|
||||
mergedAt = Date.now()
|
||||
|
||||
Reference in new issue
Block a user