The tail planner grows a group's open range; ranges:merge-tail packs the slivers
A round added a row a group whenever 100k articles or 15 minutes had passed, tuned for a tail leased within minutes; with nothing leasing the tail the rows piled up (127k planned tail ranges of a median 14k articles, 97% adjacent). Now a round grows the group's last range when it is a planned, unspooled, untaken row at the tail's priority, up to RANGE_ARTICLES, and only then adds rows; a range taken meanwhile gets the slice as a row. ranges:merge-tail (--dry-run) packs adjacent planned, uncaptured, unspooled, untaken tail ranges of a group and priority into ranges of RANGE_ARTICLES under the lease lock. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
e3957e5af5
commit
1810abd7d8
7 files changed
+423
-29
No files matched your search
@@ -117,8 +117,11 @@ export default class CoordinatorRun extends BaseCommand {
|
||||
}
|
||||
if (main && Date.now() - tailedAt >= coordinatorConfig.tail.everySeconds * 1000) {
|
||||
tailedAt = Date.now()
|
||||
const { groups, ranges } = await Planner.planTail()
|
||||
if (ranges > 0) this.logger.info(`Tail: ${ranges} ranges for ${groups} groups.`)
|
||||
const { groups, ranges, extended } = await Planner.planTail()
|
||||
if (ranges + extended > 0)
|
||||
this.logger.info(
|
||||
`Tail: ${ranges} ranges added, ${extended} grown, for ${groups} groups.`
|
||||
)
|
||||
}
|
||||
} catch (error: any) {
|
||||
this.logger.error(`Coordinator round failed: ${error.message}`)
|
||||
|
||||
Reference in new issue
Block a user