4 Commits

Include renames

Author	SHA1	Message	Date
Daan MeijerandClaude Opus 5.5	0ccb850ea4	Prober: claim ahead of the lanes, not when they run dry ... The next claim started at half a batch of four pipeline rounds a lane, 3-6 s of work, while a claim takes 1.5-4 s and more; at 20-24 lanes the lanes waited for candidates up to 19% of their time. queueLevels: the claim starts below eight rounds of every lane's pipeline (~13 s) and fills the queue to twice that. Taken candidates go back to waiting at start-up as before, so a bigger queue costs nothing on a restart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-28 14:52:48 +01:00
Daan MeijerandClaude Fable 5.1	68b480c97e	An "ancient" range a group: articles 1 … low watermark − 1, never filled ... Every group is tiled from article 1 now: one range with status 'ancient' covers what had expired on the server before the group was planned. Nothing selects it (leases, prefetcher, corpus fill, tail merger all take 'planned'), the status panel reports it apart, the search's history stats leave it out. Migration 6700 adds it for the groups planned before; Planner.historyRows (now a pure function, unit-tested) for new groups. The status bands are ordered by an explicit key: the ancient row at priority 0 tied "history 0-999" with "window 0". Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>	2026-10-28 09:12:05 +01:00
Daan MeijerandClaude Fable 5.1	1810abd7d8	The tail planner grows a group's open range; ranges:merge-tail packs the slivers ... A round added a row a group whenever 100k articles or 15 minutes had passed, tuned for a tail leased within minutes; with nothing leasing the tail the rows piled up (127k planned tail ranges of a median 14k articles, 97% adjacent). Now a round grows the group's last range when it is a planned, unspooled, untaken row at the tail's priority, up to RANGE_ARTICLES, and only then adds rows; a range taken meanwhile gets the slice as a row. ranges:merge-tail (--dry-run) packs adjacent planned, uncaptured, unspooled, untaken tail ranges of a group and priority into ranges of RANGE_ARTICLES under the lease lock. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>	2026-10-27 10:46:47 +01:00
Daan MeijerandClaude Opus 5.5	fd0f2bde75	Rebuild: the live tail ... coordinator:run plans the articles after each group's last range every TAIL_EVERY_SECONDS (120): one LIST ACTIVE per name prefix gives every group's newest article, and new articles become ranges at priority -1, leased and prefetched before any history. A group gets a tail range once it has TAIL_MIN_ARTICLES (100k, a chunk) new or its last planning is TAIL_MAX_DELAY_MINUTES (15) ago, so quiet groups don't make a range every round. History (planAll, hourly) is now only for groups not planned yet. ranges:plan --tail plans one round. Found on the way: with XFEATURE COMPRESS GZIP on, Eweka compresses LIST ACTIVE's reply too ("215 NewsGroups Follow [COMPRESS=GZIP]"), and listActive waited for a plain end that never came (groups:add --top too). It reads the compressed reply now; the fake server in the test does both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-09 10:41:16 +02:00