NNTP shares: the account's connections shared by priority
The coordinator (38) and the prober (8) had a fixed split; with history paused the coordinator's stood idle while the prober's were its limit. nntp_shares: each process syncs every 15 s (priority, minimum, what it could use, what it has open) and gets what it may open: minimums first, then by priority what each wants, growing only into connections the others closed or gave up. The pool's size and the prober's lanes follow the grant; the coordinator doubles its want while work waits for a connection, else asks for its peak in use and a margin; the prober wants its maximum unless the queue is dry. Claims scale with the lanes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
7eba53043a
commit
39480c486f
10 files changed
+373
-75
No files matched your search
@@ -131,6 +131,8 @@ export class Prefetcher {
|
||||
.catch((error) => logger.warn(`Prefetching: ${error.message}`))
|
||||
.finally(() => this.running--)
|
||||
}
|
||||
// Work left with every loop running: the pool's share could be bigger.
|
||||
if (!this.stopped && this.running >= this.loops && this.peek()) NntpPool.starved()
|
||||
}
|
||||
|
||||
stop() {
|
||||
@@ -145,6 +147,9 @@ export class Prefetcher {
|
||||
/** One connection's loop: chunks until there is nothing left to fetch. */
|
||||
private async work() {
|
||||
while (!this.stopped) {
|
||||
// The pool's share got smaller: fewer loops (waiting on the pool, they'd count as work short of a connection).
|
||||
// Two leaving at once is one too many for a tick (TICK_MS), whose loop comes back.
|
||||
if (this.running > this.loops) return
|
||||
const next = this.take()
|
||||
if (!next) return
|
||||
const { target, span } = next
|
||||
|
||||
Reference in new issue
Block a user