Rebuild: the coordinator spools replies without inflating them
On homelab (1 Gbit, 20 connections, ~630k articles/s) the coordinator used 1.3-1.7 cores, mostly GC of the text it inflated only to know a compressed reply was complete, plus a copy of every reply. A raw reply is now complete when it ends in ".\r\n" (no copy): 0.4-0.5 cores at the same rate. ".\r\n" can also end a network read inside the compressed data (~1 in 16M reads, a few dozen times over the whole target). Such a chunk doesn't inflate on the worker: it reports it (POST ranges/:id/corrupt), the spooled chunk is removed and the range goes back to the queue; the prefetcher fetches it again. The rest of that reply breaks the connection's next command, so the connection is replaced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
53bfea8d15
commit
a979b5d11c
6 files changed
+60
-10
No files matched your search
@@ -74,6 +74,12 @@ export default class NodeController {
|
||||
return this.lifecycle(ctx, (rangeId, nodeId) => Leases.finish(rangeId, nodeId))
|
||||
}
|
||||
|
||||
/** POST /ranges/:id/corrupt { from }: a chunk that doesn't inflate; the range goes back. */
|
||||
async corrupt(ctx: HttpContext) {
|
||||
const from = Number(ctx.request.input('from'))
|
||||
return this.lifecycle(ctx, (rangeId, nodeId) => Leases.corrupt(rangeId, nodeId, from))
|
||||
}
|
||||
|
||||
async release(ctx: HttpContext) {
|
||||
return this.lifecycle(ctx, (rangeId, nodeId) => Leases.release(rangeId, nodeId))
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user