Rebuild: prefetch stats a minute; the completeness check inflates off the event loop
The coordinator logs a line a minute: chunks fetched, articles/s, seconds a chunk, how busy its connections were. It showed the workstation's runs are bound by its 100 Mbit line now (10 connections 99% busy, ~89 Mbit/s: compressed XOVER is ~69 B an article on recent boneless), not by the code. The check that a compressed reply is complete (inflating it) ran on the coordinator's event loop, ~60 ms a chunk; now on a thread-pool thread. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
9dc0a27bb4
commit
cae6e4f60d
2 files changed
+25
-2
No files matched your search
@@ -1,9 +1,10 @@
|
||||
import { promisify } from 'node:util'
|
||||
import net from 'node:net'
|
||||
import tls from 'node:tls'
|
||||
import { deflate, deflateSync, unzipSync } from 'node:zlib'
|
||||
import { deflate, deflateSync, unzip } from 'node:zlib'
|
||||
|
||||
const deflateAsync = promisify(deflate)
|
||||
const unzipAsync = promisify(unzip)
|
||||
|
||||
export interface XoverClientOptions {
|
||||
host: string
|
||||
@@ -241,7 +242,8 @@ export class XoverClient {
|
||||
const all = this.buffer()
|
||||
const raw = all.subarray(0, all.length - DOT_LINE.length)
|
||||
try {
|
||||
const inflated = unzipSync(raw)
|
||||
// On a thread-pool thread: a chunk's ~25 MB of text took the coordinator's event loop ~60 ms.
|
||||
const inflated = await unzipAsync(raw)
|
||||
this.take(all.length)
|
||||
return { raw: Buffer.from(raw), text: wantText ? inflated.toString('utf8') : '' }
|
||||
} catch {
|
||||
|
||||
Reference in new issue
Block a user