Rebuild: prefetch stats a minute; the completeness check inflates off the event loop

The coordinator logs a line a minute: chunks fetched, articles/s,
seconds a chunk, how busy its connections were. It showed the
workstation's runs are bound by its 100 Mbit line now (10 connections
99% busy, ~89 Mbit/s: compressed XOVER is ~69 B an article on recent
boneless), not by the code.

The check that a compressed reply is complete (inflating it) ran on the
coordinator's event loop, ~60 ms a chunk; now on a thread-pool thread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-09 03:14:49 +02:00
1 parent 9dc0a27bb4
commit cae6e4f60d
2 files changed
+25 -2

No files matched your search

+4 -2
View File
@@ -1,9 +1,10 @@
import { promisify } from 'node:util'
import net from 'node:net'
import tls from 'node:tls'
import { deflate, deflateSync, unzipSync } from 'node:zlib'
import { deflate, deflateSync, unzip } from 'node:zlib'
const deflateAsync = promisify(deflate)
const unzipAsync = promisify(unzip)
export interface XoverClientOptions {
host: string
@@ -241,7 +242,8 @@ export class XoverClient {
const all = this.buffer()
const raw = all.subarray(0, all.length - DOT_LINE.length)
try {
const inflated = unzipSync(raw)
// On a thread-pool thread: a chunk's ~25 MB of text took the coordinator's event loop ~60 ms.
const inflated = await unzipAsync(raw)
this.take(all.length)
return { raw: Buffer.from(raw), text: wantText ? inflated.toString('utf8') : '' }
} catch {