Rebuild: the worker joins stream pieces once per frame, gzips posts off its thread

decodeFrames joined each network piece (~64 KB) onto the frame so far:
a chunk of MBs copied over and over, 5.4% of a worker's time in an
end-to-end profile (the bench reads files, so it never showed). Pieces
are now joined once a frame is complete. A batch post's gzip (1.6%) runs
on a thread-pool thread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-09 03:06:17 +02:00
1 parent bd1c846f27
commit 168605148f
3 files changed
+40 -12

No files matched your search

+11
View File
@@ -25,4 +25,15 @@ test.group('edge / stream frames', () => {
}
}, /inside a frame/)
})
test('frames come out whole whatever the piece sizes, also pieces holding several frames', async ({ assert }) => {
const sent = Array.from({ length: 12 }, (_, i) => ({ from: i * 10 + 1, to: i * 10 + 10, data: Buffer.alloc((i * 997) % 5000, i) }))
const whole = Buffer.concat(sent.map(encodeFrame))
for (const size of [1, 3, 20, 21, 64, 1000, 4096, whole.length]) {
const pieces = []
for (let at = 0; at < whole.length; at += size) pieces.push(new Uint8Array(whole.subarray(at, at + size)))
const frames = []
for await (const frame of decodeFrames(pieces)) frames.push(frame)
assert.deepEqual(frames.map((f) => [f.from, f.to, f.data.toString('hex')]), sent.map((f) => [f.from, f.to, f.data.toString('hex')]), `pieces of ${size}`)
}
})
})