Daan MeijerandClaude Opus 5.5
cfe85abc3e
Eweka sends some ranges' compressed XOVER truncated, the same bytes every time, while the plain reply is whole (a.b.models 3317215-3367214: 1,675,600 bytes that don't inflate, twice; 50,000 lines plain; tools/false_end.ts). Not a false end of reply. The worker's corrupt report removed the chunk and the range was fetched again, with the same bytes: one range was reported 540 times. (Before this morning's change the inflate check waited for more data until the 10-minute timeout, like voyager 1's 'chunk never answered' ranges.) A reported chunk's start goes into ranges.plain_chunks; the prefetcher fetches those over one connection without XFEATURE COMPRESS GZIP (NntpPool.usePlain, opened on demand) and deflates them here. A plain chunk reported again sets the range aside (status failed, error). The capture (edge:bench --capture) fetches such a chunk plain too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
34 lines
1.4 KiB
TypeScript
34 lines
1.4 KiB
TypeScript
/**
|
|
* Whether a compressed XOVER reply that doesn't inflate is the server's: the
|
|
* range fetched raw twice (compared with a saved copy), and once uncompressed.
|
|
*
|
|
* node --env-file=.env --import=@poppinss/ts-exec tools/false_end.ts <group> <from> <to> [saved.gz]
|
|
*/
|
|
import { readFileSync } from 'node:fs'
|
|
import { unzipSync } from 'node:zlib'
|
|
import { XoverClient } from '#lib/nntp/index'
|
|
|
|
const [group, from, to, saved] = process.argv.slice(2)
|
|
const env = process.env
|
|
const connect = (compress: boolean) =>
|
|
XoverClient.connect({ host: env.NNTP_HOST!, port: Number(env.NNTP_PORT ?? 563), user: env.NNTP_USER, password: env.NNTP_PASSWORD, secure: (env.NNTP_SECURE ?? 'true') !== 'false', compress, timeoutMs: 120_000 })
|
|
const gz = await connect(true)
|
|
await gz.group(group)
|
|
for (let i = 0; i < 2; i++) {
|
|
const raw = await gz.xoverRaw(Number(from), Number(to))
|
|
let inflates = 'inflates'
|
|
try {
|
|
unzipSync(raw)
|
|
} catch (e: any) {
|
|
inflates = `doesn't inflate (${e.message})`
|
|
}
|
|
const same = saved ? (Buffer.compare(raw, readFileSync(saved)) === 0 ? ', same bytes as saved' : ', differs from saved') : ''
|
|
console.log(`compressed #${i + 1}: ${raw.length} bytes, ${inflates}${same}`)
|
|
}
|
|
gz.close()
|
|
const plain = await connect(false)
|
|
await plain.group(group)
|
|
const lines = await plain.xover(Number(from), Number(to))
|
|
console.log(`plain: ${lines.length} overview lines`)
|
|
plain.close()
|