Rebuild: phase 2's prober reads the probe queue
Test and deploy / test (push) Successful in 50s
Test and deploy / deploy (push) Successful in 1m46s

probe:run (compose service prober, PROBE_CONNECTIONS, default 6) claims
waiting candidates of settled releases (lone, then par2, then archive),
fetches each article with BODY over connections of its own (no
compression) and inspects it: the yEnc name and size, the content type
from the first bytes, a par2 set's files, a RAR volume's entries and
whether they're encrypted. Results per release and kind in
release_inspections (CHUNK_TABLESPACE, the HDD); on the release
passworded, inner_name (the largest content name) and inspected_at.
430 = missing, not yEnc = error, both final; connection trouble puts
the candidate back.

Fetch and parse in one process: an article takes ~0.1 s per connection,
parsing a few ms (no queue in between). Parsers from voyager 1 (lib/yenc,
lib/rar, lib/par2, lib/content sniff) with their tests; XoverClient.body;
yencode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-11 15:32:49 +02:00
1 parent d72fe2f13f
commit 8bd6d6aa5f
30 files changed
+1666 -7

No files matched your search

+22 -1
View File
@@ -1,7 +1,7 @@
import { test } from '@japa/runner'
import net from 'node:net'
import { deflateSync } from 'node:zlib'
import { XoverClient, parseOverviews } from '#lib/nntp/index'
import { MissingArticle, XoverClient, parseOverviews } from '#lib/nntp/index'
import { unzipSync } from 'node:zlib'
const LINES = [
@@ -49,6 +49,11 @@ function server(): Promise<{ port: number; close: () => void }> {
sock.write('215 NewsGroups Follow\r\n')
send(Buffer.concat([body, Buffer.from('.\r\n')]))
}
} else if (line === 'BODY <gone@test>') sock.write('430 No Such Article\r\n')
else if (line.startsWith('BODY')) {
sock.write('222 0 <one@test> body\r\n')
// Raw bytes, a dot-stuffed line among them.
send(Buffer.concat([Buffer.from([0x3d, 0x79, 0xff, 0x00, 0x0d, 0x0a]), Buffer.from('..stuffed\r\nlast\r\n.\r\n')]))
} else if (line === 'QUIT') sock.end('205 bye\r\n')
}
})
@@ -58,6 +63,22 @@ function server(): Promise<{ port: number; close: () => void }> {
}
test.group('nntp / XoverClient', () => {
test('reads a body by message-id as bytes, raw; 430 is MissingArticle', async ({ assert }) => {
const { port, close } = await server()
const client = await XoverClient.connect({ host: '127.0.0.1', port, user: 'u', password: 'p', compress: false })
try {
const body = await client.body('one@test')
assert.deepEqual([...body.subarray(0, 6)], [0x3d, 0x79, 0xff, 0x00, 0x0d, 0x0a])
assert.equal(body.subarray(6).toString('latin1'), '..stuffed\r\nlast\r\n')
await assert.rejects(() => client.body('<gone@test>'), MissingArticle)
// The connection is still usable after both.
assert.equal((await client.body('<one@test>')).length, body.length)
} finally {
client.close()
close()
}
})
test('parses overview lines into the fields nntp-js gives', ({ assert }) => {
const [[number, fields]] = parseOverviews(LINES[0] + '\r\n')
assert.equal(number, 101)