daan/usenet-indexer
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Probe 7z listings and ZIP starts

Browse Source 
7z keeps its file list at the end of the archive, so the 7z-tail
candidates (the last segment of the highest volume) were queued but
never probed, and the .7z.001 start only gave the type. The prober now
claims 7z-tail candidates after the archives; lib/sevenzip finds the end
header in the article, decodes the LZMA-packed header (a TypeScript port
of the LZMA SDK's reference decoder) and lists the files with their
sizes. An AES-encrypted header means passworded without names. Names
feed the inner name and the judge's fake rule.

An unsplit .7z of several segments now also gets a tail candidate, and
.zip files an archive candidate: their first segment's local headers
give the first entries and whether they are encrypted. Both in the Rust
edge and the TypeScript indexer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-23 11:55:07 +02:00
1 parent 3b61215f6b
commit d446bb57df
25 files changed
+4840 -11
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

tests/unit/edge/range_indexer.spec.ts
+14
View File
@@ -55,6 +55,20 @@ test.group('edge / RangeIndexer', () => {
assert.equal(batch.next, 1_000_001)
})
test("archive candidates: a split 7z's start and its highest volume's end, an unsplit 7z's start and end, a ZIP's start", ({ assert }) => {
const indexer = new RangeIndexer({ group: GROUP, start: 1, end: 1_000_000, rules, edgeArticles: 10 })
indexer.add([
...post(100, 'Split.Seven.7z.001', 2, { counter: 'Split.Seven [1/3]' }),
...post(110, 'Split.Seven.7z.003', 2, { counter: 'Split.Seven [3/3]' }),
...post(120, 'Split.Seven.7z.002', 2, { counter: 'Split.Seven [2/3]' }),
...post(200, 'Whole.Seven.7z', 3, { counter: 'Whole.Seven [1/1]' }),
...post(300, 'Some.Zip.zip', 2, { counter: 'Some.Zip [1/1]' }),
], 200_000)
indexer.finish()
const got = indexer.take().candidates.map((c) => `${c.kind} ${c.file}`).sort()
assert.deepEqual(got, ['7z-tail Split.Seven.7z.003', '7z-tail Whole.Seven.7z', 'archive Some.Zip.zip', 'archive Split.Seven.7z.001', 'archive Whole.Seven.7z'])
})
test('files the edges cut leave as fragments; one missing parts in the middle is only counted', ({ assert }) => {
const indexer = new RangeIndexer({ group: GROUP, start: 1, end: 1_000_000, rules, edgeArticles: 1000 })
indexer.add([