Lone articles under scrambled names are per-article; inactive groups get no corpus fill
One article of a multi-part file under a name like "nYDeXeF49dieZ1J7flK-JFzZ3j.ZEDsr" or "bslXlwgPiqZadhRa" (a random subject and poster per article, with fake counters) was a release: looksRandom misses these names for their separators. 160k of alt.binaries.storage's 162k non-Bob releases, and about 60% of all single-article releases in a 2% sample. looks_scrambled / isScrambledFile: at least half the stem's letters and digits in tokens of 5+ that switch case or letter/digit 3 times or more (93% of the storage spam). Used for lone articles given up before the block's end only; at the end the strict looksRandom stays, as the rest of such a file can be in the next block. The namer's re-index (partialFiles) does the same. The prefetcher's corpus fill skips groups that aren't active. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
5132ab9083
commit
cfce8884d4
7 files changed
+81
-12
No files matched your search
@@ -1,7 +1,7 @@
|
||||
import { hash } from 'node:crypto'
|
||||
import type { Overview } from '../nntp/index.js'
|
||||
import { FileAggregator, type AggregatedFile, isArchiveStart, isLaterVolume, sevenZipVolume } from '../files/index.js'
|
||||
import { RuleSet, isRandomFile, uploadTool, type Rules } from '../rules/index.js'
|
||||
import { RuleSet, isRandomFile, isScrambledFile, uploadTool, type Rules } from '../rules/index.js'
|
||||
import { SegmentScanner, scanOverviewText, type EdgeSegment } from './scan.js'
|
||||
import { encodeChunk, type ChunkFile } from '../chunks/index.js'
|
||||
|
||||
@@ -280,8 +280,10 @@ export class RangeIndexer {
|
||||
}
|
||||
for (const list of byFile.values()) {
|
||||
if (this.options.partialFiles) {
|
||||
// As the Rust window's files(): one article of a multi-part random file is no release.
|
||||
if (list.length === 1 && (list[0].number !== 1 || list[0].totalParts > 1) && isRandomFile(list[0].file.name)) {
|
||||
// As the Rust window's files(): one article of a multi-part random file is no release; a scrambled name only
|
||||
// before the end, where the rest of the file can still come.
|
||||
const name = list[0].file.name
|
||||
if (list.length === 1 && (list[0].number !== 1 || list[0].totalParts > 1) && (isRandomFile(name) || (!atEnd && isScrambledFile(name)))) {
|
||||
this.count('per-article', uploadTool(list[0].messageId), 1, 0)
|
||||
continue
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user