Readable names: a token-by-token classifier for the readable flag
Searches with the readable filter on returned scrambles: with a random
4-letter query nearly every hit was one ("Zx5AjNaK-mh7CFg6Hf8uu",
"Description - Lt5nHjlo3RiuBb3C", "PsNdnYiugfZ", "bd5b...-addon"), and
15% of a 6,000-row sample of the catalog's readable names were. looksRandom
knows single-token shapes only; separators, spaces, a random "extension"
of any length and a packaging word hid the rest.
isUnreadableName (lib/release/readable.ts) and names::is_unreadable_name
read a name token by token: words (one case or Capitalized), scrambles
(letter/digit and case switches, lone letters between capitals, inner
capital runs, few vowels, hex) and neutral tokens (digits, codes like
S02E10 and 1080p, CamelCase, packaging words); a name is unreadable when
its scrambles outweigh its words. Scene tags' "i" for "I" and the
NZBSPLIT hash are read as such. On the evaluation sets: 554 of 559
random-search names caught, 0 of 630 real names flagged (LOG.md (129)).
The edge's readable flag and the prober's inner names use it; the drop
rules keep looksRandom, which is narrower. An unnamed release goes to the
pending store (decision f), so more of the obfuscated groups' releases
wait for a probe now. releases:readable clears the flag on the existing
catalog's rows by name (a dry run by default).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
2643266559
commit
81a5dce45a
10 files changed
+497
-12
No files matched your search
@@ -2,6 +2,7 @@ import { hash } from 'node:crypto'
|
||||
import type { Overview } from '../nntp/index.js'
|
||||
import { FileAggregator, type AggregatedFile, isArchiveStart, isLaterVolume, sevenZipVolume } from '../files/index.js'
|
||||
import { RuleSet, isRandomFile, isScrambledFile, uploadTool, type Rules } from '../rules/index.js'
|
||||
import { isUnreadableName } from '../release/index.js'
|
||||
import { SegmentScanner, scanOverviewText, type EdgeSegment } from './scan.js'
|
||||
import { encodeChunk, type ChunkFile } from '../chunks/index.js'
|
||||
|
||||
@@ -458,7 +459,7 @@ export function chunkOf(key: string, files: AggregatedFile<EdgeSegment>[], uploa
|
||||
bytes,
|
||||
firstArticle,
|
||||
lastArticle,
|
||||
readable: chunkFiles.some((f) => !isRandomFile(f.name)),
|
||||
readable: chunkFiles.some((f) => !isUnreadableName(f.name)),
|
||||
hasPar2: chunkFiles.some((f) => /\.par2$/i.test(f.name)),
|
||||
partial: files.some((f) => f.segments.length < f.segments[0].totalParts),
|
||||
payload: encodeChunk(chunkFiles),
|
||||
|
||||
Reference in new issue
Block a user