Stub-archive rule: 100 MB cap on every leg, ROT13 volume names are volumes
Test and deploy / test (push) Successful in 2m59s
Test and deploy / deploy (push) Successful in 4m1s

The first five hours in production (LOG.md (128)) dropped 522 passworded
videos of one a.b.erotica poster: ROT13-named volumes (".cnegN.rar" is
".partN.rar") of 1-270 MB in 258 kB articles to two groups, which the
article-size leg took for stubs. The cap that X and W already had now
applies to A as well (the catalog sample has 6 A-leg hits over 100 MB,
4 of them exe-only), and a ROT13 volume name is no single archive.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Fable 5.1 committed 2026-11-02 05:49:37 +01:00
1 parent e470e231d0
commit 2643266559
4 files changed
+43 -28

No files matched your search

+7 -5
View File
@@ -787,11 +787,13 @@ par2 can still join an open one; the start edge doesn't count, since a par2
follows its rar and 7% of the catalog's releases are start edges). A settled follows its rar and 7% of the catalog's releases are start edges). A settled
release of one unsplit zip, rar or 7z and nothing else is dropped as a release of one unsplit zip, rar or 7z and nothing else is dropped as a
**stub archive** (`stub-archive`, key: the legs tripped) when posted to **stub archive** (`stub-archive`, key: the legs tripped) when posted to
2+ groups and (A) its articles are ≤ 256 KiB with the release ≥ 1 MB 2+ groups, under 100 MB (the fakes stop at ~90 MB; hash-named uploads,
(Power-Post's 250 kB default; a one-article release always has small real albums and passworded videos above it don't), and (A) its articles
articles, which exempts e-books and sidecars), or (X) posted to 4+ groups, are ≤ 256 KiB with the release ≥ 1 MB (Power-Post's 250 kB default; a
or (W) by a webmail poster, X and W under 100 MB (the fakes stop at one-article release always has small articles, which exempts e-books and
~90 MB; hash-named uploads and real albums above it don't). A release sidecars), or (X) posted to 4+ groups, or (W) by a webmail poster. A
ROT13 volume name (".cnegN.rar" is ".partN.rar": a.b.erotica's passworded
videos, LOG.md (128)) is a volume, not a single archive. A release
touching a music, e-book, comics, tabs or fonts group is exempt: those touching a music, e-book, comics, tabs or fonts group is exempt: those
communities post real single small archives the same way (karaoke zips to communities post real single small archives the same way (karaoke zips to
a.b.sounds.karaoke + a.b.test with Power-Post's articles; 16 of 32,308 a.b.sounds.karaoke + a.b.test with Power-Post's articles; 16 of 32,308
+18 -11
View File
@@ -1298,7 +1298,8 @@ pub fn is_sidecar(name: &str) -> bool {
static EXECUTABLE: LazyLock<Regex> = LazyLock::new(|| re(r"(?i)\.(exe|scr|com|pif|bat|cmd|vbs|lnk)SP*$")); static EXECUTABLE: LazyLock<Regex> = LazyLock::new(|| re(r"(?i)\.(exe|scr|com|pif|bat|cmd|vbs|lnk)SP*$"));
/// An unsplit archive (the judge's ARCHIVE and VOLUME): one zip, rar or 7z that is not a volume of a set. /// An unsplit archive (the judge's ARCHIVE and VOLUME): one zip, rar or 7z that is not a volume of a set.
static ARCHIVE: LazyLock<Regex> = LazyLock::new(|| re(r"(?i)\.(zip|rar|7z)$")); static ARCHIVE: LazyLock<Regex> = LazyLock::new(|| re(r"(?i)\.(zip|rar|7z)$"));
static VOLUME: LazyLock<Regex> = LazyLock::new(|| re(r"(?i)\.(part[0-9]+\.rar|r[0-9][0-9]|z[0-9][0-9]|7z\.[0-9][0-9][0-9]|[0-9][0-9][0-9])$")); /// "cneg" is "part" in ROT13: the a.b.erotica posts named that way are volumes of passworded videos (LOG.md (128)).
static VOLUME: LazyLock<Regex> = LazyLock::new(|| re(r"(?i)\.((part|cneg)[0-9]+\.rar|r[0-9][0-9]|z[0-9][0-9]|7z\.[0-9][0-9][0-9]|[0-9][0-9][0-9])$"));
/// A poster at a free mail provider (the judge's WEBMAIL): the stub floods' throwaway accounts. /// A poster at a free mail provider (the judge's WEBMAIL): the stub floods' throwaway accounts.
static WEBMAIL: LazyLock<Regex> = LazyLock::new(|| { static WEBMAIL: LazyLock<Regex> = LazyLock::new(|| {
re(r"(?i)@(gmail|googlemail|yahoo|hotmail|outlook|live|msn|aol|gmx|web|t-online|freenet|mail|yandex|icloud|me|hushmail|protonmail|libero|orange|free|arcor|wanadoo|ymail|rocketmail|inbox|mail\.ru|bk|list)\.") re(r"(?i)@(gmail|googlemail|yahoo|hotmail|outlook|live|msn|aol|gmx|web|t-online|freenet|mail|yandex|icloud|me|hushmail|protonmail|libero|orange|free|arcor|wanadoo|ymail|rocketmail|inbox|mail\.ru|bk|list)\.")
@@ -1308,11 +1309,11 @@ static RELEASE_LIKE: LazyLock<Regex> = LazyLock::new(|| {
re(r"(?i)(\b(19|20)[0-9][0-9]\b|[._-](720p|1080p|2160p|x264|x265|hevc|bluray|web-?dl|webrip|hdtv|xvid|s[0-9][0-9]e[0-9][0-9]|german|multi|crack|repack|proper|flac|mp3|xxx|mp4|retail|keygen|portable|incl|v[0-9]+(\.[0-9]+)+)\b|-[A-Za-z0-9]{2,}$|\.[A-Za-z0-9]+\.[A-Za-z0-9]+\.)") re(r"(?i)(\b(19|20)[0-9][0-9]\b|[._-](720p|1080p|2160p|x264|x265|hevc|bluray|web-?dl|webrip|hdtv|xvid|s[0-9][0-9]e[0-9][0-9]|german|multi|crack|repack|proper|flac|mp3|xxx|mp4|retail|keygen|portable|incl|v[0-9]+(\.[0-9]+)+)\b|-[A-Za-z0-9]{2,}$|\.[A-Za-z0-9]+\.[A-Za-z0-9]+\.)")
}); });
/// The stub-archive rule (LOG.md (123)-(126)): one unsplit archive, no par2, settled, posted to 2+ groups, and its /// The stub-archive rule (LOG.md (123)-(126), (128)): one unsplit archive, no par2, settled, posted to 2+ groups,
/// articles at most STUB_ARTICLE_BYTES with the release at least STUB_MIN_BYTES (Power-Post's 250 kB default; a /// under STUB_MAX_BYTES (the fakes stop at ~90 MB; the hash-named uploads, real albums and passworded videos above
/// one-article release always has small articles, so e-books and sidecars are exempt), or 4+ groups, or a webmail /// it don't), and its articles at most STUB_ARTICLE_BYTES with the release at least STUB_MIN_BYTES (Power-Post's
/// poster, these two under STUB_MAX_BYTES (no article-size signal: the fakes stop at ~90 MB, the hash-named /// 250 kB default; a one-article release always has small articles, so e-books and sidecars are exempt), or 4+
/// uploads and real albums above it don't). /// groups, or a webmail poster.
pub const STUB_ARTICLE_BYTES: u64 = 262_144; pub const STUB_ARTICLE_BYTES: u64 = 262_144;
pub const STUB_MIN_BYTES: u64 = 1_000_000; pub const STUB_MIN_BYTES: u64 = 1_000_000;
pub const STUB_MAX_BYTES: u64 = 100_000_000; pub const STUB_MAX_BYTES: u64 = 100_000_000;
@@ -1346,21 +1347,21 @@ fn single_archive<'a, 'b>(files: &'b [JudgedFile<'a>]) -> Option<&'b JudgedFile<
/// stub-archive rule. Costs 16 of 32,308 known fakes in the 217k sample (LOG.md (127)). /// stub-archive rule. Costs 16 of 32,308 known fakes in the 217k sample (LOG.md (127)).
static STUB_EXEMPT_GROUPS: LazyLock<Regex> = LazyLock::new(|| re(r"(?i)(sound|mp3|karaoke|music|audio|flac|lossless|e-?book|epub|comic|guitar|font|magazine)")); static STUB_EXEMPT_GROUPS: LazyLock<Regex> = LazyLock::new(|| re(r"(?i)(sound|mp3|karaoke|music|audio|flac|lossless|e-?book|epub|comic|guitar|font|magazine)"));
/// The legs of the stub-archive rule a settled single archive trips ("A" article size, "X" 4+ groups, "W" webmail), /// The legs of the stub-archive rule a settled single archive under STUB_MAX_BYTES trips ("A" article size, "X" 4+
/// empty when none. /// groups, "W" webmail), empty when none.
pub fn stub_archive_legs(f: &JudgedFile) -> String { pub fn stub_archive_legs(f: &JudgedFile) -> String {
let mut legs = String::new(); let mut legs = String::new();
if f.groups.len() < 2 || f.groups.iter().any(|g| test(&STUB_EXEMPT_GROUPS, g)) { if f.groups.len() < 2 || f.bytes >= STUB_MAX_BYTES || f.groups.iter().any(|g| test(&STUB_EXEMPT_GROUPS, g)) {
return legs; return legs;
} }
let article = f.bytes / u64::from(f.segments.max(1)); let article = f.bytes / u64::from(f.segments.max(1));
if article <= STUB_ARTICLE_BYTES && f.bytes >= STUB_MIN_BYTES { if article <= STUB_ARTICLE_BYTES && f.bytes >= STUB_MIN_BYTES {
legs.push('A'); legs.push('A');
} }
if f.groups.len() >= STUB_WIDE_GROUPS && f.bytes < STUB_MAX_BYTES { if f.groups.len() >= STUB_WIDE_GROUPS {
legs.push('X'); legs.push('X');
} }
if f.bytes < STUB_MAX_BYTES && test(&WEBMAIL, f.poster) { if test(&WEBMAIL, f.poster) {
legs.push('W'); legs.push('W');
} }
legs legs
@@ -1724,6 +1725,12 @@ mod single_archive_rule_tests {
// A volume, not a single archive; a hash-named 2 GB upload from a gmail-looking poster. // A volume, not a single archive; a hash-named 2 GB upload from a gmail-looking poster.
assert_eq!(verdict("Some.Movie.2014.part01.rar", "a@gmail.com", 2, 48, 12_000_000, true, &rules), None); assert_eq!(verdict("Some.Movie.2014.part01.rar", "a@gmail.com", 2, 48, 12_000_000, true, &rules), None);
assert_eq!(verdict("cbcd32359460df1fa9b7d01d8720.rar", "b648de0d1c0f7a9a77be273d@gmail.com", 2, 3163, 2_340_000_000, true, &rules), None); assert_eq!(verdict("cbcd32359460df1fa9b7d01d8720.rar", "b648de0d1c0f7a9a77be273d@gmail.com", 2, 3163, 2_340_000_000, true, &rules), None);
// ROT13-named volumes of passworded videos in a.b.erotica, 258 kB articles, two groups, 1-270 MB; and the
// 100 MB cap on every leg (LOG.md (128)).
assert_eq!(verdict("Ghful.17.12.07.Qnvvfl.Fgbar.cneg3.rar", "nospam@nospam.nl (EvH)", 2, 5, 1_147_000, true, &rules), None);
assert_eq!(verdict("Ghful.16.07.05.Wbwb.Xvff.cneg1.rar", "nospam@nospam.nl (EvH)", 2, 1051, 271_000_000, true, &rules), None);
assert_eq!(verdict("Tinasoft EasyCafe 2.2.14 by OmerSalaj.rar", "Seven7i@mail.com ({Seven7i})", 4, 463, 117_000_000, true, &rules), None);
assert_eq!(verdict("Tipard DVD Ripper - 10 0 12 x86 x64 incl Patch.rar", "Seven7i@mail.com ({Seven7i})", 4, 244, 62_000_000, true, &rules), Some(("stub-archive", "AXW".to_string())));
let off = Rules { drop_stub_archives: false, ..Rules::defaults(Default::default()) }; let off = Rules { drop_stub_archives: false, ..Rules::defaults(Default::default()) };
assert_eq!(verdict("Chat Watch 1.2.1.zip", "jahnunsinos007@gmail.com", 2, 48, 12_000_000, true, &off), None); assert_eq!(verdict("Chat Watch 1.2.1.zip", "jahnunsinos007@gmail.com", 2, 48, 12_000_000, true, &off), None);
// A karaoke zip crossposted to a.b.test with Power-Post's articles: its group exempts it (LOG.md (127)). // A karaoke zip crossposted to a.b.test with Power-Post's articles: its group exempts it (LOG.md (127)).
+13 -12
View File
@@ -31,10 +31,10 @@ export interface Rules {
*/ */
droppedPosters: string[] droppedPosters: string[]
/** /**
* Drop settled releases of one unsplit archive without a par2, posted to 2+ groups, whose articles are at most * Drop settled releases of one unsplit archive without a par2, posted to 2+ groups, under STUB_MAX_BYTES, whose
* STUB_ARTICLE_BYTES with the release at least STUB_MIN_BYTES, or posted to 4+ groups, or by a webmail poster, the * articles are at most STUB_ARTICLE_BYTES with the release at least STUB_MIN_BYTES, or posted to 4+ groups, or by
* last two under STUB_MAX_BYTES: the exe-stub and recycled-installer floods under film and software names (55% * a webmail poster: the exe-stub and recycled-installer floods under film and software names (55% of the catalog
* of the catalog on 2026-10-30; 99.3% of the known fakes, 0 known content; LOG.md (123)-(126)). * on 2026-10-30; 99.3% of the known fakes, 0 known content; LOG.md (123)-(126), (128)).
*/ */
dropStubArchives: boolean dropStubArchives: boolean
/** /**
@@ -87,7 +87,8 @@ const SIDECAR = /\.(par2|nfo|sfv|txt|jpe?g|png|nzb)$/i
const EXECUTABLE = /\.(exe|scr|com|pif|bat|cmd|vbs|lnk)\s*$/i const EXECUTABLE = /\.(exe|scr|com|pif|bat|cmd|vbs|lnk)\s*$/i
/** One zip, rar or 7z that is not a volume of a set. */ /** One zip, rar or 7z that is not a volume of a set. */
const ARCHIVE = /\.(zip|rar|7z)$/i const ARCHIVE = /\.(zip|rar|7z)$/i
const VOLUME = /\.(part\d+\.rar|r\d\d|z\d\d|7z\.\d\d\d|\d\d\d)$/i /** "cneg" is "part" in ROT13: the a.b.erotica posts named that way are volumes of passworded videos (LOG.md (128)). */
const VOLUME = /\.((part|cneg)\d+\.rar|r\d\d|z\d\d|7z\.\d\d\d|\d\d\d)$/i
/** A poster at a free mail provider: the stub floods' throwaway accounts. */ /** A poster at a free mail provider: the stub floods' throwaway accounts. */
const WEBMAIL = /@(gmail|googlemail|yahoo|hotmail|outlook|live|msn|aol|gmx|web|t-online|freenet|mail|yandex|icloud|me|hushmail|protonmail|libero|orange|free|arcor|wanadoo|ymail|rocketmail|inbox|mail\.ru|bk|list)\./i const WEBMAIL = /@(gmail|googlemail|yahoo|hotmail|outlook|live|msn|aol|gmx|web|t-online|freenet|mail|yandex|icloud|me|hushmail|protonmail|libero|orange|free|arcor|wanadoo|ymail|rocketmail|inbox|mail\.ru|bk|list)\./i
/** A name that reads as a release: a year, a scene tag, a group suffix or dotted words. */ /** A name that reads as a release: a year, a scene tag, a group suffix or dotted words. */
@@ -106,17 +107,17 @@ function singleArchive(files: JudgedFile[]): JudgedFile | null {
} }
/** /**
* The legs of the stub-archive rule a settled single archive trips: "A" (articles <= STUB_ARTICLE_BYTES, release >= * The legs of the stub-archive rule a settled single archive under STUB_MAX_BYTES (the fakes stop at ~90 MB;
* STUB_MIN_BYTES: a one-article release always has small articles, so e-books and sidecars are exempt), "X" (4+ * hash-named uploads, real albums and passworded videos above it don't) trips: "A" (articles <= STUB_ARTICLE_BYTES,
* groups) and "W" (webmail poster), the last two under STUB_MAX_BYTES (the fakes stop at ~90 MB; hash-named uploads * release >= STUB_MIN_BYTES: a one-article release always has small articles, so e-books and sidecars are exempt),
* and real albums above it don't). Empty when none. * "X" (4+ groups) and "W" (webmail poster). Empty when none.
*/ */
export function stubArchiveLegs(f: JudgedFile): string { export function stubArchiveLegs(f: JudgedFile): string {
if (f.groups.length < 2 || f.groups.some((g) => STUB_EXEMPT_GROUPS.test(g))) return '' if (f.groups.length < 2 || f.bytes >= STUB_MAX_BYTES || f.groups.some((g) => STUB_EXEMPT_GROUPS.test(g))) return ''
let legs = '' let legs = ''
if (f.bytes / Math.max(f.segments, 1) <= STUB_ARTICLE_BYTES && f.bytes >= STUB_MIN_BYTES) legs += 'A' if (f.bytes / Math.max(f.segments, 1) <= STUB_ARTICLE_BYTES && f.bytes >= STUB_MIN_BYTES) legs += 'A'
if (f.groups.length >= STUB_WIDE_GROUPS && f.bytes < STUB_MAX_BYTES) legs += 'X' if (f.groups.length >= STUB_WIDE_GROUPS) legs += 'X'
if (f.bytes < STUB_MAX_BYTES && WEBMAIL.test(f.poster)) legs += 'W' if (WEBMAIL.test(f.poster)) legs += 'W'
return legs return legs
} }
+5
View File
@@ -83,6 +83,11 @@ test.group('rules / verdicts', () => {
assert.deepEqual(rules.judge('k', [archive('Some.Movie.2014.720p.BluRay-GRP.rar', 'poster@example.org', 2, 100, 70_000_000)]), { rule: null }) assert.deepEqual(rules.judge('k', [archive('Some.Movie.2014.720p.BluRay-GRP.rar', 'poster@example.org', 2, 100, 70_000_000)]), { rule: null })
assert.deepEqual(rules.judge('k', [archive('Some.Movie.2014.part01.rar', 'a@gmail.com', 2, 48, 12_000_000)]), { rule: null }) assert.deepEqual(rules.judge('k', [archive('Some.Movie.2014.part01.rar', 'a@gmail.com', 2, 48, 12_000_000)]), { rule: null })
assert.deepEqual(rules.judge('k', [archive('cbcd32359460df1fa9b7d01d8720.rar', 'b648de0d1c0f7a9a77be273d@gmail.com', 2, 3163, 2_340_000_000)]), { rule: null }) assert.deepEqual(rules.judge('k', [archive('cbcd32359460df1fa9b7d01d8720.rar', 'b648de0d1c0f7a9a77be273d@gmail.com', 2, 3163, 2_340_000_000)]), { rule: null })
// ROT13-named volumes of passworded videos in a.b.erotica, 258 kB articles, two groups, 1-270 MB; and the 100 MB cap on every leg (LOG.md (128)).
assert.deepEqual(rules.judge('k', [archive('Ghful.17.12.07.Qnvvfl.Fgbar.cneg3.rar', 'nospam@nospam.nl (EvH)', 2, 5, 1_147_000)]), { rule: null })
assert.deepEqual(rules.judge('k', [archive('Ghful.16.07.05.Wbwb.Xvff.cneg1.rar', 'nospam@nospam.nl (EvH)', 2, 1051, 271_000_000)]), { rule: null })
assert.deepEqual(rules.judge('k', [archive('Tinasoft EasyCafe 2.2.14 by OmerSalaj.rar', 'Seven7i@mail.com ({Seven7i})', 4, 463, 117_000_000)]), { rule: null })
assert.include(rules.judge('k', [archive('Tipard DVD Ripper - 10 0 12 x86 x64 incl Patch.rar', 'Seven7i@mail.com ({Seven7i})', 4, 244, 62_000_000)]), { rule: 'stub-archive', key: 'AXW' })
// A karaoke zip crossposted to a.b.test with Power-Post's articles: its group exempts it (LOG.md (127)). // A karaoke zip crossposted to a.b.test with Power-Post's articles: its group exempts it (LOG.md (127)).
assert.deepEqual(rules.judge('k', [{ ...archive('KV-1055806 - Walker Hayes - U Gurl.zip', 'home@home.com (TexasDj)', 2, 13, 3_300_000), groups: ['alt.binaries.sounds.karaoke', 'alt.binaries.test'] }]), { rule: null }) assert.deepEqual(rules.judge('k', [{ ...archive('KV-1055806 - Walker Hayes - U Gurl.zip', 'home@home.com (TexasDj)', 2, 13, 3_300_000), groups: ['alt.binaries.sounds.karaoke', 'alt.binaries.test'] }]), { rule: null })
// A rar with its par2 is no single archive. // A rar with its par2 is no single archive.