Rebuild: a poster's constant tag no longer makes one release of all their uploads
Test and deploy / test (push) Successful in 28s
Test and deploy / deploy (push) Skipped

Release 145637 held four albums: the poster puts '[KiLL.dA.FuCkIn.
EaStErRaBbIt]----' before every upload, and the text outside the quotes
(when it names a release) was the key with the poster. The same made
one release of a poster's albums under '(PowBeertje80 Posted)' (a post
id, with the same [i/M]), of masterposts' films, and of one volume of
every set in a DVD9 post.

Now, when the file's stem describes something itself (12+ letters and
digits, two words of 3+ letters) and the text outside the quotes shares
no word with it (4+ letters as part of the stem, 3-letter words and
numbers whole; boilerplate like www/net/posted not counted; no 5+ digit
per-post id), that text is taken for the poster's tag and the stem
says which release the file is. Albums of track files ('FTA … Ketchup'),
'DIE_ZEIT_35' with '35_China_dl', 'FTD769997' posts, scene names with
abbreviated file names ('grp-sm') and generic sidecars ('file_id.diz')
keep their grouping.

releases:regroup re-keys a sample of the catalog with the current rule:
on homelab's test data 145637 → 4 albums, 111566 → 17 albums of 27
files, the rest of the subject releases that change mostly lose an ad
image ('Flyer-download-2-Day.gif'); stem releases unchanged. Ported to
edge-bench-rs: check.ts still finds all captures identical (ath 2013:
16 → 76 releases on both sides).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-09 11:23:33 +02:00
1 parent 3d256619e5
commit 59481037fb
4 files changed
+214 -4

No files matched your search

+42 -2
View File
@@ -1,5 +1,6 @@
import { hash as digest } from 'node:crypto'
import { looksLikeFilename } from '../headers/file_key.js'
import { looksRandom } from './random.js'
export interface ReleaseIdentity {
key: string
@@ -106,6 +107,39 @@ function hash(...parts: (string | number | undefined)[]) {
return digest('sha1', parts.map((p) => p ?? '').join('\n'))
}
/** A name that says what it is: 12+ letters and digits, two words of 3+ letters. */
function describes(stem: string): boolean {
return stem.replace(/[^A-Za-z0-9]/g, '').length >= 12 && /[A-Za-z]{3,}[^A-Za-z0-9]+[A-Za-z]{3,}/.test(stem)
}
/** Words of a subject that say nothing about what is posted: hosts, posting boilerplate. */
const TAG_WORDS = new Set(['www', 'http', 'https', 'com', 'net', 'org', 'info', 'the', 'and', 'presents', 'posted', 'post', 'powered', 'yenc', 'part', 'rar', 'par2', 'nzb', 'voor', 'for', 'from'])
/**
* Whether the subject's text outside the quotes is a poster's constant tag
* rather than about the file: it shares no word with the file's stem. A word
* of 4+ letters may be part of the stem ("ketchup" in "ketchup_dance"); a
* 3-letter word or a number ("35" in "35_China_dl") must be a whole word;
* boilerplate doesn't count. A number of 5+ digits is a per-post id
* ("FTD 769997", "(781442)"), so never a constant tag.
*/
export function isPosterTag(remainder: string, stem: string): boolean {
const words = stem.toLowerCase().replace(/[^a-z0-9]+/g, ' ').trim()
const compact = words.replace(/ /g, '')
const whole = new Set(words.split(' '))
// A per-post id, also inside a word ("FTD772544").
if (/\d{5,}/.test(remainder)) return false
for (const word of remainder.toLowerCase().split(/[^a-z0-9]+/)) {
if (/^\d+$/.test(word)) {
if (word.length >= 2 && whole.has(word)) return false
continue
}
if (word.length < 3 || TAG_WORDS.has(word)) continue
if (word.length >= 4 ? compact.includes(word) : whole.has(word)) return false
}
return true
}
/**
* Decides which release (download) a file belongs to. Returns null when the
* file cannot be grouped from its name alone (random extensionless names in
@@ -125,7 +159,13 @@ export function releaseIdentity(input: ReleaseInput): ReleaseIdentity | null {
// Text outside the quotes that names the release, e.g. "Some.Release [01/45] - "x.rar"".
const remainder = subjectRemainder(input.subject)
if (namesRelease(remainder)) {
// Unless it's the poster's constant tag ("[KiLL.dA…]---- (01/10) "Album.rar"", "(X Posted)"),
// which would make one release of all their uploads: a readable file name that shares no word
// with it says which release the file is (LOG.md 2026-10-09 (8)).
// Only when the stem itself describes something ("Cephalgy.-.Weisses.Fleisch.2003", not a scene
// abbreviation like "grp-sm" under "Some.Movie.2008-GRP", nor "file_id.diz").
const tag = describes(stem) && !looksRandom(stem) && isPosterTag(remainder, stem)
if (!tag && namesRelease(remainder)) {
return {
key: hash('subject', input.poster, remainder),
name: remainder,
@@ -135,7 +175,7 @@ export function releaseIdentity(input: ReleaseInput): ReleaseIdentity | null {
}
// Not a name, but a post id (e.g. `83110629-n [10/18] - "random"`): with the
// [i/M] counter it groups the files of one post whose names are random.
if (fileTotal && isPostToken(remainder)) {
if (!tag && fileTotal && isPostToken(remainder)) {
return { key: hash('post', input.poster, remainder, fileTotal), name: remainder, fileTotal, source: 'post' }
}
// The [i/M] total is informational, not part of the key: posters often