Pending ladder by class, twins by content, dropped classes at the edge
The prober names 6% of the unnamed releases it finishes and the pending queue (44M) grows eleven times faster than it probes. A sample of 44,530 finished pending rows shows the headers predict a name: size bucket, name shape, completeness, tool and group separate 0% classes from 95% ones, while the learner keyed on tool and group alone (LOG.md (130)). - lib/release/classes.ts, names::pending_class, SQL pending_class(): a release's class (size × shape × partial, 1..70), the three in step. - Migration 6800: pending_releases.klass and target_bytes (the first target's article; candidates carry bytes now), pending_classes with each class's climbs, names, waiting and verdict, the claim's index. - The ladder claims the classes with the best record first (a prior for the unexplored, at most a quarter of a claim from one), the cheapest target first within one, then the oldest of the rest; records classes like signatures and drops them at the same thresholds; discards their waiting releases. Claimed rows sorted in code: an UPDATE returns none. - The rule set's droppedClasses: the edge keeps unnamed releases of a dropped class out of the pending store (counted as garbage), the explored sample excepted. - hash_twins.ts: a probed file whose first 16 KiB and size match a named catalog release's names a pending release the probe didn't, and sets the inner name of a catalog release without one. - pending:classes classes the rows from before and seeds the record from the finished ones; rules:classes publishes the dropped classes; releases:readable also marks the rows' candidates unreadable. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
81a5dce45a
commit
44c0c73b05
20 files changed
+927
-55
No files matched your search
@@ -12,6 +12,11 @@ export interface Rules {
|
||||
priorities: Record<string, number>
|
||||
/** Dropped signatures, as "tool\tgroup" (GarbageFilter's table). */
|
||||
droppedSignatures: string[]
|
||||
/**
|
||||
* Dropped classes of unnamed releases, as "tool\tgroup\tklass" (lib/release/classes.ts; the pending ladder's record,
|
||||
* LOG.md (130)): the edge keeps them out of the pending store, the explored sample excepted.
|
||||
*/
|
||||
droppedClasses: string[]
|
||||
/** Tools whose random-named uploads are dropped in every group (named < 1%; LOG.md 2026-10-08 (5)). */
|
||||
droppedTools: string[]
|
||||
/** A random-named file of at most this many parts: a random subject per article, needs bodies. */
|
||||
@@ -53,6 +58,7 @@ export const DEFAULT_RULES: Omit<Rules, 'priorities' | 'droppedSignatures'> = {
|
||||
droppedPosters: ['celebmovie', 'celebwow', 'celebfx', 'iluvcelebs', 'nudecelebs', 'celebsextapes'],
|
||||
dropStubArchives: true,
|
||||
dropNonsensePieces: true,
|
||||
droppedClasses: [],
|
||||
}
|
||||
|
||||
/** The stub-archive rule's bounds (edge/src/names.rs): Power-Post's 250 kB default article, the fakes' size span. */
|
||||
@@ -178,11 +184,13 @@ export function ownerOf(groups: string[], priorities: Record<string, number>): s
|
||||
|
||||
export class RuleSet {
|
||||
private signatures: Set<string>
|
||||
readonly classes: Set<string>
|
||||
private tools: Set<string>
|
||||
private posters: string[]
|
||||
constructor(readonly rules: Rules) {
|
||||
this.posters = rules.droppedPosters ?? []
|
||||
this.signatures = new Set(rules.droppedSignatures)
|
||||
this.classes = new Set(rules.droppedClasses ?? [])
|
||||
this.tools = new Set(rules.droppedTools)
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user