Pending ladder by class, twins by content, dropped classes at the edge
Test and deploy / test (push) Successful in 3m9s
Test and deploy / deploy (push) Successful in 40m8s

The prober names 6% of the unnamed releases it finishes and the pending
queue (44M) grows eleven times faster than it probes. A sample of 44,530
finished pending rows shows the headers predict a name: size bucket, name
shape, completeness, tool and group separate 0% classes from 95% ones,
while the learner keyed on tool and group alone (LOG.md (130)).

- lib/release/classes.ts, names::pending_class, SQL pending_class():
  a release's class (size × shape × partial, 1..70), the three in step.
- Migration 6800: pending_releases.klass and target_bytes (the first
  target's article; candidates carry bytes now), pending_classes with
  each class's climbs, names, waiting and verdict, the claim's index.
- The ladder claims the classes with the best record first (a prior for
  the unexplored, at most a quarter of a claim from one), the cheapest
  target first within one, then the oldest of the rest; records classes
  like signatures and drops them at the same thresholds; discards their
  waiting releases. Claimed rows sorted in code: an UPDATE returns none.
- The rule set's droppedClasses: the edge keeps unnamed releases of a
  dropped class out of the pending store (counted as garbage), the
  explored sample excepted.
- hash_twins.ts: a probed file whose first 16 KiB and size match a named
  catalog release's names a pending release the probe didn't, and sets
  the inner name of a catalog release without one.
- pending:classes classes the rows from before and seeds the record from
  the finished ones; rules:classes publishes the dropped classes;
  releases:readable also marks the rows' candidates unreadable.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Fable 5.1 committed 2026-11-02 12:22:32 +01:00
1 parent 81a5dce45a
commit 44c0c73b05
20 files changed
+927 -55

No files matched your search

+8
View File
@@ -12,6 +12,11 @@ export interface Rules {
priorities: Record<string, number>
/** Dropped signatures, as "tool\tgroup" (GarbageFilter's table). */
droppedSignatures: string[]
/**
* Dropped classes of unnamed releases, as "tool\tgroup\tklass" (lib/release/classes.ts; the pending ladder's record,
* LOG.md (130)): the edge keeps them out of the pending store, the explored sample excepted.
*/
droppedClasses: string[]
/** Tools whose random-named uploads are dropped in every group (named < 1%; LOG.md 2026-10-08 (5)). */
droppedTools: string[]
/** A random-named file of at most this many parts: a random subject per article, needs bodies. */
@@ -53,6 +58,7 @@ export const DEFAULT_RULES: Omit<Rules, 'priorities' | 'droppedSignatures'> = {
droppedPosters: ['celebmovie', 'celebwow', 'celebfx', 'iluvcelebs', 'nudecelebs', 'celebsextapes'],
dropStubArchives: true,
dropNonsensePieces: true,
droppedClasses: [],
}
/** The stub-archive rule's bounds (edge/src/names.rs): Power-Post's 250 kB default article, the fakes' size span. */
@@ -178,11 +184,13 @@ export function ownerOf(groups: string[], priorities: Record<string, number>): s
export class RuleSet {
private signatures: Set<string>
readonly classes: Set<string>
private tools: Set<string>
private posters: string[]
constructor(readonly rules: Rules) {
this.posters = rules.droppedPosters ?? []
this.signatures = new Set(rules.droppedSignatures)
this.classes = new Set(rules.droppedClasses ?? [])
this.tools = new Set(rules.droppedTools)
}