Pending ladder by class, twins by content, dropped classes at the edge
The prober names 6% of the unnamed releases it finishes and the pending queue (44M) grows eleven times faster than it probes. A sample of 44,530 finished pending rows shows the headers predict a name: size bucket, name shape, completeness, tool and group separate 0% classes from 95% ones, while the learner keyed on tool and group alone (LOG.md (130)). - lib/release/classes.ts, names::pending_class, SQL pending_class(): a release's class (size × shape × partial, 1..70), the three in step. - Migration 6800: pending_releases.klass and target_bytes (the first target's article; candidates carry bytes now), pending_classes with each class's climbs, names, waiting and verdict, the claim's index. - The ladder claims the classes with the best record first (a prior for the unexplored, at most a quarter of a claim from one), the cheapest target first within one, then the oldest of the rest; records classes like signatures and drops them at the same thresholds; discards their waiting releases. Claimed rows sorted in code: an UPDATE returns none. - The rule set's droppedClasses: the edge keeps unnamed releases of a dropped class out of the pending store (counted as garbage), the explored sample excepted. - hash_twins.ts: a probed file whose first 16 KiB and size match a named catalog release's names a pending release the probe didn't, and sets the inner name of a catalog release without one. - pending:classes classes the rows from before and seeds the record from the finished ones; rules:classes publishes the dropped classes; releases:readable also marks the rows' candidates unreadable. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
81a5dce45a
commit
44c0c73b05
20 files changed
+927
-55
No files matched your search
@@ -0,0 +1,55 @@
|
||||
/**
|
||||
* The class of an unnamed release (REBUILD.md decision f; LOG.md (130)): what the headers say about its chance of a
|
||||
* name, from the pending ladder's own record. Size bucket × name shape × partial, 1..70, kept in step with
|
||||
* names.rs pending_class and the SQL pending_class() (migration 6800): the batch store sets it with the SQL, the
|
||||
* edge skips the classes a lease marks dropped.
|
||||
*/
|
||||
|
||||
/** Size buckets: under 1 MB, 10 MB, 100 MB, 1 GB, and from 1 GB. */
|
||||
export const SIZE_BUCKETS = [1_000_000, 10_000_000, 100_000_000, 1_000_000_000]
|
||||
/** Name shapes, 1..7: hex, digits, base32, letters only, mixed letters and digits, with separators, other. */
|
||||
export const SHAPES = [
|
||||
'hex',
|
||||
'digits',
|
||||
'base32',
|
||||
'letters',
|
||||
'alnum',
|
||||
'separators',
|
||||
'other',
|
||||
] as const
|
||||
|
||||
export function sizeBucket(bytes: number): number {
|
||||
let i = 0
|
||||
while (i < SIZE_BUCKETS.length && bytes >= SIZE_BUCKETS[i]) i++
|
||||
return i
|
||||
}
|
||||
|
||||
export function nameShape(name: string): number {
|
||||
const s = name.replace(/\.[A-Za-z0-9]{1,4}$/, '')
|
||||
if (/^[0-9a-fA-F]{16,}$/.test(s)) return 1
|
||||
if (/^[0-9]{6,}$/.test(s)) return 2
|
||||
if (/^[a-z2-7]{20,}$/.test(s) && /[2-7]/.test(s)) return 3
|
||||
if (/^[A-Za-z]+$/.test(s)) return 4
|
||||
if (/^[A-Za-z0-9]+$/.test(s)) return 5
|
||||
if (/^[A-Za-z0-9._-]+$/.test(s)) return 6
|
||||
return 7
|
||||
}
|
||||
|
||||
/** 1..70: (size bucket × 7 + shape − 1) × 2 + partial + 1. */
|
||||
export function pendingClass(name: string, bytes: number, partial: boolean): number {
|
||||
return sizeBucket(bytes) * 14 + (nameShape(name) - 1) * 2 + (partial ? 1 : 0) + 1
|
||||
}
|
||||
|
||||
/** A class as droppedClasses lists it: "tool\tgroup\tklass". */
|
||||
export function classKey(tool: string, group: string, klass: number): string {
|
||||
return `${tool}\t${group}\t${klass}`
|
||||
}
|
||||
|
||||
/** A class in words ("10-100M hex partial"). */
|
||||
export function describeClass(klass: number): string {
|
||||
const k = klass - 1
|
||||
const size = Math.floor(k / 14)
|
||||
const shape = SHAPES[Math.floor((k % 14) / 2)]
|
||||
const sizes = ['<1M', '1-10M', '10-100M', '100M-1G', '1G+']
|
||||
return `${sizes[size]} ${shape}${k % 2 ? ' partial' : ''}`
|
||||
}
|
||||
@@ -4,3 +4,4 @@
|
||||
export { releaseIdentity, displayName, par2ReleaseIdentity, par2ReleaseName, rarReleaseIdentity, isGenericName, fileStem, sizeAfterName, runOn, subjectRemainder, namesRelease, isPostToken, type ReleaseIdentity, type ReleaseInput } from './identity.js'
|
||||
export { looksRandom } from './random.js'
|
||||
export { isUnreadableName, tokenKind } from './readable.js'
|
||||
export { pendingClass, classKey, describeClass, nameShape, sizeBucket } from './classes.js'
|
||||
Reference in new issue
Block a user