Daan MeijerandClaude Fable 5.1
44c0c73b05
The prober names 6% of the unnamed releases it finishes and the pending queue (44M) grows eleven times faster than it probes. A sample of 44,530 finished pending rows shows the headers predict a name: size bucket, name shape, completeness, tool and group separate 0% classes from 95% ones, while the learner keyed on tool and group alone (LOG.md (130)). - lib/release/classes.ts, names::pending_class, SQL pending_class(): a release's class (size × shape × partial, 1..70), the three in step. - Migration 6800: pending_releases.klass and target_bytes (the first target's article; candidates carry bytes now), pending_classes with each class's climbs, names, waiting and verdict, the claim's index. - The ladder claims the classes with the best record first (a prior for the unexplored, at most a quarter of a claim from one), the cheapest target first within one, then the oldest of the rest; records classes like signatures and drops them at the same thresholds; discards their waiting releases. Claimed rows sorted in code: an UPDATE returns none. - The rule set's droppedClasses: the edge keeps unnamed releases of a dropped class out of the pending store (counted as garbage), the explored sample excepted. - hash_twins.ts: a probed file whose first 16 KiB and size match a named catalog release's names a pending release the probe didn't, and sets the inner name of a catalog release without one. - pending:classes classes the rows from before and seeds the record from the finished ones; rules:classes publishes the dropped classes; releases:readable also marks the rows' candidates unreadable. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
28 lines
1.3 KiB
TypeScript
28 lines
1.3 KiB
TypeScript
import db from '@adonisjs/lucid/services/db'
|
|
|
|
/**
|
|
* Twins by content (LOG.md (130)): a probed file's first 16 KiB and size (file_hashes, what the par2 merge and the
|
|
* repeated verdict match) in a catalog release that has a name, readable or found inside. The same film posted
|
|
* under scrambles by other tools gets that name without another fetch. Releases with a junk verdict don't count.
|
|
*/
|
|
export async function twinNames(
|
|
pairs: { hash16k: string; size: number }[],
|
|
trx: { rawQuery: typeof db.rawQuery } = db
|
|
): Promise<Map<string, string>> {
|
|
const out = new Map<string, string>()
|
|
if (pairs.length === 0) return out
|
|
const { rows } = await trx.rawQuery(
|
|
`SELECT DISTINCT ON (h.hash16k, h.size) h.hash16k, h.size, coalesce(r.inner_name, r.name) AS name
|
|
FROM jsonb_to_recordset(?::jsonb) AS v(hash16k text, size bigint)
|
|
JOIN file_hashes h ON h.hash16k = v.hash16k AND h.size = v.size
|
|
JOIN releases r ON r.id = h.release_id AND (r.readable OR r.inner_name IS NOT NULL)
|
|
WHERE NOT EXISTS (SELECT 1 FROM release_verdicts x WHERE x.release_id = r.id)
|
|
ORDER BY h.hash16k, h.size, r.readable DESC, r.id`,
|
|
[JSON.stringify(pairs)]
|
|
)
|
|
for (const r of rows) out.set(`${r.hash16k}\t${r.size}`, r.name)
|
|
return out
|
|
}
|
|
|
|
export const twinKey = (hash16k: string, size: number) => `${hash16k}\t${size}`
|