Daan MeijerandClaude Fable 5.1
66980829dc
With the hash typed as the column (f537add) the planner walks every twin of a
pair by index; 55 pairs have over 500 twins, up to 131k, and a flush with such
a pair ran the query for 37 s inside its transaction, the other flushes and
node-api's class counters waiting on it. A LATERAL per pair now reads its
lowest 500 release ids and takes the first named one: 49 ms for 305 pairs with
the five hottest among them. LOG.md (138).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
37 lines
1.9 KiB
TypeScript
37 lines
1.9 KiB
TypeScript
import db from '@adonisjs/lucid/services/db'
|
|
|
|
/**
|
|
* Twins by content (LOG.md (130)): a probed file's first 16 KiB and size (file_hashes, what the par2 merge and the
|
|
* repeated verdict match) in a catalog release that has a name, readable or found inside. The same film posted
|
|
* under scrambles by other tools gets that name without another fetch. Releases with a junk verdict don't count.
|
|
*/
|
|
/** Twins read per pair: the hottest pairs have 131k. */
|
|
const TWIN_SCAN = 500
|
|
|
|
export async function twinNames(
|
|
pairs: { hash16k: string; size: number }[],
|
|
trx: { rawQuery: typeof db.rawQuery } = db
|
|
): Promise<Map<string, string>> {
|
|
const out = new Map<string, string>()
|
|
if (pairs.length === 0) return out
|
|
// Per pair the first named twin among its lowest TWIN_SCAN release ids. hash16k is typed as the column
|
|
// (char(32)): as text the join cast the column and scanned all of file_hashes, 4 s a call; and with the index
|
|
// the hottest pairs (55 with over 500 twins, up to 131k, all readable) made a call walk every twin, 37 s inside
|
|
// the flush's transaction (LOG.md (138)). One name is all a pair needs.
|
|
const { rows } = await trx.rawQuery(
|
|
`SELECT v.hash16k, v.size, t.name
|
|
FROM jsonb_to_recordset(?::jsonb) AS v(hash16k char(32), size bigint)
|
|
CROSS JOIN LATERAL (
|
|
SELECT coalesce(r.inner_name, r.name) AS name
|
|
FROM (SELECT release_id FROM file_hashes h WHERE h.hash16k = v.hash16k AND h.size = v.size ORDER BY release_id LIMIT ?) h
|
|
JOIN releases r ON r.id = h.release_id AND (r.readable OR r.inner_name IS NOT NULL)
|
|
WHERE NOT EXISTS (SELECT 1 FROM release_verdicts x WHERE x.release_id = r.id)
|
|
ORDER BY r.readable DESC, r.id LIMIT 1) t`,
|
|
[JSON.stringify(pairs), TWIN_SCAN]
|
|
)
|
|
for (const r of rows) out.set(`${r.hash16k}\t${r.size}`, r.name)
|
|
return out
|
|
}
|
|
|
|
export const twinKey = (hash16k: string, size: number) => `${hash16k}\t${size}`
|