Twins by content: at most 500 twins read per pair
Test and deploy / test (push) Successful in 3m3s
Test and deploy / deploy (push) Successful in 1m16s

With the hash typed as the column (f537add) the planner walks every twin of a
pair by index; 55 pairs have over 500 twins, up to 131k, and a flush with such
a pair ran the query for 37 s inside its transaction, the other flushes and
node-api's class counters waiting on it. A LATERAL per pair now reads its
lowest 500 release ids and takes the first named one: 49 ms for 305 pairs with
the five hottest among them. LOG.md (138).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Fable 5.1 committed 2026-11-04 09:49:40 +01:00
1 parent f537add573
commit 66980829dc
1 file changed
+13 -6
+13 -6
View File
@@ -5,22 +5,29 @@ import db from '@adonisjs/lucid/services/db'
* repeated verdict match) in a catalog release that has a name, readable or found inside. The same film posted
* under scrambles by other tools gets that name without another fetch. Releases with a junk verdict don't count.
*/
/** Twins read per pair: the hottest pairs have 131k. */
const TWIN_SCAN = 500
export async function twinNames(
pairs: { hash16k: string; size: number }[],
trx: { rawQuery: typeof db.rawQuery } = db
): Promise<Map<string, string>> {
const out = new Map<string, string>()
if (pairs.length === 0) return out
// hash16k typed as the column (char(32)): as text the join cast the column and scanned all of file_hashes
// instead of file_hashes_hash_size_release_idx, 4 s a call (LOG.md (138)).
// Per pair the first named twin among its lowest TWIN_SCAN release ids. hash16k is typed as the column
// (char(32)): as text the join cast the column and scanned all of file_hashes, 4 s a call; and with the index
// the hottest pairs (55 with over 500 twins, up to 131k, all readable) made a call walk every twin, 37 s inside
// the flush's transaction (LOG.md (138)). One name is all a pair needs.
const { rows } = await trx.rawQuery(
`SELECT DISTINCT ON (h.hash16k, h.size) h.hash16k, h.size, coalesce(r.inner_name, r.name) AS name
`SELECT v.hash16k, v.size, t.name
FROM jsonb_to_recordset(?::jsonb) AS v(hash16k char(32), size bigint)
JOIN file_hashes h ON h.hash16k = v.hash16k AND h.size = v.size
CROSS JOIN LATERAL (
SELECT coalesce(r.inner_name, r.name) AS name
FROM (SELECT release_id FROM file_hashes h WHERE h.hash16k = v.hash16k AND h.size = v.size ORDER BY release_id LIMIT ?) h
JOIN releases r ON r.id = h.release_id AND (r.readable OR r.inner_name IS NOT NULL)
WHERE NOT EXISTS (SELECT 1 FROM release_verdicts x WHERE x.release_id = r.id)
ORDER BY h.hash16k, h.size, r.readable DESC, r.id`,
[JSON.stringify(pairs)]
ORDER BY r.readable DESC, r.id LIMIT 1) t`,
[JSON.stringify(pairs), TWIN_SCAN]
)
for (const r of rows) out.set(`${r.hash16k}\t${r.size}`, r.name)
return out