Twins by content: at most 500 twins read per pair
With the hash typed as the column (f537add) the planner walks every twin of a
pair by index; 55 pairs have over 500 twins, up to 131k, and a flush with such
a pair ran the query for 37 s inside its transaction, the other flushes and
node-api's class counters waiting on it. A LATERAL per pair now reads its
lowest 500 release ids and takes the first named one: 49 ms for 305 pairs with
the five hottest among them. LOG.md (138).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
f537add573
commit
66980829dc
1 file changed
+15
-8
@@ -5,22 +5,29 @@ import db from '@adonisjs/lucid/services/db'
|
||||
* repeated verdict match) in a catalog release that has a name, readable or found inside. The same film posted
|
||||
* under scrambles by other tools gets that name without another fetch. Releases with a junk verdict don't count.
|
||||
*/
|
||||
/** Twins read per pair: the hottest pairs have 131k. */
|
||||
const TWIN_SCAN = 500
|
||||
|
||||
export async function twinNames(
|
||||
pairs: { hash16k: string; size: number }[],
|
||||
trx: { rawQuery: typeof db.rawQuery } = db
|
||||
): Promise<Map<string, string>> {
|
||||
const out = new Map<string, string>()
|
||||
if (pairs.length === 0) return out
|
||||
// hash16k typed as the column (char(32)): as text the join cast the column and scanned all of file_hashes
|
||||
// instead of file_hashes_hash_size_release_idx, 4 s a call (LOG.md (138)).
|
||||
// Per pair the first named twin among its lowest TWIN_SCAN release ids. hash16k is typed as the column
|
||||
// (char(32)): as text the join cast the column and scanned all of file_hashes, 4 s a call; and with the index
|
||||
// the hottest pairs (55 with over 500 twins, up to 131k, all readable) made a call walk every twin, 37 s inside
|
||||
// the flush's transaction (LOG.md (138)). One name is all a pair needs.
|
||||
const { rows } = await trx.rawQuery(
|
||||
`SELECT DISTINCT ON (h.hash16k, h.size) h.hash16k, h.size, coalesce(r.inner_name, r.name) AS name
|
||||
`SELECT v.hash16k, v.size, t.name
|
||||
FROM jsonb_to_recordset(?::jsonb) AS v(hash16k char(32), size bigint)
|
||||
JOIN file_hashes h ON h.hash16k = v.hash16k AND h.size = v.size
|
||||
JOIN releases r ON r.id = h.release_id AND (r.readable OR r.inner_name IS NOT NULL)
|
||||
WHERE NOT EXISTS (SELECT 1 FROM release_verdicts x WHERE x.release_id = r.id)
|
||||
ORDER BY h.hash16k, h.size, r.readable DESC, r.id`,
|
||||
[JSON.stringify(pairs)]
|
||||
CROSS JOIN LATERAL (
|
||||
SELECT coalesce(r.inner_name, r.name) AS name
|
||||
FROM (SELECT release_id FROM file_hashes h WHERE h.hash16k = v.hash16k AND h.size = v.size ORDER BY release_id LIMIT ?) h
|
||||
JOIN releases r ON r.id = h.release_id AND (r.readable OR r.inner_name IS NOT NULL)
|
||||
WHERE NOT EXISTS (SELECT 1 FROM release_verdicts x WHERE x.release_id = r.id)
|
||||
ORDER BY r.readable DESC, r.id LIMIT 1) t`,
|
||||
[JSON.stringify(pairs), TWIN_SCAN]
|
||||
)
|
||||
for (const r of rows) out.set(`${r.hash16k}\t${r.size}`, r.name)
|
||||
return out
|
||||
|
||||
Reference in new issue
Block a user