Daan MeijerandClaude Fable 5.1
2580f20a43
Each release chunk is a release inserted once: release_key the SHA-1 of the identity, the owner group and the first article (deterministic; no numbering under a group lock), identity_key the identity's SHA-1 bytes, poster_id and group_set_id through catalog_refs (cached per process), source as a code (weak-stem added), the name into release_texts. A finished release's candidates are claimable at once; a provisional tail release is finished by a later chunk of its key, a finished one sent again is left alone. Gone: joins into earlier releases and across groups, key numbering, the group lock, the fragment merger. Interim: chunks, counts and samples keyed by ranges.block_id, a range without a block gets one. Readers follow: search (posters and group_sets joined, words through release_texts, posters by id, source by code), the par2 merger, the sidecar folder, the judge's scraped rule, the spot inspector, the spotter, the NZB service, releases:regroup, edge:bench. Tests insert fixtures through tests/functional/helpers/releases.ts; the suites pass again. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
35 lines
1.5 KiB
TypeScript
35 lines
1.5 KiB
TypeScript
import { createHash } from 'node:crypto'
|
|
|
|
/**
|
|
* The catalog's keys as voyager 3 stores them (migration 6000): 20-byte SHA-1s in bytea columns.
|
|
* - identity_key: the identity's hash (lib/release/identity.ts gives it as 40 hex characters).
|
|
* - release_key: one upload of an identity, unique: the identity, the owner group and the first article the window
|
|
* saw of it. The same release sent twice (a tail release provisionally, then finished) hits the same row; a repost
|
|
* starts at another article and is a release of its own (decision d). Deterministic, so no numbering under a lock.
|
|
*/
|
|
const HEX40 = /^[0-9a-f]{40}$/i
|
|
|
|
function sha1(text: string): Buffer {
|
|
return createHash('sha1').update(text).digest()
|
|
}
|
|
|
|
/** The identity key's bytes: 40 hex characters decoded, any other text (a spot's, a test's) hashed. */
|
|
export function identityKey(key: string): Buffer {
|
|
return HEX40.test(key) ? Buffer.from(key, 'hex') : sha1(key)
|
|
}
|
|
|
|
/** The release key's bytes for an identity in a group, first seen at an article (null for releases without one: a spot's). */
|
|
export function releaseKey(
|
|
identity: string,
|
|
groupId: number | null,
|
|
firstArticle: number | null
|
|
): Buffer {
|
|
return sha1(`release\n${identity}\n${groupId ?? ''}\n${firstArticle ?? ''}`)
|
|
}
|
|
|
|
/** A stored key as hex, for logs, the API and release_merges.from_key. */
|
|
export function keyHex(key: Buffer | Uint8Array | string | null | undefined): string | null {
|
|
if (key === null || key === undefined) return null
|
|
return typeof key === 'string' ? key : Buffer.from(key).toString('hex')
|
|
}
|