voyager 3: the batch store writes releases once, by id references and bytea keys
Each release chunk is a release inserted once: release_key the SHA-1 of the identity, the owner group and the first article (deterministic; no numbering under a group lock), identity_key the identity's SHA-1 bytes, poster_id and group_set_id through catalog_refs (cached per process), source as a code (weak-stem added), the name into release_texts. A finished release's candidates are claimable at once; a provisional tail release is finished by a later chunk of its key, a finished one sent again is left alone. Gone: joins into earlier releases and across groups, key numbering, the group lock, the fragment merger. Interim: chunks, counts and samples keyed by ranges.block_id, a range without a block gets one. Readers follow: search (posters and group_sets joined, words through release_texts, posters by id, source by code), the par2 merger, the sidecar folder, the judge's scraped rule, the spot inspector, the spotter, the NZB service, releases:regroup, edge:bench. Tests insert fixtures through tests/functional/helpers/releases.ts; the suites pass again. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
bc4e17e86f
commit
2580f20a43
32 files changed
+962
-497
No files matched your search
@@ -56,7 +56,7 @@ export class SidecarFolder {
|
||||
// A candidate whose release went (merged, deleted) leaves with its batch: a separate scan for those over the whole
|
||||
// queue took 5+ min a round at 13M (LOG.md 2026-10-17 (72)).
|
||||
const { rows: queued } = await db.rawQuery(
|
||||
`SELECT f.release_id AS queued_id, r.id, r.release_key, r.group_id, r.poster, r.posted_at, r.files, r.file_total, r.readable
|
||||
`SELECT f.release_id AS queued_id, r.id, r.group_id, r.poster_id, r.posted_at, r.files, r.file_total, r.readable
|
||||
FROM fold_candidates f LEFT JOIN releases r ON r.id = f.release_id ORDER BY f.created_at LIMIT ?`,
|
||||
[BATCH]
|
||||
)
|
||||
@@ -70,10 +70,10 @@ export class SidecarFolder {
|
||||
round.sidecars++
|
||||
if (!c.posted_at) continue
|
||||
const { rows: siblings } = await db.rawQuery(
|
||||
`SELECT r.id, r.release_key, r.group_id, r.files, r.file_total, r.posted_at, r.readable, r.name FROM releases r
|
||||
`SELECT r.id, r.group_id, r.files, r.file_total, r.posted_at, r.readable, r.name FROM releases r
|
||||
WHERE r.posted_at BETWEEN ?::timestamptz - make_interval(mins => ?) AND ?::timestamptz + make_interval(mins => ?)
|
||||
AND r.group_id = ? AND r.poster = ? AND r.id <> ? AND r.files > ?`,
|
||||
[c.posted_at, WINDOW_MINUTES, c.posted_at, WINDOW_MINUTES, c.group_id, c.poster, c.id, c.files]
|
||||
AND r.group_id = ? AND r.poster_id = ? AND r.id <> ? AND r.files > ?`,
|
||||
[c.posted_at, WINDOW_MINUTES, c.posted_at, WINDOW_MINUTES, c.group_id, c.poster_id, c.id, c.files]
|
||||
)
|
||||
const sibling = pickSibling(files, c.file_total ? Number(c.file_total) : null, siblings.map((s: any) => ({ ...s, id: Number(s.id), files: Number(s.files), file_total: s.file_total === null ? null : Number(s.file_total) })))
|
||||
if (!sibling) continue
|
||||
|
||||
Reference in new issue
Block a user