Rebuild: completeness as indexed, file totals from any counter, pausable tails, progress by articles
Test and deploy / test (push) Successful in 21s
Test and deploy / deploy (push) Skipped

- releases.complete is computed with every batch (files >= the
  counter's total) instead of only when a release settles after 6 quiet
  hours: the 'complete' filter matched nothing in a fresh catalog.
- A release's file total comes from any counter form (counterTotal:
  '(01/10)', 'File 27 of 132', …), not only [i/M]; the post rule's key
  keeps the [i/M] total, so keys are unchanged. Both implementations.
- groups.tail: a group's live tail can be paused (its history and owner
  priority stay), e.g. to let other groups' history go first.
- Stats: history by articles, and the current pass (history ranges
  before those deferred 2000 priorities behind) as its own figure; the
  header showed 0.1% of ranges counting 145k deferred ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-09 13:06:03 +02:00
1 parent 7b7175c229
commit d0f3497ec9
8 files changed
+57 -17

No files matched your search

+6 -2
View File
@@ -111,8 +111,9 @@ class BatchStore {
for (let i = 0; i < sorted.length; i += 1000) {
const chunk = sorted.slice(i, i + 1000)
const { rows } = await trx.rawQuery(
`INSERT INTO releases (release_key, name, source, poster, groups, posted_at, file_total, files, segments, bytes, chunks, first_article, last_article, readable, has_par2)
SELECT v.key, v.name, v.source, v.poster, v.groups, to_timestamp(v.posted_at / 1000.0), v.file_total, v.files, v.segments, v.bytes, 1, v.first_article, v.last_article, v.readable, v.has_par2
`INSERT INTO releases (release_key, name, source, poster, groups, posted_at, file_total, files, segments, bytes, chunks, first_article, last_article, readable, has_par2, complete)
SELECT v.key, v.name, v.source, v.poster, v.groups, to_timestamp(v.posted_at / 1000.0), v.file_total, v.files, v.segments, v.bytes, 1, v.first_article, v.last_article, v.readable, v.has_par2,
CASE WHEN v.file_total IS NULL THEN NULL ELSE v.files >= v.file_total END
FROM jsonb_to_recordset(?::jsonb) AS v(key text, name text, source text, poster text, groups jsonb, posted_at bigint, file_total int, files int, segments bigint, bytes bigint, first_article bigint, last_article bigint, readable boolean, has_par2 boolean)
ON CONFLICT (release_key) DO UPDATE SET
files = releases.files + excluded.files,
@@ -125,6 +126,9 @@ class BatchStore {
last_article = greatest(releases.last_article, excluded.last_article),
readable = releases.readable OR excluded.readable,
has_par2 = releases.has_par2 OR excluded.has_par2,
-- As it stands (the counter's files all found); settling makes it final.
complete = CASE WHEN coalesce(releases.file_total, excluded.file_total) IS NULL THEN NULL
ELSE releases.files + excluded.files >= coalesce(releases.file_total, excluded.file_total) END,
last_seen_at = now()
RETURNING id, release_key, (xmax = 0) AS inserted`,
[