This website requires JavaScript.
5ab5399b43
Prober: the claim's random starting points drawn in code
Daan Meijer and Claude Opus 5.5
2026-10-11 16:00:23 +02:00
18f5d27413
Prober: claim from random points of the queue, not in id order
Daan Meijer and Claude Opus 5.5
2026-10-11 15:55:23 +02:00
4ed7f89951
Prober: no packaging or release-name-plus-extension as inner name
Daan Meijer and Claude Opus 5.5
2026-10-11 15:44:08 +02:00
738402aefa
Prober: an inspection per candidate; no inner name that is the release's name
Daan Meijer and Claude Opus 5.5
2026-10-11 15:37:32 +02:00
8bd6d6aa5f
Rebuild: phase 2's prober reads the probe queue
Daan Meijer and Claude Opus 5.5
2026-10-11 15:32:49 +02:00
d72fe2f13f
Tests: a corrupt chunk fails its range on the third report
Daan Meijer and Claude Opus 5.5
2026-10-11 14:22:10 +02:00
fa52d9256b
Rebuild: a chunk reported corrupt gets a second plain try before the range fails
Daan Meijer and Claude Opus 5.5
2026-10-11 14:21:44 +02:00
fd06b16bd9
deploy.sh: remove the images a deploy replaced
Daan Meijer and Claude Opus 5.5
2026-10-11 04:21:39 +02:00
40df52bafd
Rebuild: 8 more ranges being fetched than the connections need
Daan Meijer and Claude Opus 5.5
2026-10-11 03:36:24 +02:00
27a26c9614
Rebuild: a release key taken by another group's writer gets the next number
Daan Meijer and Claude Opus 5.5
2026-10-10 23:22:49 +02:00
b87203e41a
edge-node: a range whose batch is too large goes on in smaller batches
Daan Meijer and Claude Opus 5.5
2026-10-10 22:07:54 +02:00
7b07e00f90
Rebuild: the stream waits up to an hour for a chunk
Daan Meijer and Claude Opus 5.5
2026-10-10 17:21:24 +02:00
887df12ead
Rebuild: give up stalled requests, not slow ones
Daan Meijer and Claude Opus 5.5
2026-10-10 16:51:55 +02:00
dbccec932c
Rebuild: a chunk's deadline doubles each time it's hit
Daan Meijer and Claude Opus 5.5
2026-10-10 16:36:36 +02:00
d81a202172
Rebuild: chunk deadline 90 s
Daan Meijer and Claude Opus 5.5
2026-10-10 16:06:30 +02:00
fc1c64d5c0
Rebuild: drop NUL from batches; chunk deadline 180 s
Daan Meijer and Claude Opus 5.5
2026-10-10 13:12:27 +02:00
f4c0417066
Rebuild: enough ranges being fetched for every connection
Daan Meijer and Claude Opus 5.5
2026-10-10 08:44:09 +02:00
40aa78cda3
Rebuild: a deadline per chunk; batches with a huge fragment
Daan Meijer and Claude Opus 5.5
2026-10-10 08:30:04 +02:00
e095232d89
REBUILD.md: the deployment
Daan Meijer and Claude Opus 5.5
2026-10-10 00:10:44 +02:00
84d3af5511
Rebuild: a proper deployment for voyager 2
Daan Meijer and Claude Opus 5.5
2026-10-10 00:01:38 +02:00
678b3dd438
Rebuild: spread prefetch connections over ranges and groups
Daan Meijer and Claude Opus 5.5
2026-10-09 23:33:43 +02:00
4bf526ab6e
Rebuild: prefetch report samples what the connections do
Daan Meijer and Claude Opus 5.5
2026-10-09 22:40:50 +02:00
8475969145
Rebuild: hedged chunk fetches and connection max age (both off by default)
Daan Meijer and Claude Opus 5.5
2026-10-09 22:13:44 +02:00
fc8ec9ff48
Rebuild: tools/compress_ab.ts, compressed vs plain XOVER side by side
Daan Meijer and Claude Opus 5.5
2026-10-09 21:36:43 +02:00
8442c4fc44
Rebuild: 2-minute NNTP reply timeout; the Rust worker runs on homelab
Daan Meijer and Claude Opus 5.5
2026-10-09 21:05:37 +02:00
a8a139c6d9
edge-node: the edge worker in Rust
Daan Meijer and Claude Opus 5.5
2026-10-09 20:42:09 +02:00
dce9967806
edge-bench-rs: a library for the worker; rules from the lease; the wire format
Daan Meijer and Claude Opus 5.5
2026-10-09 20:27:17 +02:00
fc86dd00d1
Rebuild: a batch locks its range with an advisory lock, not the row
Daan Meijer and Claude Opus 5.5
2026-10-09 18:05:00 +02:00
99fffe30f4
Rebuild: stats from a fixed-size sample, refreshed every 5 minutes, given up after 2
Daan Meijer and Claude Opus 5.5
2026-10-09 17:49:45 +02:00
f642a49d7d
Rebuild: post a batch early when 100k files wait; log failing stats refreshes
Daan Meijer and Claude Opus 5.5
2026-10-09 17:45:00 +02:00
508d8b5723
Rebuild: one advisory lock per group when writing releases, not per key
Daan Meijer and Claude Opus 5.5
2026-10-09 14:10:48 +02:00
7836436605
edge-bench-rs README: the a.b.models capture (image sets, re-uploads); where the captures are
Daan Meijer and Claude Opus 5.5
2026-10-09 13:55:19 +02:00
cfe85abc3e
Rebuild: chunks the server's compression cuts short are fetched plain
Daan Meijer and Claude Opus 5.5
2026-10-09 13:51:25 +02:00
3737ea1542
Rebuild: tools to rank groups by real span (GROUP's low) and sample how full they are
Daan Meijer and Claude Opus 5.5
2026-10-09 13:31:55 +02:00
0613984de9
Rebuild: re-uploads are releases of their own
Daan Meijer and Claude Opus 5.5
2026-10-09 13:31:35 +02:00
d0f3497ec9
Rebuild: completeness as indexed, file totals from any counter, pausable tails, progress by articles
Daan Meijer and Claude Opus 5.5
2026-10-09 13:06:03 +02:00
7b7175c229
Rebuild: an image without words around it is keyed as a stem
Daan Meijer and Claude Opus 5.5
2026-10-09 12:23:47 +02:00
14758a5b7a
Rebuild: image sets, documents and audio albums as releases
Daan Meijer and Claude Opus 5.5
2026-10-09 12:21:00 +02:00
59481037fb
Rebuild: a poster's constant tag no longer makes one release of all their uploads
Daan Meijer and Claude Opus 5.5
2026-10-09 11:23:33 +02:00
3d256619e5
Rebuild spec: status (live tail, API, web interface, signatures)
Daan Meijer and Claude Opus 5.5
2026-10-09 10:50:54 +02:00
b97edf02c5
Rebuild: the web interface, from voyager 1
Daan Meijer and Claude Opus 5.5
2026-10-09 10:47:59 +02:00
8e52bff213
Rebuild: the catalog API and Newznab, from voyager 1
Daan Meijer and Claude Opus 5.5
2026-10-09 10:45:44 +02:00
57dfa3d749
Rebuild: rules:signatures, and voyager 1's 29 learned garbage signatures
Daan Meijer and Claude Opus 5.5
2026-10-09 10:42:00 +02:00
fd0f2bde75
Rebuild: the live tail
Daan Meijer and Claude Opus 5.5
2026-10-09 10:41:16 +02:00
df89045f98
Rebuild spec: decision f, no chunks for random-named releases in the header pass
Daan Meijer and Claude Opus 5.5
2026-10-09 10:17:33 +02:00
c5e0ad1f9f
Rebuild: tools/xover_eras.ts, XOVER speed per era of a group
Daan Meijer and Claude Opus 5.5
2026-10-09 09:57:19 +02:00
356d8db6b6
Rebuild spec: point to the edge-bench-rs README
Daan Meijer and Claude Opus 5.5
2026-10-09 09:40:50 +02:00
122a054eab
edge-bench-rs: README with the findings and how to compare the outputs
Daan Meijer and Claude Opus 5.5
2026-10-09 09:40:45 +02:00
b1a6f02de6
Rebuild spec: TypeScript vs Rust for the edge worker, measured
Daan Meijer and Claude Opus 5.5
2026-10-09 09:38:03 +02:00
c7b4ebe470
edge-bench-rs: libdeflate for chunks, memchr for lines and tabs
Daan Meijer and Claude Opus 5.5
2026-10-09 09:34:38 +02:00
b1c2db5984
edge-bench-rs: upload tools, release-name checks and common Date headers by hand
Daan Meijer and Claude Opus 5.5
2026-10-09 09:31:56 +02:00
43e74199d4
edge-bench-rs: file stems and subject remainders by hand, mimalloc
Daan Meijer and Claude Opus 5.5
2026-10-09 09:28:08 +02:00
bacc9aa3b3
Rebuild: edge-bench-rs, the edge worker's indexing in Rust, for comparison
Daan Meijer and Claude Opus 5.5
2026-10-09 09:20:53 +02:00
42b6cac3ee
Rebuild: edge:node --max-mbit caps a node's download
Daan Meijer and Claude Opus 5.5
2026-10-09 09:01:57 +02:00
104248d8a3
Rebuild spec: homelab measurements
Daan Meijer and Claude Opus 5.5
2026-10-09 07:48:14 +02:00
babd3d781d
Rebuild: test the corrupt-chunk report
Daan Meijer and Claude Opus 5.5
2026-10-09 07:48:07 +02:00
a979b5d11c
Rebuild: the coordinator spools replies without inflating them
Daan Meijer and Claude Opus 5.5
2026-10-09 07:40:06 +02:00
53bfea8d15
Rebuild spec: measurements after the second performance round
Daan Meijer and Claude Opus 5.5
2026-10-09 03:37:04 +02:00
69f1cc63b2
Rebuild: coordinator:run --shard=i/n splits prefetching over processes
Daan Meijer and Claude Opus 5.5
2026-10-09 03:36:24 +02:00
462da5c225
Rebuild: a stopping worker notices it on keep-alive frames too
Daan Meijer and Claude Opus 5.5
2026-10-09 03:26:20 +02:00
883d85e91d
Rebuild: the coordinator stops on SIGTERM
Daan Meijer and Claude Opus 5.5
2026-10-09 03:20:11 +02:00
cae6e4f60d
Rebuild: prefetch stats a minute; the completeness check inflates off the event loop
Daan Meijer and Claude Opus 5.5
2026-10-09 03:14:49 +02:00
9dc0a27bb4
Rebuild: a lease never waits for prefetching; one-shot hashes
Daan Meijer and Claude Opus 5.5
2026-10-09 03:08:19 +02:00
168605148f
Rebuild: the worker joins stream pieces once per frame, gzips posts off its thread
Daan Meijer and Claude Opus 5.5
2026-10-09 03:06:17 +02:00
bd1c846f27
Rebuild: prefetching no longer waits for merging, nor stalls on finished ranges
Daan Meijer and Claude Opus 5.5
2026-10-09 03:04:00 +02:00
2b4f673ddb
Rebuild: index lookups for the merger's neighbours and the prefetcher's ranges
Daan Meijer and Claude Opus 5.5
2026-10-09 02:59:30 +02:00
583ea1ded6
Rebuild spec: status and measurements so far
Daan Meijer and Claude Opus 5.5
2026-10-09 02:53:36 +02:00
821de668d0
Rebuild: the merger stops when a pass over the file keys decides nothing
Daan Meijer and Claude Opus 5.5
2026-10-09 02:49:45 +02:00
73fc9aabdf
Rebuild: strings that outlive their chunk are copied (worker 1.14 → 0.66 GB)
Daan Meijer and Claude Opus 5.5
2026-10-09 02:36:34 +02:00
80c920f678
Rebuild: settling quiet releases
Daan Meijer and Claude Opus 5.5
2026-10-09 02:33:47 +02:00
fde1e6cc90
Rebuild: NZBs from the chunk store, and a catalog search
Daan Meijer and Claude Opus 5.5
2026-10-09 02:33:27 +02:00
e9592dfd33
Rebuild: chunk format v2 — nyuu and hex message-ids as bits
Daan Meijer and Claude Opus 5.5
2026-10-09 02:31:59 +02:00
31493a98be
Rebuild: merger rounds with one neighbour query, a cursor and a time budget; planning skips failing groups
Daan Meijer and Claude Opus 5.5
2026-10-09 02:25:45 +02:00
b862201991
Rebuild: the fragment merger
Daan Meijer and Claude Opus 5.5
2026-10-09 02:19:53 +02:00
2c14315f94
Rebuild: fragments only for files an edge cut; lone random articles are per-article
Daan Meijer and Claude Opus 5.5
2026-10-09 02:17:22 +02:00
5ba93f558c
Rebuild: XOVER chunks of 100k lines; batches by articles
Daan Meijer and Claude Opus 5.5
2026-10-09 02:09:04 +02:00
08c876b690
Rebuild: XOVER replies are joined once, when complete
Daan Meijer and Claude Opus 5.5
2026-10-09 01:36:28 +02:00
ead2790515
Rebuild: edge:node --processes (one worker per core, supervised)
Daan Meijer and Claude Opus 5.5
2026-10-09 01:32:37 +02:00
fa7d4f3394
Rebuild: end-to-end fixes — spool accounting, keep-alive frames, resume point, overlapped worker
Daan Meijer and Claude Opus 5.5
2026-10-09 01:31:19 +02:00
db882f0b48
Rebuild: the segment counter by a backwards scan, one cache from poster+subject to the file
Daan Meijer and Claude Opus 5.5
2026-10-09 01:16:04 +02:00
638cfa1dcd
Rebuild: a lean overview scan for the edge worker
Daan Meijer and Claude Opus 5.5
2026-10-09 01:13:41 +02:00
981bc01810
Rebuild: a file's identity is computed once per batch, not per article
Daan Meijer and Claude Opus 5.5
2026-10-09 01:08:58 +02:00
cd06f96f85
Rebuild: first end-to-end runs; prefetch per chunk, executable rule, wire fixes
Daan Meijer and Claude Opus 5.5
2026-10-09 01:07:11 +02:00
d5f168d8d2
Rebuild: coordinator (planner, NNTP pool, spool, prefetcher, leases, rule sets), node API and the edge:node worker
Daan Meijer and Claude Opus 5.5
2026-10-09 00:31:06 +02:00
20910647fa
Rebuild: the batch store (catalog upserts, write-once chunks, candidates, rule counts, fragments, resume point)
Daan Meijer and Claude Opus 5.5
2026-10-09 00:21:17 +02:00
0714b2c81a
Rebuild: a dedicated Postgres for development (its own database, as in production)
Daan Meijer and Claude Opus 5.5
2026-10-09 00:19:51 +02:00
2209921846
Rebuild: schema for groups, rule sets, ranges, release catalog, chunk store, fragments, probe queue, rule bookkeeping and nodes
Daan Meijer and Claude Opus 5.5
2026-10-09 00:19:00 +02:00
a18ead666b
Rebuild: edge:bench, and its result in the spec (TypeScript is fast enough)
Daan Meijer and Claude Opus 5.5
2026-10-09 00:17:26 +02:00
7dc316c987
Rebuild: the edge worker's range indexer
Daan Meijer and Claude Opus 5.5
2026-10-09 00:10:16 +02:00
d773c91ad0
Rebuild: phase 1 modules from the current app, ordered by pipeline stage
Daan Meijer and Claude Opus 5.5
2026-10-09 00:08:44 +02:00
96f0ade10e
Rebuild: fresh AdonisJS 7 API app on Postgres
Daan Meijer and Claude Opus 5.5
2026-10-09 00:05:24 +02:00
699395c51f
Rebuild spec: a worker streams its whole range in one request
Daan Meijer and Claude Opus 5.5
2026-10-09 00:01:03 +02:00
06f5d81fbb
Rebuild spec: d and e decided; coordinator prefetches ranges into a spool
Daan Meijer and Claude Opus 5.5
2026-10-08 23:57:17 +02:00
76ecef81af
Rebuild spec: a, b, c decided; group list with priority-based crosspost owners
Daan Meijer and Claude Opus 5.5
2026-10-08 23:54:20 +02:00
1931c7f66c
Rebuild spec: rules and recovery (counts per range and rule, rule versions, a sample)
Daan Meijer and Claude Opus 5.5
2026-10-08 23:39:36 +02:00
2d4f3f48cf
Rebuild spec: workers return probe candidates per release; a probe queue
Daan Meijer and Claude Opus 5.5
2026-10-08 23:31:18 +02:00
7c43abd230
files:move-details: short batches, so files deleted meanwhile don't fail the copy
Daan Meijer and Claude Opus 5.5
2026-10-08 23:08:31 +02:00
16e504a8fb
Rebuild: spec draft (header-first indexer)
Daan Meijer and Claude Opus 5.5
2026-10-08 19:37:26 +02:00
fad1716d3a
Content type and archive listing move from files to file_details
Daan Meijer and Claude Opus 5.5
2026-10-08 19:36:36 +02:00
1520a8dc35
Spots: a corrupt NZB is a result, not a reason to read the listing again
Daan Meijer and Claude Opus 5.5
2026-10-08 15:23:24 +02:00
