edge-bench-rs: README with the findings and how to compare the outputs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed
1 parent
b1a6f02de6
commit
122a054eab
1 file changed
+190
@@ -0,0 +1,190 @@
|
||||
# edge-bench-rs: the edge worker's indexing in Rust
|
||||
|
||||
A measurement, not a product. It answers whether a Rust edge worker would be
|
||||
worth a second implementation of the TypeScript one (`edge:node`, REBUILD.md
|
||||
part 2). It ports the path that `edge:bench` replays over a captured range:
|
||||
|
||||
- inflate each XOVER chunk;
|
||||
- scan the overview lines (no object per line);
|
||||
- apply the owner rule from the Xref;
|
||||
- build segments (file key, release identity);
|
||||
- aggregate segments into files, and cut off idle ones as fragments,
|
||||
per-article or incomplete files;
|
||||
- every N chunks, judge the completed releases (rules: executables,
|
||||
readable names, garbage tools and signatures, per-article random
|
||||
subjects), encode their chunks (format v2) and pick probe candidates.
|
||||
|
||||
It does not port NNTP, the coordinator API, batch posting or anything on the
|
||||
server side.
|
||||
|
||||
## Findings (2026-10-09)
|
||||
|
||||
Same machine (the workstation, otherwise idle), same captures, 2M headers
|
||||
each:
|
||||
|
||||
| Capture | TypeScript (`edge:bench`) | Rust (`edge-bench-rs`) | Less CPU |
|
||||
|---|---|---|---|
|
||||
| a.b.boneless 2026 (160950000000–160951999999) | 11.1 CPU-s (~215k headers/s), 0.70 GB peak | 2.9 CPU-s (~690k/s), 0.24 GB | 3.8× |
|
||||
| a.b.boneless 2026 (160952000000–160953999999) | 11.8 CPU-s (~205k/s), 0.70 GB | 3.1 CPU-s (~640k/s), 0.24 GB | 3.8× |
|
||||
| a.b.ath 2013 (2549992990–2551992989) | 7.5 CPU-s (~320k/s), 0.74 GB | 2.15 CPU-s (~930k/s), 0.20 GB | 3.5× |
|
||||
|
||||
**Comparing the numbers:**
|
||||
- **CPU seconds are the fair comparison.** `edge:bench`'s headers/s leaves
|
||||
out the time Node's GC threads use; the Rust binary is single-threaded, so
|
||||
its wall time is its CPU time.
|
||||
- **Homelab's cores** (Xeon E5-1650 v4) run the TypeScript ~10% slower than
|
||||
the workstation's.
|
||||
|
||||
**The Rust optimizations,** measured on boneless 2026 (all checked identical
|
||||
after each step):
|
||||
|
||||
| Step | Headers/s |
|
||||
|---|---|
|
||||
| Straight port: regex crate for the patterns, chrono for dates, zlib-rs, std allocator | 360k |
|
||||
| File stems and subject remainders by hand: their 12 + 5 regexes were most of a new file's description (333k per boneless range, many of them per-article random subjects) | |
|
||||
| mimalloc for the many small strings | 495k |
|
||||
| Upload tools, release-name checks and the common Date formats by hand | 500k |
|
||||
| libdeflate: inflates a chunk whole in half of zlib's time (1.09 → 0.51 s per range) | |
|
||||
| memchr (SIMD) for line ends and tabs | ~700k |
|
||||
| Fewer allocations in `describe` (hashing the parts without joining them): no measurable gain, reverted | |
|
||||
|
||||
Rows without a figure weren't measured on their own; each figure covers the
|
||||
rows above it.
|
||||
|
||||
**What's left** is spread out: scanning and building segments, two sha1s per
|
||||
new file, aggregation, cut-off and chunk encoding. `perf` isn't available on
|
||||
the workstation (`perf_event_paranoid` = 3), so these come from coarse
|
||||
timers. The binary prints them as "of which aggregate …, evict+cutoff …,
|
||||
judge+encode …" on stderr.
|
||||
|
||||
**What it means for the full target** (~1.25M articles/s with 50 NNTP
|
||||
connections):
|
||||
- **Worker cores:** ~1.5–2 instead of ~6–7. Spreading workers over other
|
||||
machines (batcave, poweredge-kantoor) becomes mostly unnecessary.
|
||||
- **Total speed:** unchanged. The account's NNTP connections set it (~30k
|
||||
articles/s each), not the worker.
|
||||
- **The cost:** a second implementation of `lib/headers`, `lib/release`,
|
||||
`lib/rules`, `lib/files` (aggregation, volumes) and the chunk format.
|
||||
Every rule change has to land in both and be checked identical (below)
|
||||
for as long as both exist.
|
||||
|
||||
## Running it
|
||||
|
||||
You need a captured range. `edge:bench --capture` saves a range's compressed
|
||||
XOVER chunks once (it needs `NNTP_*` in the environment and one NNTP
|
||||
connection):
|
||||
|
||||
```sh
|
||||
node ace edge:bench --capture --dir /path/to/capture --group alt.binaries.boneless --start 160950000000
|
||||
# 2M articles in chunks of 50k (--articles, --chunk), plus range.json
|
||||
```
|
||||
|
||||
Benchmark both on it:
|
||||
|
||||
```sh
|
||||
node ace edge:bench --dir /path/to/capture # TypeScript: headers/s, CPU, inflate/index, peak RSS
|
||||
cd tools/edge-bench-rs && cargo build --release
|
||||
./target/release/edge-bench-rs /path/to/capture # Rust: the same summary
|
||||
```
|
||||
|
||||
Both take results every 10 chunks (`--batch-chunks`) and give a file up after
|
||||
5 idle chunks (`--idle`). Keep these equal on both sides when comparing.
|
||||
The rules are `DEFAULT_RULES` with only the range's group indexed and no
|
||||
dropped signatures, as in `edge:bench`.
|
||||
|
||||
The two "kept: …" summary lines should already agree: release chunks,
|
||||
files, segments, payload size, candidates, fragments and samples. The
|
||||
summary can hide compensating differences, so compare the outputs
|
||||
themselves.
|
||||
|
||||
## Comparing the output
|
||||
|
||||
```sh
|
||||
./target/release/edge-bench-rs /path/to/capture --dump /tmp/out.jsonl
|
||||
cd ../.. && node --import=@poppinss/ts-exec tools/edge-bench-rs/check.ts /path/to/capture /tmp/out.jsonl
|
||||
```
|
||||
|
||||
`--dump` writes every output of every batch as JSON lines:
|
||||
|
||||
| Line `t` | Contents |
|
||||
|---|---|
|
||||
| `b` | batch boundaries: from, next, headers |
|
||||
| `r` | releases: every ReleaseChunk field, the payload in base64 |
|
||||
| `s` | samples: dropped releases kept in full, with their rule |
|
||||
| `c` | probe candidates |
|
||||
| `n` | rule counts |
|
||||
| `f` | fragments |
|
||||
|
||||
`check.ts` runs the TypeScript `RangeIndexer` (from `lib/edge`, not a copy)
|
||||
on the same capture with the same batching. It matches items by batch and
|
||||
key: release key, candidate (release, kind, message-id), rule and key,
|
||||
fragment file key. Then it compares them field by field:
|
||||
|
||||
```
|
||||
same batches: 4 TypeScript, 4 Rust
|
||||
same releases: 1055 TypeScript, 1055 Rust
|
||||
same samples: 2 TypeScript, 2 Rust
|
||||
same candidates: 1090 TypeScript, 1090 Rust
|
||||
same rule counts: 34 TypeScript, 34 Rust
|
||||
same fragments: 130 TypeScript, 130 Rust
|
||||
payload bytes: 13728351 TypeScript, 13727247 Rust
|
||||
```
|
||||
|
||||
It exits 0 only when everything matches. Otherwise each kind says how many
|
||||
items are only on one side or differ, and shows the first differing fields.
|
||||
|
||||
- **Payloads are compared decoded.** Both sides go through `decodeChunk`
|
||||
from `lib/chunks`, and their files are compared, ordered by name and
|
||||
subject. Node's built-in zstd and the zstd crate frame the same data a
|
||||
little differently, so the compressed sizes differ by about 0.1%. The
|
||||
checker prints them, but they don't count as a difference. Decoding the
|
||||
Rust payloads with the TypeScript decoder also checks the Rust encoder.
|
||||
- **Order doesn't count.** Order within a batch isn't compared (the two
|
||||
hash maps iterate differently). Grouping into batches, the order of a
|
||||
release's files and the order of a fragment's segments are compared,
|
||||
because they're part of the values.
|
||||
|
||||
Run it after any change on either side. All three captures above are
|
||||
identical as of c7b4ebe.
|
||||
|
||||
## JavaScript semantics the port keeps (`src/js.rs`, `src/names.rs`)
|
||||
|
||||
Differences found with `check.ts` or the tests, kept as JavaScript has
|
||||
them:
|
||||
|
||||
- **Regexes without the `u` flag:** `\d` is [0-9], `\b` an ASCII word
|
||||
boundary, `/i` ASCII-only. They run as `(?-u)` byte regexes, or by hand.
|
||||
The hand-written versions (file stems, subject remainders, upload tools,
|
||||
release-name and file-counter checks) are tested against the regex
|
||||
versions, which are kept for that (`*_re`).
|
||||
- **JS `\s` and `trim()`** include Unicode spaces (NBSP, U+2000–200A,
|
||||
U+3000, BOM…); `split(/\s+/)` keeps leading and trailing empty words.
|
||||
- **`lastIndexOf(c, position)`** finds a match at or before the position
|
||||
(the owner rule, the counter and filename scans). **`.length`** is in
|
||||
UTF-16 units (`len16`). **`parseInt`** reads leading digits.
|
||||
- **`Date.parse`:** chrono follows RFC 5322 and rejects "UTC", which
|
||||
`Date.parse` takes. It ends about a quarter of boneless's Date headers, so
|
||||
it's mapped to GMT; check.ts found this. The common formats are parsed by
|
||||
hand:
|
||||
|
||||
```
|
||||
[Day, ]D[D] Mon YY[YY] HH:MM:SS (GMT|UTC|UT|±hhmm)
|
||||
```
|
||||
|
||||
They ignore the weekday as `Date.parse` does (chrono rejects a weekday
|
||||
that doesn't fit). Anything else goes to chrono. Rare forms `Date.parse`
|
||||
takes but chrono doesn't would show up in check.ts as a `postedAt`
|
||||
difference.
|
||||
- **Invalid UTF-8** becomes U+FFFD on both sides (`toString('utf8')`,
|
||||
`from_utf8_lossy`).
|
||||
|
||||
## Files
|
||||
|
||||
| File | Contents |
|
||||
|---|---|
|
||||
| `src/main.rs` | arguments, the replay loop, timing, `--dump` |
|
||||
| `src/indexer.rs` | scan.ts (SegmentScanner, scanOverviewText), aggregator.ts, range_indexer.ts |
|
||||
| `src/names.rs` | lib/release (identity, random), lib/files/volumes, lib/rules (upload tools, judge) |
|
||||
| `src/codec.rs` | lib/chunks encodeChunk, format v2 |
|
||||
| `src/js.rs` | JavaScript semantics, sha1 |
|
||||
| `check.ts` | the comparison |
|
||||
Reference in new issue
Block a user