The replay harness: a group's corpus through one carried window into a scratch catalog
Test and deploy / test (push) Successful in 48s
Test and deploy / edge (push) Successful in 1m43s
Test and deploy / deploy (push) Skipped

replay:corpus runs edge-bench --group over a group's complete corpus ranges
in article order (one window, carried from range to range, finished after
the last range; edge/src/lease.rs holds the lease JSON the worker and the
bench share) and stores every batch through BatchStore as the worker
would, then reports releases finished, provisional, complete and the bytes
per header of the NVMe and HDD tables. Refuses a database not named
*_replay or *_scratch. The app image carries edge-bench; compose gets a
replay profile for homelab.

Design amended (REBUILD.md 1, 2, 5): the window is carried by one worker a
group, no block leases and no block merger; provisional writes stay for
the lease ends.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Fable 5.1 committed 2026-10-26 00:27:09 +01:00
1 parent cf51db047f
commit 72e973d006
10 files changed
+735 -151

No files matched your search

+1 -58
View File
@@ -9,7 +9,7 @@
//! (the next worker resumes at the last stored batch).
use edge::indexer::{Batch, RangeIndexer};
use edge::names::Rules;
use edge::lease::LeaseJson;
use edge::wire;
use flate2::write::GzEncoder;
use serde::Deserialize;
@@ -34,63 +34,6 @@ fn log(level: &str, message: &str) {
println!("[ {level} ] {message}");
}
#[derive(Deserialize)]
struct LeaseJson {
range: RangeJson,
rules: RulesJson,
}
#[derive(Deserialize)]
struct RangeJson {
id: i64,
group: String,
start: i64,
end: i64,
next: i64,
}
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
struct RulesJson {
version: i64,
priorities: HashMap<String, u32>,
#[serde(default)]
dropped_signatures: Vec<String>,
#[serde(default)]
dropped_tools: Option<Vec<String>>,
#[serde(default)]
per_article_max_parts: Option<u32>,
#[serde(default)]
sample_percent: Option<f64>,
#[serde(default)]
drop_executables: Option<bool>,
#[serde(default)]
dropped_posters: Option<Vec<String>>,
}
impl RulesJson {
fn rules(self) -> Rules {
let mut rules = Rules::defaults(self.priorities.into_iter().collect());
rules.version = self.version;
rules.dropped_signatures = self.dropped_signatures.into_iter().collect();
if let Some(tools) = self.dropped_tools {
rules.dropped_tools = tools.into_iter().collect();
}
if let Some(n) = self.per_article_max_parts {
rules.per_article_max_parts = n;
}
if let Some(p) = self.sample_percent {
rules.sample_percent = p;
}
if let Some(e) = self.drop_executables {
rules.drop_executables = e;
}
if let Some(posters) = self.dropped_posters {
rules.dropped_posters = posters;
}
rules
}
}
#[derive(Deserialize)]
struct StoredJson {