Corpus: stored XOVER chunks for the grouping sweeps
Ranges marked `capture` keep their XOVER chunks in CORPUS_DIR (corpus/<group>/<start>-<end>/, edge-bench's layout, range.json once complete; only chunks that inflate). The prefetcher reads a stored chunk instead of fetching it, so a reset re-indexes them without the news server; a chunk a worker reports corrupt goes from the corpus too. edge-bench gets --releases (release chunks without payload, with their first subjects) and ships in the edge image; deploy/corpus_sweep.py marks more ranges (capture), replays the corpus with the deployed build (replay) and reports suspects and the change since the previous build (report). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
d43603a43f
commit
9ea2669b2a
13 files changed
+389
-11
No files matched your search
@@ -47,8 +47,12 @@ pub struct ReleaseChunk {
|
||||
pub readable: bool,
|
||||
pub has_par2: bool,
|
||||
pub payload: Vec<u8>,
|
||||
/// The first files' subjects (at most SUBJECTS): edge-bench's --releases dump for the grouping sweeps.
|
||||
pub subjects: Vec<String>,
|
||||
}
|
||||
|
||||
pub const SUBJECTS: usize = 3;
|
||||
|
||||
pub struct Candidate {
|
||||
pub release_key: String,
|
||||
pub upload: usize,
|
||||
@@ -830,6 +834,7 @@ fn chunk_of(key: &str, files: &[Vec<Seg>], upload: usize) -> ReleaseChunk {
|
||||
readable: chunk_files.iter().any(|f| !is_random_file(f.name)),
|
||||
has_par2: chunk_files.iter().any(|f| names::is_par2(f.name)),
|
||||
payload: encode_chunk(&chunk_files),
|
||||
subjects: chunk_files.iter().take(SUBJECTS).map(|f| f.subject.to_string()).collect(),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user