Rebuild: first end-to-end runs; prefetch per chunk, executable rule, wire fixes
Test and deploy / test (push) Successful in 20s
Test and deploy / deploy (push) Skipped

End to end on the workstation (coordinator:run, the API, edge:node, real
NNTP): a.b.e-book.french and a.b.cd.image.playstation2.dvdiso ranges.

- Prefetcher: every connection takes the next missing chunk (leased
  ranges first, in order); one connection per range (~26k lines/s) kept
  the worker waiting
- executable rule: releases of bare executables (malware floods named
  after films and software) are dropped; they were 92% of the catalog rows
  of the test ranges, 21% of the segments
- fragments per file (details once) and gzipped batch bodies; the JSON
  limit (1 MB by default) raised to 256 MB, empty strings kept (a rule
  count's key is '')
- release_chunks.batch_from is a bigint (boneless article numbers)
- tests get their own database; edge:bench --store times BatchStore:
  1.3 s for a boneless range (1.24 ms per release chunk)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Daan MeijerandClaude Opus 5.5 committed 2026-10-09 01:07:11 +02:00
1 parent d5f168d8d2
commit cd06f96f85
15 files changed
+201 -99

No files matched your search

+6 -4
View File
@@ -33,18 +33,20 @@ export class AuthAccessTokenSchema extends BaseModel {
}
export class FragmentSchema extends BaseModel {
static $columns = ['articleNumber', 'fileKey', 'id', 'rangeId', 'segment'] as const
static $columns = ['articleNumber', 'fileKey', 'fragment', 'id', 'rangeId', 'segments'] as const
$columns = FragmentSchema.$columns
@column()
declare articleNumber: bigint | number
@column()
declare fileKey: string
@column()
declare fragment: any
@column({ isPrimary: true })
declare id: bigint | number
@column()
declare rangeId: bigint | number
@column()
declare segment: any
declare segments: number
}
export class GroupSchema extends BaseModel {
@@ -165,10 +167,10 @@ export class RangeSchema extends BaseModel {
}
export class ReleaseChunkSchema extends BaseModel {
static $columns = ['batchSeq', 'createdAt', 'files', 'id', 'payload', 'rangeId', 'releaseId', 'segments'] as const
static $columns = ['batchFrom', 'createdAt', 'files', 'id', 'payload', 'rangeId', 'releaseId', 'segments'] as const
$columns = ReleaseChunkSchema.$columns
@column()
declare batchSeq: number
declare batchFrom: bigint | number
@column.dateTime({ autoCreate: true })
declare createdAt: DateTime
@column()