Spotnet import, header backfill, staging and counter grouping, faster search

Spotnet (CONCEPT D29):
- Index free.pt fully and import spots with their descriptions, categories
  and NZBs; a spot renames its release, since Spotnet names are more
  trustworthy than names derived from subjects.
- Find passwords in spot descriptions too, and tell "needs a password we
  have" from "needs a password we don't have".

Content inspection:
- List the contents of 7z (.7z.001) archives, SFV sets, nicovideo uploads
  and other archive-like files; mark yEncBin .bin containers as encrypted.
- Take real names from yEnc headers for randomly named files, and merge
  uploads that were split across posts.

Storage (D31, D32):
- Store segment lists zstd-compressed behind the FileStore accessor.
- group:backfill indexes older headers at low priority and pauses when
  the NVMe or HDD part of the database gets too large.

Grouping (D35, D36):
- Lone header segments wait in staged_segments until a second segment of
  their file arrives, which drops the per-article junk binaries.
- Counter grouping joins random-subject uploads by poster, [i/M] counter,
  part count and day.

Search and maintenance (D33, D34):
- A trigram-indexed search_text column, smaller indexes, estimated stats.
- Passes that clean up nonsense names; weekly index maintenance; GC budgets
  per kind.
- Web UI: grey out stale results while a search runs and cancel superseded
  requests. Count only up to 1,000 matches for the page, which made filter
  clicks take seconds (Newznab keeps the 10,000 cap).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

commands/GroupAdd.ts

@@ -33,6 +33,9 @@ export default class GroupAdd extends BaseCommand {
   @flags.boolean({ description: 'With --backfill: also for groups indexed before, skipping what was missed since' })
   declare reset: boolean
 
+  @flags.boolean({ description: "Start at the server's oldest article: the group's full history (also for groups indexed before)" })
+  declare fromStart: boolean
+
   public async run() {
     for (const name of this.names) {
       if (!/^[a-z0-9][a-z0-9.+_-]*$/i.test(name)) {
@@ -42,26 +45,30 @@ export default class GroupAdd extends BaseCommand {
       const group = await Group.firstOrCreate({ name }, { name, active: !this.disable, fetchBodies: this.bodies })
       group.active = !this.disable
       if (this.bodies) group.fetchBodies = true
-      if (this.backfill !== undefined && (this.reset || BigInt(group.lastQueuedId ?? 0) === 0n)) {
-        // The scheduler continues after last_queued_id.
-        const last = await this.lastArticle(name)
+      if (this.fromStart) {
+        // The scheduler continues after last_queued_id; 0 would mean "new group" to it.
+        const { first } = await this.groupInfo(name)
+        group.lastQueuedId = first > 1n ? first - 1n : 1n
+        group.lastIndexedId = group.lastQueuedId
+      } else if (this.backfill !== undefined && (this.reset || BigInt(group.lastQueuedId ?? 0) === 0n)) {
+        const { last } = await this.groupInfo(name)
         group.lastQueuedId = (last > BigInt(this.backfill) ? last - BigInt(this.backfill) : 0n)
         group.lastIndexedId = group.lastQueuedId
       }
       await group.save()
       this.logger.info(
         `${name}: ${group.active ? 'active' : 'inactive'}, bodies ${group.fetchBodies ? 'on' : 'off'}` +
-          (this.backfill !== undefined ? `, starting after article ${group.lastQueuedId}` : '')
+          (this.backfill !== undefined || this.fromStart ? `, starting after article ${group.lastQueuedId}` : '')
       )
     }
-    if (this.backfill !== undefined) await NntpService.shutdown()
+    if (this.backfill !== undefined || this.fromStart) await NntpService.shutdown()
   }
 
-  private async lastArticle(name: string): Promise<bigint> {
+  private async groupInfo(name: string) {
     const conn = await NntpService.acquire()
     let failed = false
     try {
-      return parseGroupResponse((await conn.group(name)).resp).last
+      return parseGroupResponse((await conn.group(name)).resp)
     } catch (error) {
       failed = true
       throw error