Files
usenet-indexer/public/index.html
T
Daan MeijerandClaude Opus 5.5 be8a3d4580 Spotnet import, header backfill, staging and counter grouping, faster search
Spotnet (CONCEPT D29):
- Index free.pt fully and import spots with their descriptions, categories
  and NZBs; a spot renames its release, since Spotnet names are more
  trustworthy than names derived from subjects.
- Find passwords in spot descriptions too, and tell "needs a password we
  have" from "needs a password we don't have".

Content inspection:
- List the contents of 7z (.7z.001) archives, SFV sets, nicovideo uploads
  and other archive-like files; mark yEncBin .bin containers as encrypted.
- Take real names from yEnc headers for randomly named files, and merge
  uploads that were split across posts.

Storage (D31, D32):
- Store segment lists zstd-compressed behind the FileStore accessor.
- group:backfill indexes older headers at low priority and pauses when
  the NVMe or HDD part of the database gets too large.

Grouping (D35, D36):
- Lone header segments wait in staged_segments until a second segment of
  their file arrives, which drops the per-article junk binaries.
- Counter grouping joins random-subject uploads by poster, [i/M] counter,
  part count and day.

Search and maintenance (D33, D34):
- A trigram-indexed search_text column, smaller indexes, estimated stats.
- Passes that clean up nonsense names; weekly index maintenance; GC budgets
  per kind.
- Web UI: grey out stale results while a search runs and cancel superseded
  requests. Count only up to 1,000 matches for the page, which made filter
  clicks take seconds (Newznab keeps the 10,000 cap).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 19:59:23 +02:00

108 lines
4.9 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Usenet Indexer</title>
<link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' rx='7' fill='%232f6fed'/%3E%3Cpath d='M9 10h14M9 16h14M9 22h9' stroke='white' stroke-width='3' stroke-linecap='round'/%3E%3C/svg%3E" />
<link rel="stylesheet" href="/app.css" />
</head>
<body>
<header class="topbar">
<a class="brand" href="/" aria-label="Usenet Indexer, home">
<span class="brand-mark" aria-hidden="true"></span>
<span>Usenet Indexer</span>
</a>
<dl class="stats" id="stats" aria-label="Index statistics"></dl>
</header>
<main class="layout">
<section class="search" aria-label="Search">
<div class="search-row">
<label class="search-box">
<svg aria-hidden="true" viewBox="0 0 24 24"><circle cx="11" cy="11" r="7" /><path d="m20 20-3.5-3.5" /></svg>
<input id="q" type="search" autocomplete="off" spellcheck="false"
placeholder="Search releases, file names and archive contents…" aria-label="Search" />
<kbd aria-hidden="true">/</kbd>
</label>
<label class="select">
<span class="visually-hidden">Sort</span>
<select id="sort">
<option value="newest">Newest</option>
<option value="oldest">Oldest</option>
<option value="largest">Largest</option>
<option value="files">Most files</option>
<option value="name">Name</option>
</select>
</label>
</div>
<div class="filters" role="group" aria-label="Filters">
<button class="chip" type="button" data-filter="readable" aria-pressed="false">Readable names</button>
<button class="chip" type="button" data-filter="complete" aria-pressed="false">Complete</button>
<button class="chip" type="button" data-filter="hide_encrypted" aria-pressed="false" title="Hide releases that need a password we don't know">Hide password needed</button>
<button class="chip" type="button" data-filter="par2_only" aria-pressed="false" title="Also show releases that are only par2 files whose data never arrived">Show par2-only</button>
<label class="select select-small">
<span class="visually-hidden">Recognised by</span>
<select id="source">
<option value="">Any source</option>
<option value="stem">Filename</option>
<option value="subject">Subject</option>
<option value="post">Post id</option>
<option value="par2">par2 contents</option>
<option value="rar">RAR headers</option>
<option value="weak-stem">Obfuscated filename</option>
<option value="nzb">Posted NZB</option>
<option value="spot">Spotnet</option>
</select>
</label>
<span class="chip chip-active" id="poster-filter" hidden>
<span id="poster-filter-label"></span>
<button type="button" id="poster-clear" aria-label="Remove poster filter">×</button>
</span>
</div>
</section>
<section class="results" aria-live="polite">
<p class="summary" id="summary"></p>
<ol class="release-list" id="list"></ol>
<div class="more">
<button type="button" class="button" id="more" hidden>Load more</button>
</div>
<p class="empty" id="empty" hidden>No releases match. Try fewer words or turn off a filter.</p>
</section>
</main>
<div class="scrim" id="scrim" hidden></div>
<aside class="drawer" id="drawer" aria-labelledby="detail-name" hidden>
<div class="drawer-head">
<div class="drawer-title">
<p class="eyebrow" id="detail-source"></p>
<h2 id="detail-name"></h2>
</div>
<button type="button" class="icon-button" id="close" aria-label="Close details">
<svg aria-hidden="true" viewBox="0 0 24 24"><path d="M6 6l12 12M18 6 6 18" /></svg>
</button>
</div>
<div class="drawer-body" id="detail"></div>
</aside>
<template id="row-template">
<li class="release">
<button type="button" class="release-main">
<span class="release-name"></span>
<span class="release-meta"></span>
</button>
<div class="release-side">
<span class="badges"></span>
<span class="release-size"></span>
<a class="icon-button nzb" title="Download NZB" aria-label="Download NZB">
<svg aria-hidden="true" viewBox="0 0 24 24"><path d="M12 4v11m0 0 4-4m-4 4-4-4M5 19h14" /></svg>
</a>
</div>
</li>
</template>
<script src="/app.js" type="module"></script>
</body>
</html>