Add backup-first duplicate repair

docs/design/diagnostic-drilldowns.md

@@ -0,0 +1,35 @@
+# Diagnostic Drill-Downs
+
+## Problem
+
+Aggregate health numbers are useful, but DJs need to understand what is behind
+each number before they can trust it. A count like "60 missing tracks" should be
+clickable enough to answer: which tracks, which saved paths, and why did Serato
+Doctor count them?
+
+## Architecture
+
+The health engine remains responsible for aggregate scoring. The web layer adds
+a separate `details` payload beside the existing health report so the UI can show
+examples without changing the core score model.
+
+The drill-down payload is intentionally capped. Serato Doctor should explain the
+finding quickly in the local browser, not dump an entire user library into the
+page.
+
+## Edge Cases
+
+- Smart/dynamic crate references stay excluded from old-reference scoring.
+- The same missing filename can appear more than once in Serato's database.
+- Duplicate filename and cloud-conflict groups are informational only.
+- Suggested matches remain read-only evidence and must never trigger repair.
+- Empty drill-downs should feel reassuring, not broken.
+
+## Testing
+
+- Web analysis should include detail sections for old crate references,
+  suggested matches, unused tracks, duplicates, cloud conflicts, broken
+  symlinks, and missing Serato database tracks.
+- Static assets should expose clickable diagnostic hooks.
+- Browser behavior should be verified manually when the local browser policy
+  allows access to the development server.

docs/design/duplicate-repair.md

@@ -0,0 +1,39 @@
+# Duplicate repair
+
+## Problem
+
+Duplicate and cloud-conflict files waste space, but deleting either path can
+unmap tracks in crates or Serato's database. DJs need to choose the authoritative
+copy and understand every change before it happens.
+
+## Design
+
+The web interface requires an analysis, an explicit keeper selection, and a
+dry-run preview. Applying the plan first creates a timestamped backup beneath
+`_Serato_/.serato-doctor-backups/`. The snapshot contains every replaced audio
+file, loaded crate/smart-crate metadata, database V2, and a JSON restore manifest.
+
+The non-kept audio path is then replaced with a symbolic link to the keeper.
+This removes the extra audio payload while preserving every existing saved path.
+Crate files and database V2 are never rewritten. The UI exposes immediate
+restore using the manifest.
+
+Users may retain all backups or set a positive rotation limit. Rotation occurs
+only after a repair succeeds.
+
+## Edge cases
+
+- The duplicate group is rescanned and validated immediately before preview and
+  apply.
+- Existing symlinks cannot be selected as disposable duplicate files.
+- A partial failure restores already-changed files before reporting the error.
+- Restore refuses to overwrite a real file.
+- Healthy symlink aliases are excluded from future duplicate counts.
+
+## Tests
+
+- Backup creation includes audio and Serato metadata.
+- The old path resolves to the selected keeper after repair.
+- Restore returns the original file contents.
+- Invalid keeper choices are rejected.
+- Limited and unlimited retention behave deterministically.

docs/design/plain-language-diagnostics.md

@@ -0,0 +1,25 @@
+# Plain-language Diagnostics
+
+## Problem
+
+“Broken references” combined crate occurrences with Serato's own missing-track
+concept. DJs naturally compared that number with orange/unmapped tracks in Serato,
+even though the two counts describe different layers.
+
+## Language
+
+- **Missing tracks in Serato** means database entries whose saved file location no
+  longer exists. This corresponds most closely to orange or unmapped tracks.
+- **Old crate references** means saved appearances in regular crates whose exact
+  filename was not found in the selected music folder. One track can appear in
+  several crates, so both appearances and unique filenames are shown.
+
+Every health card and diagnostic row has an accessible information button. Hover
+shows its explanation on desktop; click or tap keeps it open; Escape or clicking
+elsewhere closes it. Explanations describe uncertainty and avoid implying repair.
+
+## Verification
+
+Health tests verify database missing-path and unique-filename counts. Browser tests
+cover the two separate metrics, hover/click explanations, keyboard dismissal, and
+mobile layout.

samples/small-library/generate.py

@@ -60,7 +60,7 @@ def build_sample(output: Optional[Path] = None) -> Path:
     ]
     for relative_path in manifest["tracks"]:
         fields = database_record(
-            b"pfil", f"{SERATO_PATH_PREFIX}{relative_path}".encode("utf-16-be")
+            b"pfil", str(music_root / relative_path).encode("utf-16-be")
         )
         database_records.append(database_record(b"otrk", fields))
     (serato_root / "database V2").write_bytes(b"".join(database_records))

serato_doctor/cli.py

@@ -106,6 +106,11 @@ def main():
         print(f"Database Entries: {health.database_entries}")
         print(f"Database / Library Matches: {health.database_library_matches}")
         print(f"Database Entries Outside Scan: {health.database_unmatched_entries}")
+        print(f"Missing Tracks in Serato: {health.database_missing_paths}")
+        print(
+            "Unique Missing Tracks in Serato: "
+            f"{health.database_missing_unique_filenames}"
+        )
         print(f"Tracks Missing From Database: {health.tracks_missing_from_database}")
         print(f"Duplicate Database Paths: {health.duplicate_database_paths}")
     else:

serato_doctor/duplicates.py

@@ -11,7 +11,9 @@ def find_duplicate_groups(
 ) -> Tuple[DuplicateGroup, ...]:
     """Find exact-name duplicates and suspected numeric conflict copies."""
 
-    track_tuple = tuple(tracks)
+    # Healthy symlinks preserve legacy Serato paths without consuming another
+    # copy of the audio, so they are aliases rather than duplicate files.
+    track_tuple = tuple(track for track in tracks if not track.path.is_symlink())
     by_name: DefaultDict[str, List[DiskTrack]] = defaultdict(list)
     by_conflict_name: DefaultDict[str, List[DiskTrack]] = defaultdict(list)
 

serato_doctor/health.py

@@ -52,6 +52,9 @@ def analyze_health(library: Library) -> HealthReport:
     database_path_counts = Counter(
         normalize(str(track.path)) for track in database_tracks
     )
+    missing_database_tracks = [
+        track for track in database_tracks if not track.path.exists()
+    ]
 
     return HealthReport(
         score=score,
@@ -99,9 +102,13 @@ def analyze_health(library: Library) -> HealthReport:
         database_unmatched_entries=sum(
             normalize(track.filename) not in library_names for track in database_tracks
         ),
+        database_missing_paths=len(missing_database_tracks),
+        database_missing_unique_filenames=len(
+            {normalize(track.filename) for track in missing_database_tracks}
+        ),
         tracks_missing_from_database=sum(
             normalize(track.filename) not in database_names for track in library.tracks
-        ),
+        ) if library.database else 0,
         duplicate_database_paths=sum(
             count - 1 for count in database_path_counts.values() if count > 1
         ),

serato_doctor/models/health.py

@@ -29,6 +29,8 @@ class HealthReport:
     database_entries: int
     database_library_matches: int
     database_unmatched_entries: int
+    database_missing_paths: int
+    database_missing_unique_filenames: int
     tracks_missing_from_database: int
     duplicate_database_paths: int
 

serato_doctor/repair.py

@@ -0,0 +1,160 @@
+"""Backup-first duplicate consolidation.
+
+Serato's binary metadata is deliberately not rewritten.  Removed duplicate files
+are replaced with symbolic links, so every existing path continues to resolve.
+"""
+
+import json
+import shutil
+from dataclasses import dataclass
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Iterable, Optional, Tuple
+from uuid import uuid4
+
+
+BACKUP_FOLDER = ".serato-doctor-backups"
+
+
+@dataclass(frozen=True)
+class DuplicateRepairPlan:
+    keeper: Path
+    replaced: Tuple[Path, ...]
+    metadata_files: Tuple[Path, ...]
+
+    @property
+    def changes(self) -> int:
+        return len(self.replaced)
+
+
+@dataclass(frozen=True)
+class RepairReceipt:
+    backup: Path
+    keeper: Path
+    replaced: Tuple[Path, ...]
+
+
+def plan_duplicate_repair(
+    keeper: Path, duplicates: Iterable[Path], serato_root: Path
+) -> DuplicateRepairPlan:
+    keeper = keeper.expanduser().resolve()
+    candidates = tuple(path.expanduser().resolve() for path in duplicates)
+    if keeper not in candidates:
+        raise ValueError("The file to keep must belong to this duplicate group")
+    if not keeper.is_file():
+        raise ValueError(f"The file to keep no longer exists: {keeper}")
+    replaced = tuple(path for path in candidates if path != keeper)
+    if not replaced:
+        raise ValueError("Choose a duplicate group containing at least two files")
+    if any(not path.is_file() or path.is_symlink() for path in replaced):
+        raise ValueError("A duplicate changed since the analysis; analyze again")
+    metadata = tuple(
+        sorted(
+            path
+            for path in serato_root.expanduser().resolve().rglob("*")
+            if path.is_file()
+            and BACKUP_FOLDER not in path.parts
+            and (
+                path.name == "database V2"
+                or path.suffix.casefold() in {".crate", ".scrate"}
+            )
+        )
+    )
+    return DuplicateRepairPlan(keeper, replaced, metadata)
+
+
+def apply_duplicate_repair(
+    plan: DuplicateRepairPlan,
+    serato_root: Path,
+    backup_limit: Optional[int] = 10,
+) -> RepairReceipt:
+    """Create a complete rollback snapshot, then replace extras with symlinks."""
+    if backup_limit is not None and backup_limit < 1:
+        raise ValueError("Backup limit must be at least 1, or unlimited")
+    backup_root = serato_root.expanduser().resolve() / BACKUP_FOLDER
+    backup = backup_root / _backup_name()
+    files_root = backup / "files"
+    metadata_root = backup / "serato-metadata"
+    backup.mkdir(parents=True)
+
+    entries = []
+    try:
+        for path in plan.replaced:
+            destination = files_root / _safe_backup_path(path)
+            destination.parent.mkdir(parents=True, exist_ok=True)
+            shutil.copy2(path, destination)
+            entries.append({"original": str(path), "backup": str(destination)})
+        for path in plan.metadata_files:
+            relative = path.relative_to(serato_root.expanduser().resolve())
+            destination = metadata_root / relative
+            destination.parent.mkdir(parents=True, exist_ok=True)
+            shutil.copy2(path, destination)
+        manifest = {
+            "created_at": datetime.now(timezone.utc).isoformat(),
+            "keeper": str(plan.keeper),
+            "replaced": entries,
+            "strategy": "symlink",
+        }
+        (backup / "manifest.json").write_text(
+            json.dumps(manifest, indent=2), encoding="utf-8"
+        )
+        for path in plan.replaced:
+            path.unlink()
+            path.symlink_to(plan.keeper)
+    except Exception:
+        _rollback_entries(entries)
+        shutil.rmtree(backup, ignore_errors=True)
+        raise
+
+    rotate_backups(backup_root, backup_limit)
+    return RepairReceipt(backup, plan.keeper, plan.replaced)
+
+
+def restore_backup(backup: Path) -> Tuple[Path, ...]:
+    manifest_path = backup.expanduser().resolve() / "manifest.json"
+    manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
+    restored = []
+    for entry in manifest["replaced"]:
+        original = Path(entry["original"])
+        saved = Path(entry["backup"])
+        if original.exists() and not original.is_symlink():
+            raise ValueError(f"Restore would overwrite a real file: {original}")
+        if original.is_symlink():
+            original.unlink()
+        original.parent.mkdir(parents=True, exist_ok=True)
+        shutil.copy2(saved, original)
+        restored.append(original)
+    return tuple(restored)
+
+
+def rotate_backups(backup_root: Path, limit: Optional[int]) -> None:
+    if limit is None or not backup_root.is_dir():
+        return
+    backups = sorted(
+        (path for path in backup_root.iterdir() if (path / "manifest.json").is_file()),
+        key=lambda path: path.name,
+        reverse=True,
+    )
+    for expired in backups[limit:]:
+        shutil.rmtree(expired)
+
+
+def _backup_name() -> str:
+    stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
+    return f"{stamp}-{uuid4().hex[:8]}"
+
+
+def _safe_backup_path(path: Path) -> Path:
+    anchorless = path.as_posix().lstrip("/").replace(":", "_")
+    return Path(anchorless)
+
+
+def _rollback_entries(entries: Iterable[dict]) -> None:
+    for entry in entries:
+        original = Path(entry["original"])
+        saved = Path(entry["backup"])
+        if original.is_symlink():
+            original.unlink()
+        if not original.exists() and saved.is_file():
+            original.parent.mkdir(parents=True, exist_ok=True)
+            shutil.copy2(saved, original)

serato_doctor/web.py

@@ -1,19 +1,31 @@
 import argparse
 import json
+from collections import Counter
 from dataclasses import asdict
 from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
 from importlib import resources
 from pathlib import Path
-from typing import Iterable
+from typing import Iterable, Optional
 
 from serato_doctor.crate_parser import load_library_crates
 from serato_doctor.database_parser import parse_database
+from serato_doctor.duplicates import find_duplicate_groups
 from serato_doctor.health import analyze_health
+from serato_doctor.matching import MatchingEngine, normalize
+from serato_doctor.models.crate import CrateKind
+from serato_doctor.models.duplicate import DuplicateKind
 from serato_doctor.models.library import Library
 from serato_doctor.scanner import scan_filesystem
+from serato_doctor.repair import (
+    BACKUP_FOLDER,
+    apply_duplicate_repair,
+    plan_duplicate_repair,
+    restore_backup,
+)
 
 
 MAX_REQUEST_BYTES = 64 * 1024
+DETAIL_LIMIT = 50
 STATIC_FILES = {
     "/": ("index.html", "text/html; charset=utf-8"),
     "/app.css": ("app.css", "text/css; charset=utf-8"),
@@ -21,6 +33,176 @@ STATIC_FILES = {
 }
 
 
+def _display_path(path: Path) -> str:
+    return str(path)
+
+
+def _first_reason(match) -> str:
+    for item in match.evidence:
+        if item.matched:
+            return item.explanation
+    return "Filename-related candidate"
+
+
+def diagnostic_details(library: Library, limit: int = DETAIL_LIMIT) -> dict:
+    dynamic_sources = {
+        crate.path for crate in library.crates if crate.kind is CrateKind.SMART
+    }
+    results = tuple(
+        result
+        for result in library.reconcile_by_filename()
+        if result.reference.source not in dynamic_sources
+    )
+    missing = [result for result in results if not result.exists_by_filename]
+
+    matcher = MatchingEngine(library.tracks)
+    suggested_matches = []
+    for result in missing:
+        candidates = matcher.candidates_for(result.reference)
+        if not candidates:
+            continue
+        best = candidates[0]
+        suggested_matches.append(
+            {
+                "filename": result.reference.filename,
+                "crate": _display_path(result.reference.source),
+                "saved_path": _display_path(result.reference.path),
+                "candidate": _display_path(best.track.path),
+                "score": f"{best.score_percent}%",
+                "reason": _first_reason(best),
+            }
+        )
+
+    duplicate_groups = find_duplicate_groups(library.tracks)
+    exact_duplicates = [
+        group
+        for group in duplicate_groups
+        if group.kind is DuplicateKind.EXACT_NAME
+    ]
+    cloud_conflicts = [
+        group
+        for group in duplicate_groups
+        if group.kind is DuplicateKind.CLOUD_CONFLICT
+    ]
+
+    database_tracks = library.database.tracks if library.database else ()
+    missing_database_tracks = [
+        track for track in database_tracks if not track.path.exists()
+    ]
+    database_filename_counts = Counter(
+        normalize(track.filename) for track in missing_database_tracks
+    )
+    referenced_names = {reference.filename for reference in library.references}
+    unused_tracks = [
+        track for track in library.tracks if track.filename not in referenced_names
+    ]
+
+    return {
+        "database_missing_tracks": {
+            "title": "Missing tracks in Serato",
+            "summary": (
+                "These are Serato database entries whose saved file location "
+                "does not currently exist on disk."
+            ),
+            "total": len(missing_database_tracks),
+            "items": [
+                {
+                    "filename": track.filename,
+                    "saved_path": _display_path(track.path),
+                    "artist": track.artist or "Unknown artist",
+                    "title": track.title or track.filename,
+                    "repeated_filename": database_filename_counts[
+                        normalize(track.filename)
+                    ]
+                    > 1,
+                }
+                for track in missing_database_tracks[:limit]
+            ],
+        },
+        "old_crate_references": {
+            "title": "Old crate references",
+            "summary": (
+                "These are regular crate appearances whose exact filename was "
+                "not found in the selected music folder."
+            ),
+            "total": len(missing),
+            "items": [
+                {
+                    "filename": result.reference.filename,
+                    "crate": _display_path(result.reference.source),
+                    "saved_path": _display_path(result.reference.path),
+                }
+                for result in missing[:limit]
+            ],
+        },
+        "suggested_matches": {
+            "title": "Suggested matches",
+            "summary": (
+                "These are read-only guesses where Serato Doctor found a "
+                "filename-related candidate on disk."
+            ),
+            "total": len(suggested_matches),
+            "items": suggested_matches[:limit],
+        },
+        "duplicate_filenames": {
+            "title": "Duplicate filenames",
+            "summary": (
+                "These groups contain different files with the same cleaned-up "
+                "filename. Review before making any decisions."
+            ),
+            "total": len(exact_duplicates),
+            "items": [
+                {
+                    "filename": group.display_name,
+                    "files": [_display_path(track.path) for track in group.tracks],
+                }
+                for group in exact_duplicates[:limit]
+            ],
+        },
+        "cloud_conflicts": {
+            "title": "Possible cloud conflicts",
+            "summary": (
+                "These filename families look like cloud sync conflict copies, "
+                "such as a duplicate ending in a number."
+            ),
+            "total": len(cloud_conflicts),
+            "items": [
+                {
+                    "filename": group.display_name,
+                    "files": [_display_path(track.path) for track in group.tracks],
+                }
+                for group in cloud_conflicts[:limit]
+            ],
+        },
+        "broken_symlinks": {
+            "title": "Broken shortcuts",
+            "summary": (
+                "These symbolic links point somewhere that no longer resolves."
+            ),
+            "total": len(library.broken_symlinks),
+            "items": [
+                {
+                    "path": _display_path(link.path),
+                    "target": _display_path(link.target) if link.target else "Unknown",
+                }
+                for link in library.broken_symlinks[:limit]
+            ],
+        },
+        "unused_tracks": {
+            "title": "Unused tracks",
+            "summary": (
+                "These scanned files were not referenced by any loaded crate. "
+                "That does not mean they should be deleted."
+            ),
+            "total": len(unused_tracks),
+            "items": [
+                {"filename": track.filename, "path": _display_path(track.path)}
+                for track in unused_tracks[:limit]
+            ],
+        },
+    }
+
+
 def analyze_paths(
     serato: Path, music: Path, reference_roots: Iterable[Path] = ()
 ) -> dict:
@@ -45,6 +227,45 @@ def analyze_paths(
     report = analyze_health(library)
     result = asdict(report)
     result["score_basis"] = report.score_basis
+    result["details"] = diagnostic_details(library)
+    return result
+
+
+def duplicate_repair(
+    serato: Path,
+    music: Path,
+    keeper: Path,
+    group_files: Iterable[Path],
+    backup_limit: Optional[int],
+    apply: bool = False,
+) -> dict:
+    """Validate a current duplicate group and preview or apply consolidation."""
+    serato = serato.expanduser().resolve()
+    music = music.expanduser().resolve()
+    keeper = keeper.expanduser().resolve()
+    requested = tuple(path.expanduser().resolve() for path in group_files)
+    if not serato.is_dir() or not music.is_dir():
+        raise ValueError("Analyze the library again before repairing duplicates")
+    tracks = scan_filesystem(music).tracks
+    groups = find_duplicate_groups(tracks)
+    valid_groups = [
+        {track.path.resolve() for track in group.tracks} for group in groups
+    ]
+    if set(requested) not in valid_groups:
+        raise ValueError("This duplicate group changed; analyze the library again")
+    plan = plan_duplicate_repair(keeper, requested, serato)
+    result = {
+        "keeper": str(plan.keeper),
+        "replaced": [str(path) for path in plan.replaced],
+        "metadata_backups": len(plan.metadata_files),
+        "strategy": "shortcut",
+        "database_v2_modified": False,
+    }
+    if apply:
+        receipt = apply_duplicate_repair(plan, serato, backup_limit)
+        result.update({"applied": True, "backup": str(receipt.backup)})
+    else:
+        result["applied"] = False
     return result
 
 
@@ -67,7 +288,13 @@ class SeratoDoctorHandler(BaseHTTPRequestHandler):
         self.wfile.write(content)
 
     def do_POST(self) -> None:
-        if self.path != "/api/analyze":
+        allowed = {
+            "/api/analyze",
+            "/api/duplicates/preview",
+            "/api/duplicates/apply",
+            "/api/backups/restore",
+        }
+        if self.path not in allowed:
             self._json_response(404, {"error": "Not found"})
             return
         try:
@@ -77,11 +304,37 @@ class SeratoDoctorHandler(BaseHTTPRequestHandler):
             payload = json.loads(self.rfile.read(length))
             if not isinstance(payload, dict):
                 raise ValueError("Request body must be a JSON object")
+            if self.path == "/api/analyze":
                 roots = [Path(value) for value in payload.get("reference_roots", [])]
                 result = analyze_paths(
                     Path(payload["serato"]), Path(payload["music"]), roots
                 )
-        except (KeyError, TypeError, json.JSONDecodeError, ValueError) as error:
+            elif self.path == "/api/backups/restore":
+                serato = Path(payload["serato"]).expanduser().resolve()
+                backup = Path(payload["backup"]).expanduser().resolve()
+                backup_root = (serato / BACKUP_FOLDER).resolve()
+                if backup.parent != backup_root:
+                    raise ValueError("That backup does not belong to this library")
+                restored = restore_backup(backup)
+                result = {"restored": [str(path) for path in restored]}
+            else:
+                raw_limit = payload.get("backup_limit", 10)
+                backup_limit = None if raw_limit is None else int(raw_limit)
+                result = duplicate_repair(
+                    Path(payload["serato"]),
+                    Path(payload["music"]),
+                    Path(payload["keeper"]),
+                    (Path(value) for value in payload["group_files"]),
+                    backup_limit,
+                    apply=self.path.endswith("/apply"),
+                )
+        except (
+            KeyError,
+            TypeError,
+            OSError,
+            json.JSONDecodeError,
+            ValueError,
+        ) as error:
             self._json_response(400, {"error": str(error)})
             return
         self._json_response(200, result)

serato_doctor/webui/app.js

@@ -2,12 +2,152 @@ const form = document.querySelector('#analysis-form');
 const button = document.querySelector('#analyze-button');
 const errorBox = document.querySelector('#error-message');
 const results = document.querySelector('#dashboard');
+const infoButtons = document.querySelectorAll('.info-button');
+const drillTriggers = document.querySelectorAll('[data-detail]');
+const drilldownTitle = document.querySelector('#drilldown-title');
+const drilldownCount = document.querySelector('#drilldown-count');
+const drilldownSummary = document.querySelector('#drilldown-summary');
+const drilldownList = document.querySelector('#drilldown-list');
+const repairPanel = document.querySelector('#duplicate-repair');
+const repairChoice = document.querySelector('#repair-choice');
+const repairPreview = document.querySelector('#repair-preview');
+const repairMessage = document.querySelector('#repair-message');
+const previewRepairButton = document.querySelector('#preview-repair');
+const applyRepairButton = document.querySelector('#apply-repair');
+const restoreRepairButton = document.querySelector('#restore-repair');
+let latestAnalysis = null;
+let selectedDuplicateGroup = null;
+let previewedRepair = null;
+let latestBackup = null;
 
 function expandHome(path) {
   return path.trim();
 }
 
+function escapeHtml(value) {
+  return String(value ?? '').replace(/[&<>"']/g, (character) => ({
+    '&': '&amp;',
+    '<': '&lt;',
+    '>': '&gt;',
+    '"': '&quot;',
+    "'": '&#039;',
+  }[character]));
+}
+
+function detailLines(item) {
+  if (item.files) {
+    return item.files.map((file) => `<li>${escapeHtml(file)}</li>`).join('');
+  }
+  const lines = [];
+  if (item.artist || item.title) lines.push(`${item.artist || 'Unknown artist'} — ${item.title || item.filename}`);
+  if (item.crate) lines.push(`Crate: ${item.crate}`);
+  if (item.saved_path) lines.push(`Saved path: ${item.saved_path}`);
+  if (item.candidate) lines.push(`Candidate: ${item.candidate}`);
+  if (item.path) lines.push(`File: ${item.path}`);
+  if (item.target) lines.push(`Target: ${item.target}`);
+  if (item.score || item.reason) lines.push(`${item.score || 'Match'} · ${item.reason || 'Candidate found'}`);
+  if (item.repeated_filename) lines.push('Same filename appears more than once in Serato’s missing list.');
+  return lines.map((line) => `<li>${escapeHtml(line)}</li>`).join('');
+}
+
+function renderDetail(key) {
+  const detail = latestAnalysis?.details?.[key];
+  if (!detail) return;
+
+  drillTriggers.forEach((trigger) => {
+    trigger.classList.toggle('selected', trigger.dataset.detail === key);
+  });
+  drilldownTitle.textContent = detail.title;
+  drilldownCount.textContent = `${detail.total ?? 0} found`;
+  drilldownSummary.textContent = detail.summary;
+
+  repairPanel.hidden = true;
+  selectedDuplicateGroup = null;
+  previewedRepair = null;
+  latestBackup = null;
+  restoreRepairButton.hidden = true;
+  if (!detail.items?.length) {
+    drilldownList.innerHTML = '<div class="empty-detail">Nothing to review here. Tiny victory parade, very tasteful.</div>';
+    return;
+  }
+
+  drilldownList.innerHTML = detail.items.map((item, index) => `
+    <article class="detail-item${item.files ? ' selectable-duplicate' : ''}" ${item.files ? `data-duplicate-index="${index}" role="button" tabindex="0"` : ''}>
+      <strong>${escapeHtml(item.filename || item.path || 'Untitled item')}</strong>
+      <ul>${detailLines(item)}</ul>
+      ${item.files ? '<small>Choose this group to review a safe cleanup →</small>' : ''}
+    </article>
+  `).join('');
+}
+
+function chooseDuplicate(detailKey, index) {
+  const group = latestAnalysis?.details?.[detailKey]?.items?.[index];
+  if (!group?.files) return;
+  selectedDuplicateGroup = group;
+  previewedRepair = null;
+  applyRepairButton.disabled = true;
+  repairPreview.hidden = true;
+  repairMessage.textContent = '';
+  repairChoice.innerHTML = group.files.map((file, fileIndex) => `
+    <label class="keeper-option"><input type="radio" name="keeper" value="${escapeHtml(file)}" ${fileIndex === 0 ? 'checked' : ''}><span><strong>${fileIndex === 0 ? 'Keep this file' : 'Keep instead'}</strong><small>${escapeHtml(file)}</small></span></label>
+  `).join('');
+  repairPanel.hidden = false;
+  repairPanel.scrollIntoView({behavior: 'smooth', block: 'start'});
+}
+
+function repairPayload() {
+  const keeper = document.querySelector('input[name="keeper"]:checked')?.value;
+  if (!selectedDuplicateGroup || !keeper) throw new Error('Choose a file to keep');
+  return {serato: expandHome(document.querySelector('#serato-path').value), music: expandHome(document.querySelector('#music-path').value), keeper, group_files: selectedDuplicateGroup.files, backup_limit: document.querySelector('#keep-all-backups').checked ? null : Number(document.querySelector('#backup-limit').value)};
+}
+
+async function requestRepair(endpoint) {
+  const response = await fetch(endpoint, {method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify(repairPayload())});
+  const data = await response.json();
+  if (!response.ok) throw new Error(data.error || 'Duplicate cleanup failed');
+  return data;
+}
+
+previewRepairButton.addEventListener('click', async () => {
+  repairMessage.textContent = 'Checking the plan…'; applyRepairButton.disabled = true;
+  try {
+    previewedRepair = await requestRepair('/api/duplicates/preview');
+    repairPreview.innerHTML = `<strong>Ready to protect and consolidate</strong><p>${previewedRepair.replaced.length} duplicate file(s) will be backed up, then replaced with shortcuts to the keeper. ${previewedRepair.metadata_backups} Serato metadata file(s) will also be copied into the rollback snapshot. Database V2 will not be changed.</p>`;
+    repairPreview.hidden = false; applyRepairButton.disabled = false;
+    repairMessage.textContent = 'Preview complete. Nothing has changed yet.';
+  } catch (error) { repairMessage.textContent = error.message; }
+});
+
+applyRepairButton.addEventListener('click', async () => {
+  if (!previewedRepair) return;
+  applyRepairButton.disabled = true; repairMessage.textContent = 'Creating the backup before making changes…';
+  try {
+    const result = await requestRepair('/api/duplicates/apply');
+    latestBackup = result.backup;
+    repairMessage.textContent = `Cleanup complete. Restore backup: ${result.backup}`;
+    previewRepairButton.disabled = true;
+    restoreRepairButton.hidden = false;
+  } catch (error) { repairMessage.textContent = error.message; applyRepairButton.disabled = false; }
+});
+
+restoreRepairButton.addEventListener('click', async () => {
+  if (!latestBackup) return;
+  restoreRepairButton.disabled = true; repairMessage.textContent = 'Restoring the duplicate files…';
+  try {
+    const response = await fetch('/api/backups/restore', {method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify({serato: expandHome(document.querySelector('#serato-path').value), backup: latestBackup})});
+    const result = await response.json();
+    if (!response.ok) throw new Error(result.error || 'Restore failed');
+    repairMessage.textContent = `Restore complete. ${result.restored.length} original file(s) returned.`;
+    restoreRepairButton.hidden = true;
+  } catch (error) { repairMessage.textContent = error.message; restoreRepairButton.disabled = false; }
+});
+
+repairChoice.addEventListener('change', () => { previewedRepair = null; applyRepairButton.disabled = true; repairPreview.hidden = true; repairMessage.textContent = 'Keeper changed. Preview the plan again.'; });
+drilldownList.addEventListener('click', (event) => { const item = event.target.closest('[data-duplicate-index]'); if (item) chooseDuplicate(document.querySelector('.drill-trigger.selected')?.dataset.detail, Number(item.dataset.duplicateIndex)); });
+drilldownList.addEventListener('keydown', (event) => { if (event.key !== 'Enter' && event.key !== ' ') return; const item = event.target.closest('[data-duplicate-index]'); if (item) { event.preventDefault(); chooseDuplicate(document.querySelector('.drill-trigger.selected')?.dataset.detail, Number(item.dataset.duplicateIndex)); } });
+
 function render(data) {
+  latestAnalysis = data;
   document.querySelectorAll('[data-field]').forEach((element) => {
     const value = data[element.dataset.field];
     element.textContent = value ?? '—';
@@ -20,10 +160,45 @@ function render(data) {
     : score >= 95 ? 'Looking excellent' : score >= 80 ? 'A few things need attention' : 'Review recommended';
   document.querySelector('#score-basis').textContent = data.score_basis;
   document.querySelector('#analysis-time').textContent = `Completed ${new Date().toLocaleTimeString([], {hour: '2-digit', minute: '2-digit'})}`;
+  renderDetail(data.database_missing_paths > 0 ? 'database_missing_tracks' : 'old_crate_references');
   results.hidden = false;
   results.scrollIntoView({behavior: 'smooth', block: 'start'});
 }
 
+function closeInfoButtons(except = null) {
+  infoButtons.forEach((infoButton) => {
+    if (infoButton !== except) {
+      infoButton.classList.remove('open');
+      infoButton.setAttribute('aria-expanded', 'false');
+    }
+  });
+}
+
+infoButtons.forEach((infoButton) => {
+  infoButton.addEventListener('click', (event) => {
+    event.stopPropagation();
+    const willOpen = !infoButton.classList.contains('open');
+    closeInfoButtons(infoButton);
+    infoButton.classList.toggle('open', willOpen);
+    infoButton.setAttribute('aria-expanded', String(willOpen));
+  });
+});
+
+drillTriggers.forEach((trigger) => {
+  trigger.addEventListener('click', () => renderDetail(trigger.dataset.detail));
+  trigger.addEventListener('keydown', (event) => {
+    if (event.key === 'Enter' || event.key === ' ') {
+      event.preventDefault();
+      renderDetail(trigger.dataset.detail);
+    }
+  });
+});
+
+document.addEventListener('click', () => closeInfoButtons());
+document.addEventListener('keydown', (event) => {
+  if (event.key === 'Escape') closeInfoButtons();
+});
+
 form.addEventListener('submit', async (event) => {
   event.preventDefault();
   errorBox.hidden = true;

serato_doctor/webui/index.html

@@ -23,7 +23,7 @@
       </nav>
       <div class="safety-card">
         <span class="safety-icon">✓</span>
-        <div><strong>Read-only mode</strong><p>Your library will not be modified.</p></div>
+        <div><strong>Protected mode</strong><p>Analysis is read-only. Repairs require a preview and backup.</p></div>
       </div>
       <div class="sidebar-foot">Local interface · v0.1</div>
     </aside>
@@ -59,28 +59,52 @@
         <div class="section-heading"><div><p class="eyebrow">Latest analysis</p><h2>Library health</h2></div><span id="analysis-time"></span></div>
         <div class="hero-grid">
           <article class="score-card panel">
+            <button class="info-button" type="button" aria-label="About library health" aria-expanded="false" data-info="Your health score is the percentage of saved, non-smart crate entries whose filenames were found in the selected music folder. Smart crates are left out because Serato rebuilds them from rules.">i</button>
             <div class="score-ring" id="score-ring"><div><strong id="health-score">—</strong><span>health</span></div></div>
             <div><p class="score-label">Reference integrity</p><h3 id="health-message">Ready to analyze</h3><p id="score-basis">We only score evidence we can defend.</p></div>
           </article>
           <div class="metrics-grid">
-            <article class="metric panel"><span>Tracks</span><strong data-field="disk_tracks">—</strong><small>audio files found</small></article>
-            <article class="metric panel warning"><span>Broken references</span><strong data-field="missing_references">—</strong><small>static crate entries</small></article>
-            <article class="metric panel"><span>Unused tracks</span><strong data-field="unused_tracks">—</strong><small>not referenced by crates</small></article>
-            <article class="metric panel"><span>Suggested matches</span><strong data-field="suggested_matches">—</strong><small>explainable candidates</small></article>
+            <article class="metric panel"><button class="info-button" type="button" aria-label="About tracks" aria-expanded="false" data-info="Audio files found inside the music folder you selected. This is the collection Serato Doctor compared with your crates and database.">i</button><span>Tracks scanned</span><strong data-field="disk_tracks">—</strong><small>audio files found</small></article>
+            <article class="metric panel warning drill-trigger" role="button" tabindex="0" data-detail="database_missing_tracks"><button class="info-button" type="button" aria-label="About missing tracks in Serato" aria-expanded="false" data-info="Tracks in Serato's database whose saved file location no longer exists. This should be close to the orange or unmapped track count you see in Serato.">i</button><span>Missing tracks in Serato</span><strong data-field="database_missing_paths">—</strong><small><b data-field="database_missing_unique_filenames">—</b> unique filenames · click for list</small></article>
+            <article class="metric panel drill-trigger" role="button" tabindex="0" data-detail="unused_tracks"><button class="info-button" type="button" aria-label="About unused tracks" aria-expanded="false" data-info="Files in the selected music folder whose filename is not used by any loaded crate. They may still be valid library tracks; this is informational, not a deletion recommendation.">i</button><span>Unused tracks</span><strong data-field="unused_tracks">—</strong><small>not referenced by crates · click for list</small></article>
+            <article class="metric panel drill-trigger" role="button" tabindex="0" data-detail="suggested_matches"><button class="info-button" type="button" aria-label="About suggested matches" aria-expanded="false" data-info="Missing crate entries with a filename-related candidate, such as an added OneDrive conflict number. Suggestions are evidence for review, never automatic repairs.">i</button><span>Suggested matches</span><strong data-field="suggested_matches">—</strong><small>explainable candidates · click for list</small></article>
           </div>
         </div>
 
         <div id="diagnostics" class="diagnostics panel">
           <div class="section-heading"><div><p class="eyebrow">Full picture</p><h2>Diagnostics</h2></div><span class="read-only-tag">No changes made</span></div>
           <div class="diagnostic-list">
-            <div><span class="diag-icon violet">◇</span><p><strong>Crates</strong><small><b data-field="static_crates">—</b> static · <b data-field="smart_crates">—</b> smart · <b data-field="smart_crate_containers">—</b> dynamic containers · <b data-field="dynamic_references_excluded">—</b> references excluded</small></p></div>
-            <div><span class="diag-icon amber">≋</span><p><strong>Duplicate filenames</strong><small><b data-field="duplicate_filename_groups">—</b> exact groups · <b data-field="duplicate_files">—</b> extra files</small></p></div>
-            <div><span class="diag-icon blue">⌁</span><p><strong>Cloud conflicts</strong><small><b data-field="suspected_cloud_conflict_groups">—</b> suspected groups · <b data-field="suspected_cloud_conflict_files">—</b> extra files</small></p></div>
-            <div><span class="diag-icon red">↗</span><p><strong>Broken symlinks</strong><small><b data-field="broken_symlinks">—</b> unresolved links</small></p></div>
-            <div><span class="diag-icon violet">▦</span><p><strong>Serato database V2</strong><small><b data-field="database_entries">—</b> entries · <b data-field="database_library_matches">—</b> match scanned filenames · <b data-field="tracks_missing_from_database">—</b> scanned tracks absent</small></p></div>
-            <div><span class="diag-icon amber">⌕</span><p><strong>Database review</strong><small><b data-field="database_unmatched_entries">—</b> entries outside this music scan · <b data-field="duplicate_database_paths">—</b> duplicate paths</small></p></div>
+            <div><span class="diag-icon violet">◇</span><p><strong>Crates</strong><small><b data-field="static_crates">—</b> regular · <b data-field="smart_crates">—</b> smart · <b data-field="smart_crate_containers">—</b> dynamic containers</small></p><button class="info-button" type="button" aria-label="About crates" aria-expanded="false" data-info="Regular crates are lists you maintain by hand. Smart crates are rebuilt by Serato from rules, so their generated references are not scored as broken.">i</button></div>
+            <div class="drill-trigger" role="button" tabindex="0" data-detail="duplicate_filenames"><span class="diag-icon amber">≋</span><p><strong>Duplicate filenames</strong><small><b data-field="duplicate_filename_groups">—</b> exact groups · <b data-field="duplicate_files">—</b> extra files · click for list</small></p><button class="info-button" type="button" aria-label="About duplicate filenames" aria-expanded="false" data-info="Different files with the same filename after case and Unicode cleanup. They need review, but matching names alone do not mean either file should be deleted.">i</button></div>
+            <div class="drill-trigger" role="button" tabindex="0" data-detail="cloud_conflicts"><span class="diag-icon blue">⌁</span><p><strong>Possible cloud conflicts</strong><small><b data-field="suspected_cloud_conflict_groups">—</b> groups · <b data-field="suspected_cloud_conflict_files">—</b> extra files · click for list</small></p><button class="info-button" type="button" aria-label="About cloud conflicts" aria-expanded="false" data-info="Filename families such as Track.mp3 and Track 2.mp3. OneDrive often creates these during sync conflicts, but numbered song titles can also be legitimate.">i</button></div>
+            <div class="drill-trigger" role="button" tabindex="0" data-detail="broken_symlinks"><span class="diag-icon red">↗</span><p><strong>Broken shortcuts</strong><small><b data-field="broken_symlinks">—</b> unresolved symbolic links · click for list</small></p><button class="info-button" type="button" aria-label="About broken shortcuts" aria-expanded="false" data-info="Shortcut-style symbolic links whose destination no longer exists. Serato Doctor reports them but never removes or recreates them automatically.">i</button></div>
+            <div class="drill-trigger" role="button" tabindex="0" data-detail="old_crate_references"><span class="diag-icon violet">▦</span><p><strong>Old crate references</strong><small><b data-field="missing_references">—</b> appearances · <b data-field="unique_missing_filenames">—</b> unique filenames · click for list</small></p><button class="info-button" type="button" aria-label="About old crate references" aria-expanded="false" data-info="Saved spots in regular crates whose exact filename was not found in the selected music folder. The same track can appear in several crates, so appearances are higher than unique filenames. These are separate from Serato's unmapped-track count.">i</button></div>
+            <div><span class="diag-icon amber">⌕</span><p><strong>Database coverage</strong><small><b data-field="database_entries">—</b> Serato entries · <b data-field="tracks_missing_from_database">—</b> scanned tracks absent</small></p><button class="info-button" type="button" aria-label="About database coverage" aria-expanded="false" data-info="Compares filenames in Serato's database with the selected music folder. A scanned track absent from the database may not have been imported, or may be represented under another filename.">i</button></div>
           </div>
         </div>
+
+        <div id="drilldowns" class="drilldowns panel">
+          <div class="section-heading"><div><p class="eyebrow">Look closer</p><h2 id="drilldown-title">Choose a diagnostic</h2></div><span id="drilldown-count">Read-only examples</span></div>
+          <p id="drilldown-summary">Click a metric above to see example files and saved paths behind that number.</p>
+          <div id="drilldown-list" class="detail-list"></div>
+        </div>
+
+        <div id="duplicate-repair" class="repair-panel panel" hidden>
+          <div class="section-heading"><div><p class="eyebrow">Backup-first cleanup</p><h2>Safely consolidate duplicates</h2></div><span class="repair-tag">Preview required</span></div>
+          <p>Choose the real file to keep. Every other path will be backed up and replaced with a shortcut to it, so existing Serato crates remain mapped. Serato’s database V2 is never edited.</p>
+          <div id="repair-choice" class="repair-choice"></div>
+          <div class="backup-options">
+            <label>Backups to keep <input id="backup-limit" type="number" min="1" value="10"></label>
+            <label class="check-label"><input id="keep-all-backups" type="checkbox"> Keep every backup</label>
+          </div>
+          <div id="repair-preview" class="repair-preview" hidden></div>
+          <div class="repair-actions">
+            <button id="preview-repair" type="button">Preview changes</button>
+            <button id="apply-repair" class="danger-action" type="button" disabled>Apply backed-up cleanup</button>
+            <button id="restore-repair" type="button" hidden>Restore this backup</button>
+          </div>
+          <div id="repair-message" class="repair-message" role="status"></div>
+        </div>
       </section>
     </main>
   </div>

tests/test_health.py

@@ -58,6 +58,8 @@ def test_empty_library_has_no_health_score():
     assert report.score is None
     assert report.total_references == 0
     assert report.scored_references == 0
+    assert not report.database_present
+    assert report.tracks_missing_from_database == 0
 
 
 def test_smart_crate_references_are_reported_but_not_scored():

tests/test_repair.py

@@ -0,0 +1,67 @@
+from pathlib import Path
+
+import pytest
+
+from serato_doctor.repair import (
+    BACKUP_FOLDER,
+    apply_duplicate_repair,
+    plan_duplicate_repair,
+    restore_backup,
+    rotate_backups,
+)
+
+
+def library(tmp_path):
+    serato = tmp_path / "_Serato_"
+    crate = serato / "Subcrates" / "House.crate"
+    crate.parent.mkdir(parents=True)
+    crate.write_bytes(b"crate-data")
+    (serato / "database V2").write_bytes(b"database-data")
+    music = tmp_path / "Music"
+    keeper = music / "Main" / "Track.mp3"
+    duplicate = music / "Old" / "Track.mp3"
+    keeper.parent.mkdir(parents=True)
+    duplicate.parent.mkdir(parents=True)
+    keeper.write_bytes(b"keeper")
+    duplicate.write_bytes(b"duplicate")
+    return serato, keeper, duplicate
+
+
+def test_duplicate_repair_backs_up_then_preserves_old_path_as_link(tmp_path):
+    serato, keeper, duplicate = library(tmp_path)
+    plan = plan_duplicate_repair(keeper, (keeper, duplicate), serato)
+
+    receipt = apply_duplicate_repair(plan, serato, backup_limit=3)
+
+    assert duplicate.is_symlink()
+    assert duplicate.resolve() == keeper.resolve()
+    assert (receipt.backup / "manifest.json").is_file()
+    assert any((receipt.backup / "serato-metadata").rglob("House.crate"))
+    assert any((receipt.backup / "serato-metadata").rglob("database V2"))
+
+    restored = restore_backup(receipt.backup)
+    assert restored == (duplicate.resolve(),)
+    assert not duplicate.is_symlink()
+    assert duplicate.read_bytes() == b"duplicate"
+
+
+def test_plan_rejects_keeper_outside_duplicate_group(tmp_path):
+    serato, keeper, duplicate = library(tmp_path)
+    outsider = tmp_path / "outsider.mp3"
+    outsider.write_bytes(b"other")
+
+    with pytest.raises(ValueError, match="must belong"):
+        plan_duplicate_repair(outsider, (keeper, duplicate), serato)
+
+
+def test_backup_rotation_can_be_limited_or_unlimited(tmp_path):
+    root = tmp_path / BACKUP_FOLDER
+    for name in ("001", "002", "003"):
+        backup = root / name
+        backup.mkdir(parents=True)
+        (backup / "manifest.json").write_text("{}", encoding="utf-8")
+
+    rotate_backups(root, None)
+    assert len(list(root.iterdir())) == 3
+    rotate_backups(root, 2)
+    assert {path.name for path in root.iterdir()} == {"002", "003"}

tests/test_web.py

@@ -3,7 +3,7 @@ from pathlib import Path
 
 import pytest
 
-from serato_doctor.web import STATIC_FILES, analyze_paths
+from serato_doctor.web import STATIC_FILES, analyze_paths, duplicate_repair
 
 
 def test_web_analysis_uses_production_health_pipeline(tmp_path):
@@ -23,7 +23,13 @@ def test_web_analysis_uses_production_health_pipeline(tmp_path):
     assert result["smart_crates"] == 2
     assert result["database_entries"] == 10
     assert result["database_library_matches"] == 10
+    assert result["database_missing_paths"] == 0
+    assert result["database_missing_unique_filenames"] == 0
     assert result["tracks_missing_from_database"] == 0
+    assert result["details"]["old_crate_references"]["total"] == 2
+    assert result["details"]["old_crate_references"]["items"][0]["crate"]
+    assert result["details"]["suggested_matches"]["total"] == 0
+    assert result["details"]["unused_tracks"]["total"] == 1
 
 
 def test_web_analysis_rejects_missing_folders(tmp_path):
@@ -36,3 +42,31 @@ def test_web_static_assets_are_declared_and_packaged():
 
     assert set(STATIC_FILES) == {"/", "/app.css", "/app.js"}
     assert all((asset_root / filename).is_file() for filename, _ in STATIC_FILES.values())
+    html = (asset_root / "index.html").read_text(encoding="utf-8")
+    assert "Missing tracks in Serato" in html
+    assert "Old crate references" in html
+    assert "Choose a diagnostic" in html
+    assert 'data-detail="database_missing_tracks"' in html
+    assert 'data-detail="old_crate_references"' in html
+    assert html.count('class="info-button"') >= 10
+
+
+def test_duplicate_repair_preview_does_not_change_files(tmp_path):
+    serato = tmp_path / "_Serato_"
+    serato.mkdir()
+    music = tmp_path / "Music"
+    first = music / "A" / "Track.mp3"
+    second = music / "B" / "Track.mp3"
+    first.parent.mkdir(parents=True)
+    second.parent.mkdir(parents=True)
+    first.write_bytes(b"first")
+    second.write_bytes(b"second")
+
+    result = duplicate_repair(
+        serato, music, first, (first, second), backup_limit=10
+    )
+
+    assert result["applied"] is False
+    assert result["database_v2_modified"] is False
+    assert second.read_bytes() == b"second"
+    assert not second.is_symlink()