Compare commits

..

13 Commits

Author SHA1 Message Date
Philip Guzman ec646479d2 Show detailed batch repair preview 2026-07-01 16:26:04 -07:00
Philip Guzman f3cfed316d Add batch duplicate review workflow 2026-07-01 16:18:30 -07:00
Philip Guzman 4315d8d4d4 Add duplicate audio comparison controls 2026-07-01 16:12:35 -07:00
Philip Guzman 139822f8a5 Fix recovery layout and interactive assets 2026-07-01 16:03:28 -07:00
Philip Guzman da72419ce9 Add backup recovery center 2026-07-01 15:34:58 -07:00
Philip Guzman c4f1b4535d Add backup-first duplicate repair 2026-07-01 15:29:44 -07:00
Philip Guzman d2f625ed18 Add diagnostic drilldowns 2026-07-01 09:17:25 -07:00
Philip Guzman fb3d70e579 Clarify diagnostic language and explanations 2026-07-01 08:37:18 -07:00
Philip Guzman ce37b45058 Merge feature/database-v2-parser into develop 2026-07-01 08:30:53 -07:00
Philip Guzman 37786612b7 Add read-only database V2 parser 2026-07-01 08:13:20 -07:00
Philip Guzman 8ff4db4951 Merge feature/smart-crate-discovery into develop 2026-07-01 08:08:43 -07:00
Philip Guzman a384cdc88d Discover Serato smart crate definitions 2026-07-01 08:06:19 -07:00
Philip Guzman 71bca064ed Merge feature/web-interface into develop 2026-07-01 08:03:19 -07:00
34 changed files with 2272 additions and 46 deletions
+5 -5
View File
@@ -10,7 +10,7 @@
- [x] HTML health dashboard
- [x] Test suite
- [x] Sample library fixtures
- [ ] Database V2 read-only parser
- [x] Database V2 read-only parser
- [x] Configuration
- [x] Logging
- [x] Matching engine
@@ -28,12 +28,12 @@
## v0.3 — Safe Repair
- [ ] Dry-run repair plan
- [ ] Backup before repair
- [ ] Compatibility symlink creation
- [x] Dry-run repair plan
- [x] Backup before repair
- [x] Compatibility symlink creation
- [ ] Compatibility copy creation
- [ ] Rename repair
- [ ] Rollback log
- [x] Rollback log and recovery center
## v0.4 — Migration Wizard
+32
View File
@@ -0,0 +1,32 @@
# Audio comparison for duplicate review
## Problem
Filenames alone are weak evidence. DJs need to hear each candidate and locate it
on disk before deciding which copy is authoritative.
## Design
Exact-duplicate and cloud-conflict groups expose a shared audio player with a
preview action for each file. Switching files reuses the same player, making
back-to-back comparison quick. A separate action reveals the selected file in
macOS Finder.
The browser never receives unrestricted filesystem access. Each analyzed audio
path gets a short, process-local HMAC token. Preview and Finder endpoints reject
altered, expired, missing, non-audio, or otherwise unsigned paths. Audio serving
supports HTTP byte ranges so playback can seek without loading an entire track.
## Edge cases
- A file moved after analysis is rejected.
- Forged or stale tokens cannot select another local file.
- Preview controls remain useful if autoplay is blocked because native audio
controls stay visible.
- Finder failures are reported beside the button without affecting analysis.
## Tests
- Valid signed audio resolves to the analyzed file.
- A modified token is rejected.
- Duplicate detail payloads include preview and Finder controls for every file.
+30
View File
@@ -0,0 +1,30 @@
# Backup recovery center
## Problem
An immediate undo button is not enough. A DJ may restart Serato Doctor, notice
an issue later, or need to understand how much disk space safety snapshots use.
## Architecture
The recovery center reads the existing JSON manifests beneath
`_Serato_/.serato-doctor-backups/`. It reports creation time, selected keeper,
affected paths, restore status, and snapshot size. Invalid or incomplete backup
folders are ignored rather than presented as restorable.
Restore remains conservative: it only replaces a symbolic-link alias and
refuses to overwrite a real file. A successful restore records its timestamp in
the manifest so the UI cannot accidentally offer the same rollback twice.
## Edge cases
- No backup directory is treated as an empty history.
- Malformed manifests do not prevent valid backups from loading.
- Restored snapshots remain visible for audit purposes and disk accounting.
- Backups can only be restored through the Serato library that owns them.
## Tests
- Backup history reports size and ready status.
- Restore changes the persistent status to restored.
- Static recovery assets are included in the package.
+37
View File
@@ -0,0 +1,37 @@
# Batch duplicate review
## Problem
Opening a diagnostic, opening each group again, scrolling to a separate repair
form, and applying one change at a time creates unnecessary friction and many
small backups.
## Design
Duplicate and cloud-conflict diagnostics now open as a sequential review queue.
Each group presents its candidates together with audio preview and Finder
controls. Choosing a keeper records the decision and advances immediately.
Uncertain groups can be skipped without changing them.
All approved decisions are submitted as one dry-run plan. Apply creates one
backup containing every affected audio file plus the Serato metadata snapshot,
then installs all compatibility shortcuts as one rollback unit. Any validation
or filesystem failure prevents or rolls back the batch.
The dry-run opens in a closable modal that names every selected keeper and every
path that will become a compatibility shortcut. Closing the modal preserves the
queue position and all decisions so the DJ can continue reviewing before apply.
## Edge cases
- The same group cannot be submitted twice.
- A file cannot be both a keeper and a disposable file across choices.
- Every group is rescanned and revalidated before preview and apply.
- Skipped and unreviewed groups remain untouched.
- Revisiting a group preserves and visibly marks its selected keeper.
## Tests
- Two groups produce one backup and one restorable receipt.
- Batch preview reports combined changes without modifying files.
- Existing single-group repair remains a wrapper around the batch engine.
+7 -2
View File
@@ -9,10 +9,15 @@ broken manual references or given the same health-score weight.
## Architecture
Crates carry a `static`, `smart`, or `unknown` kind. Folder provenance is the
primary signal: Serato stores regular definitions in `Subcrates` and smart
definitions in `Smartcrates`. `Compatible by key.crate` is also treated as smart
primary signal: Serato stores regular `.crate` files in `Subcrates` and smart
`.scrate` definitions in `SmartCrates`. `Compatible by key.crate` is also treated as smart
when encountered in `Subcrates`, based on the original migration case study.
Smart crate names use `≫≫` to encode hierarchy. The model preserves those segments,
so `Compatible by key≫≫10A.scrate` has a parent of `Compatible by key` and a display
name of `10A`. Smart definitions and dynamic `.crate` containers are counted
separately.
The library loader reads both folders. Health analysis reports all references but
scores only non-smart references. Unknown crates remain scoreable so incomplete
classification cannot silently hide potential problems.
+28
View File
@@ -0,0 +1,28 @@
# Database V2 Read-only Parser
## Problem
Crates and files do not explain every orange track in Serato. The legacy
`database V2` contains Serato's library-level track paths and metadata, so it must
be inspected independently from crate references.
## Architecture
The parser reads the file as a big-endian tag-length-value stream. Top-level
`otrk` records contain nested fields including `pfil` (path), `tsng` (title),
`tart` (artist), `talb` (album), and `tgen` (genre). Text is UTF-16 big-endian.
Analysis reports total database entries, entries whose filenames occur in the
selected music scan, entries outside that scan, scanned tracks absent from the
database, and duplicate database paths. “Outside scan” is deliberately not called
missing because Serato databases can include samples and tracks from other roots.
## Safety
The parser calls only `read_bytes`; it never opens the database for writing. No
metadata values or personal paths are sent to logs or the dashboard.
## Verification
Synthetic TLV fixtures cover version, paths, metadata, incomplete records, and
health integration. The sample library includes a generated ten-entry database.
+35
View File
@@ -0,0 +1,35 @@
# Diagnostic Drill-Downs
## Problem
Aggregate health numbers are useful, but DJs need to understand what is behind
each number before they can trust it. A count like "60 missing tracks" should be
clickable enough to answer: which tracks, which saved paths, and why did Serato
Doctor count them?
## Architecture
The health engine remains responsible for aggregate scoring. The web layer adds
a separate `details` payload beside the existing health report so the UI can show
examples without changing the core score model.
The drill-down payload is intentionally capped. Serato Doctor should explain the
finding quickly in the local browser, not dump an entire user library into the
page.
## Edge Cases
- Smart/dynamic crate references stay excluded from old-reference scoring.
- The same missing filename can appear more than once in Serato's database.
- Duplicate filename and cloud-conflict groups are informational only.
- Suggested matches remain read-only evidence and must never trigger repair.
- Empty drill-downs should feel reassuring, not broken.
## Testing
- Web analysis should include detail sections for old crate references,
suggested matches, unused tracks, duplicates, cloud conflicts, broken
symlinks, and missing Serato database tracks.
- Static assets should expose clickable diagnostic hooks.
- Browser behavior should be verified manually when the local browser policy
allows access to the development server.
+39
View File
@@ -0,0 +1,39 @@
# Duplicate repair
## Problem
Duplicate and cloud-conflict files waste space, but deleting either path can
unmap tracks in crates or Serato's database. DJs need to choose the authoritative
copy and understand every change before it happens.
## Design
The web interface requires an analysis, an explicit keeper selection, and a
dry-run preview. Applying the plan first creates a timestamped backup beneath
`_Serato_/.serato-doctor-backups/`. The snapshot contains every replaced audio
file, loaded crate/smart-crate metadata, database V2, and a JSON restore manifest.
The non-kept audio path is then replaced with a symbolic link to the keeper.
This removes the extra audio payload while preserving every existing saved path.
Crate files and database V2 are never rewritten. The UI exposes immediate
restore using the manifest.
Users may retain all backups or set a positive rotation limit. Rotation occurs
only after a repair succeeds.
## Edge cases
- The duplicate group is rescanned and validated immediately before preview and
apply.
- Existing symlinks cannot be selected as disposable duplicate files.
- A partial failure restores already-changed files before reporting the error.
- Restore refuses to overwrite a real file.
- Healthy symlink aliases are excluded from future duplicate counts.
## Tests
- Backup creation includes audio and Serato metadata.
- The old path resolves to the selected keeper after repair.
- Restore returns the original file contents.
- Invalid keeper choices are rejected.
- Limited and unlimited retention behave deterministically.
+25
View File
@@ -0,0 +1,25 @@
# Plain-language Diagnostics
## Problem
“Broken references” combined crate occurrences with Serato's own missing-track
concept. DJs naturally compared that number with orange/unmapped tracks in Serato,
even though the two counts describe different layers.
## Language
- **Missing tracks in Serato** means database entries whose saved file location no
longer exists. This corresponds most closely to orange or unmapped tracks.
- **Old crate references** means saved appearances in regular crates whose exact
filename was not found in the selected music folder. One track can appear in
several crates, so both appearances and unique filenames are shown.
Every health card and diagnostic row has an accessible information button. Hover
shows its explanation on desktop; click or tap keeps it open; Escape or clicking
elsewhere closes it. Explanations describe uncertainty and avoid implying repair.
## Verification
Health tests verify database missing-path and unique-filename counts. Browser tests
cover the two separate metrics, hover/click explanations, keyboard dismissal, and
mobile layout.
+19
View File
@@ -0,0 +1,19 @@
# Smart Crate Discovery Correction
## Problem
The first classifier searched `SmartCrates` for `*.crate`. Real Serato smart-crate
definitions use `.scrate`, so a library with 26 definitions displayed only the
single name-based `Compatible by key.crate` fallback.
## Correction
Library discovery now reads `.scrate` definitions case-insensitively from the
`SmartCrates` folder. Dynamic `.crate` containers remain excluded from manual
reference scoring but are reported separately. The `≫≫` filename separator is
preserved as smart-crate hierarchy metadata.
## Verification
Synthetic tests cover `.scrate` discovery, case-correct folder names, hierarchy,
definition/container counts, and the existing five-static/two-smart sample.
+29 -5
View File
@@ -10,6 +10,10 @@ SAMPLE_ROOT = Path(__file__).parent
SERATO_PATH_PREFIX = "Users/sample-user/OneDrive/Jukebox/"
def database_record(tag: bytes, payload: bytes) -> bytes:
return tag + len(payload).to_bytes(4, "big") + payload
def load_manifest() -> dict:
return json.loads((SAMPLE_ROOT / "manifest.json").read_text(encoding="utf-8"))
@@ -28,18 +32,38 @@ def build_sample(output: Optional[Path] = None) -> Path:
)
for crate in manifest["crates"]:
folder_name = "Smartcrates" if crate["type"] == "smart" else "Subcrates"
is_smart = crate["type"] == "smart"
folder_name = "SmartCrates" if is_smart else "Subcrates"
crate_root = serato_root / folder_name
crate_root.mkdir(parents=True, exist_ok=True)
other_folder = "Subcrates" if folder_name == "Smartcrates" else "Smartcrates"
stale_path = serato_root / other_folder / crate["name"]
if stale_path.exists():
output_name = (
Path(crate["name"]).with_suffix(".scrate").name
if is_smart
else crate["name"]
)
for other_folder in ("Subcrates", "Smartcrates", "SmartCrates"):
for stale_name in (
crate["name"],
Path(crate["name"]).with_suffix(".scrate").name,
):
stale_path = serato_root / other_folder / stale_name
if stale_path.exists() and stale_path != crate_root / output_name:
stale_path.unlink()
records = "".join(
f"{SERATO_PATH_PREFIX}{relative_path}otrk"
for relative_path in crate["references"]
)
(crate_root / crate["name"]).write_bytes(records.encode("utf-16-le"))
(crate_root / output_name).write_bytes(records.encode("utf-16-le"))
database_records = [
database_record(b"vrsn", "2.0/Serato Doctor Fixture".encode("utf-16-be"))
]
for relative_path in manifest["tracks"]:
fields = database_record(
b"pfil", str(music_root / relative_path).encode("utf-16-be")
)
database_records.append(database_record(b"otrk", fields))
(serato_root / "database V2").write_bytes(b"".join(database_records))
return root
+15
View File
@@ -3,6 +3,7 @@ import argparse
from serato_doctor.config import ScanConfig
from serato_doctor.crate_parser import load_library_crates
from serato_doctor.database_parser import parse_database
from serato_doctor.health import analyze_health
from serato_doctor.logging import configure_logging
from serato_doctor.models.library import Library
@@ -57,10 +58,13 @@ def main():
crates = load_library_crates(config.serato, config.reference_roots)
filesystem = scan_filesystem(config.music)
database_path = config.serato / "database V2"
database = parse_database(database_path) if database_path.is_file() else None
library = Library.from_crates(
crates=crates,
tracks=filesystem.tracks,
broken_symlinks=filesystem.broken_symlinks,
database=database,
)
results = library.reconcile_by_filename()
missing_count = sum(1 for result in results if not result.exists_by_filename)
@@ -95,9 +99,20 @@ def main():
print(f"Broken Symlinks: {health.broken_symlinks}")
print(f"Static Crates: {health.static_crates}")
print(f"Smart Crates: {health.smart_crates}")
print(f"Smart Crate Containers: {health.smart_crate_containers}")
print(
f"Dynamic References Excluded: {health.dynamic_references_excluded}"
)
print(f"Database Entries: {health.database_entries}")
print(f"Database / Library Matches: {health.database_library_matches}")
print(f"Database Entries Outside Scan: {health.database_unmatched_entries}")
print(f"Missing Tracks in Serato: {health.database_missing_paths}")
print(
"Unique Missing Tracks in Serato: "
f"{health.database_missing_unique_filenames}"
)
print(f"Tracks Missing From Database: {health.tracks_missing_from_database}")
print(f"Duplicate Database Paths: {health.duplicate_database_paths}")
else:
write_csv(results, config.out)
write_missing_report(results, config.report)
+13 -3
View File
@@ -103,9 +103,19 @@ def load_library_crates(
serato_root: Path, reference_roots: Iterable[Path] = ()
) -> Tuple[Crate, ...]:
reference_roots = tuple(reference_roots)
if not serato_root.is_dir():
return ()
crates = []
for folder_name in ("Subcrates", "Smartcrates"):
folder = serato_root / folder_name
for crate_path in folder.rglob("*.crate"):
folder_patterns = {
"subcrates": ("*.crate",),
"smartcrates": ("*.scrate", "*.crate"),
}
for folder in serato_root.iterdir():
patterns = folder_patterns.get(folder.name.casefold())
if patterns is None or not folder.is_dir():
continue
for pattern in patterns:
for crate_path in folder.rglob(pattern):
crates.append(load_crate(crate_path, reference_roots))
return tuple(sorted(crates, key=lambda crate: str(crate.path)))
+67
View File
@@ -0,0 +1,67 @@
from pathlib import Path
from typing import Dict, Iterator, Optional, Tuple
from serato_doctor.models.database import DatabaseTrack, SeratoDatabase
TEXT_FIELDS = {
b"tsng": "title",
b"tart": "artist",
b"talb": "album",
b"tgen": "genre",
}
def iter_records(data: bytes) -> Iterator[Tuple[bytes, bytes]]:
"""Yield complete big-endian tag-length-value records."""
offset = 0
while offset + 8 <= len(data):
tag = data[offset : offset + 4]
length = int.from_bytes(data[offset + 4 : offset + 8], "big")
payload_start = offset + 8
payload_end = payload_start + length
if payload_end > len(data):
break
yield tag, data[payload_start:payload_end]
offset = payload_end
def decode_text(payload: bytes) -> Optional[str]:
value = payload.decode("utf-16-be", errors="ignore").strip("\x00").strip()
return value or None
def normalize_database_path(value: str) -> Path:
if value.startswith(("Users/", "Volumes/")):
value = "/" + value
return Path(value)
def parse_track(payload: bytes) -> Optional[DatabaseTrack]:
fields: Dict[str, Optional[str]] = {}
path = None
for tag, value in iter_records(payload):
if tag == b"pfil":
decoded_path = decode_text(value)
if decoded_path:
path = normalize_database_path(decoded_path)
elif tag in TEXT_FIELDS:
fields[TEXT_FIELDS[tag]] = decode_text(value)
if path is None:
return None
return DatabaseTrack(path=path, filename=path.name, **fields)
def parse_database(database_path: Path) -> SeratoDatabase:
data = database_path.read_bytes()
version = None
tracks = []
for tag, payload in iter_records(data):
if tag == b"vrsn":
version = decode_text(payload)
elif tag == b"otrk":
track = parse_track(payload)
if track is not None:
tracks.append(track)
return SeratoDatabase(database_path, version, tuple(tracks))
+3 -1
View File
@@ -11,7 +11,9 @@ def find_duplicate_groups(
) -> Tuple[DuplicateGroup, ...]:
"""Find exact-name duplicates and suspected numeric conflict copies."""
track_tuple = tuple(tracks)
# Healthy symlinks preserve legacy Serato paths without consuming another
# copy of the audio, so they are aliases rather than duplicate files.
track_tuple = tuple(track for track in tracks if not track.path.is_symlink())
by_name: DefaultDict[str, List[DiskTrack]] = defaultdict(list)
by_conflict_name: DefaultDict[str, List[DiskTrack]] = defaultdict(list)
+37 -2
View File
@@ -1,5 +1,7 @@
from collections import Counter
from serato_doctor.duplicates import find_duplicate_groups
from serato_doctor.matching import MatchingEngine
from serato_doctor.matching import MatchingEngine, normalize
from serato_doctor.models.crate import CrateKind
from serato_doctor.models.duplicate import DuplicateKind
from serato_doctor.models.health import HealthReport
@@ -44,6 +46,16 @@ def analyze_health(library: Library) -> HealthReport:
bool(matcher.candidates_for(result.reference)) for result in missing
)
database_tracks = library.database.tracks if library.database else ()
database_names = {normalize(track.filename) for track in database_tracks}
library_names = {normalize(track.filename) for track in library.tracks}
database_path_counts = Counter(
normalize(str(track.path)) for track in database_tracks
)
missing_database_tracks = [
track for track in database_tracks if not track.path.exists()
]
return HealthReport(
score=score,
total_references=len(library.references),
@@ -67,7 +79,12 @@ def analyze_health(library: Library) -> HealthReport:
crate.kind is CrateKind.STATIC for crate in library.crates
),
smart_crates=sum(
crate.kind is CrateKind.SMART for crate in library.crates
crate.kind is CrateKind.SMART and crate.is_smart_definition
for crate in library.crates
),
smart_crate_containers=sum(
crate.kind is CrateKind.SMART and not crate.is_smart_definition
for crate in library.crates
),
unknown_crates=sum(
crate.kind is CrateKind.UNKNOWN for crate in library.crates
@@ -77,4 +94,22 @@ def analyze_health(library: Library) -> HealthReport:
for crate in library.crates
if crate.kind is CrateKind.SMART
),
database_present=library.database is not None,
database_entries=len(database_tracks),
database_library_matches=sum(
normalize(track.filename) in library_names for track in database_tracks
),
database_unmatched_entries=sum(
normalize(track.filename) not in library_names for track in database_tracks
),
database_missing_paths=len(missing_database_tracks),
database_missing_unique_filenames=len(
{normalize(track.filename) for track in missing_database_tracks}
),
tracks_missing_from_database=sum(
normalize(track.filename) not in database_names for track in library.tracks
) if library.database else 0,
duplicate_database_paths=sum(
count - 1 for count in database_path_counts.values() if count > 1
),
)
+3
View File
@@ -1,4 +1,5 @@
from serato_doctor.models.crate import Crate, CrateKind
from serato_doctor.models.database import DatabaseTrack, SeratoDatabase
from serato_doctor.models.duplicate import DuplicateGroup, DuplicateKind
from serato_doctor.models.filesystem import BrokenSymlink, FilesystemScan
from serato_doctor.models.health import HealthReport
@@ -10,6 +11,7 @@ from serato_doctor.models.track import DiskTrack
__all__ = [
"Crate",
"CrateKind",
"DatabaseTrack",
"DiskTrack",
"DuplicateGroup",
"DuplicateKind",
@@ -19,6 +21,7 @@ __all__ = [
"Library",
"MatchEvidence",
"ReferenceResult",
"SeratoDatabase",
"TrackMatch",
"TrackReference",
]
+12
View File
@@ -19,3 +19,15 @@ class Crate:
path: Path
references: Tuple[TrackReference, ...]
kind: CrateKind = CrateKind.UNKNOWN
@property
def hierarchy(self) -> Tuple[str, ...]:
return tuple(self.path.stem.split("≫≫"))
@property
def display_name(self) -> str:
return self.hierarchy[-1]
@property
def is_smart_definition(self) -> bool:
return self.path.suffix.casefold() == ".scrate"
+22
View File
@@ -0,0 +1,22 @@
from dataclasses import dataclass
from pathlib import Path
from typing import Optional, Tuple
@dataclass(frozen=True)
class DatabaseTrack:
"""Read-only metadata extracted from one database V2 track record."""
path: Path
filename: str
title: Optional[str] = None
artist: Optional[str] = None
album: Optional[str] = None
genre: Optional[str] = None
@dataclass(frozen=True)
class SeratoDatabase:
path: Path
version: Optional[str]
tracks: Tuple[DatabaseTrack, ...]
+9
View File
@@ -22,8 +22,17 @@ class HealthReport:
broken_symlinks: int
static_crates: int
smart_crates: int
smart_crate_containers: int
unknown_crates: int
dynamic_references_excluded: int
database_present: bool
database_entries: int
database_library_matches: int
database_unmatched_entries: int
database_missing_paths: int
database_missing_unique_filenames: int
tracks_missing_from_database: int
duplicate_database_paths: int
@property
def score_basis(self) -> str:
+7 -1
View File
@@ -1,7 +1,8 @@
from dataclasses import dataclass
from typing import Iterable, Tuple
from typing import Iterable, Optional, Tuple
from serato_doctor.models.crate import Crate
from serato_doctor.models.database import SeratoDatabase
from serato_doctor.models.filesystem import BrokenSymlink
from serato_doctor.models.reference import ReferenceResult, TrackReference
from serato_doctor.models.track import DiskTrack
@@ -15,6 +16,7 @@ class Library:
tracks: Tuple[DiskTrack, ...]
crates: Tuple[Crate, ...] = ()
broken_symlinks: Tuple[BrokenSymlink, ...] = ()
database: Optional[SeratoDatabase] = None
@classmethod
def build(
@@ -22,11 +24,13 @@ class Library:
references: Iterable[TrackReference],
tracks: Iterable[DiskTrack],
broken_symlinks: Iterable[BrokenSymlink] = (),
database: Optional[SeratoDatabase] = None,
) -> "Library":
return cls(
tuple(references),
tuple(tracks),
broken_symlinks=tuple(broken_symlinks),
database=database,
)
@classmethod
@@ -35,6 +39,7 @@ class Library:
crates: Iterable[Crate],
tracks: Iterable[DiskTrack],
broken_symlinks: Iterable[BrokenSymlink] = (),
database: Optional[SeratoDatabase] = None,
) -> "Library":
crate_tuple = tuple(crates)
references = tuple(
@@ -47,6 +52,7 @@ class Library:
tuple(tracks),
crate_tuple,
tuple(broken_symlinks),
database,
)
def reconcile_by_filename(self) -> Tuple[ReferenceResult, ...]:
+242
View File
@@ -0,0 +1,242 @@
"""Backup-first duplicate consolidation.
Serato's binary metadata is deliberately not rewritten. Removed duplicate files
are replaced with symbolic links, so every existing path continues to resolve.
"""
import json
import shutil
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Iterable, Optional, Tuple
from uuid import uuid4
BACKUP_FOLDER = ".serato-doctor-backups"
@dataclass(frozen=True)
class DuplicateRepairPlan:
keeper: Path
replaced: Tuple[Path, ...]
metadata_files: Tuple[Path, ...]
@property
def changes(self) -> int:
return len(self.replaced)
@dataclass(frozen=True)
class RepairReceipt:
backup: Path
keeper: Path
replaced: Tuple[Path, ...]
@dataclass(frozen=True)
class BackupSummary:
path: Path
created_at: str
keeper: Path
replaced: Tuple[Path, ...]
size_bytes: int
restored_at: Optional[str]
@property
def status(self) -> str:
return "restored" if self.restored_at else "ready"
def plan_duplicate_repair(
keeper: Path, duplicates: Iterable[Path], serato_root: Path
) -> DuplicateRepairPlan:
keeper = keeper.expanduser().resolve()
candidates = tuple(path.expanduser().resolve() for path in duplicates)
if keeper not in candidates:
raise ValueError("The file to keep must belong to this duplicate group")
if not keeper.is_file():
raise ValueError(f"The file to keep no longer exists: {keeper}")
replaced = tuple(path for path in candidates if path != keeper)
if not replaced:
raise ValueError("Choose a duplicate group containing at least two files")
if any(not path.is_file() or path.is_symlink() for path in replaced):
raise ValueError("A duplicate changed since the analysis; analyze again")
metadata = tuple(
sorted(
path
for path in serato_root.expanduser().resolve().rglob("*")
if path.is_file()
and BACKUP_FOLDER not in path.parts
and (
path.name == "database V2"
or path.suffix.casefold() in {".crate", ".scrate"}
)
)
)
return DuplicateRepairPlan(keeper, replaced, metadata)
def apply_duplicate_repair(
plan: DuplicateRepairPlan,
serato_root: Path,
backup_limit: Optional[int] = 10,
) -> RepairReceipt:
"""Create a complete rollback snapshot, then replace extras with symlinks."""
return apply_duplicate_repair_batch((plan,), serato_root, backup_limit)
def apply_duplicate_repair_batch(
plans: Iterable[DuplicateRepairPlan],
serato_root: Path,
backup_limit: Optional[int] = 10,
) -> RepairReceipt:
"""Apply several approved duplicate choices as one atomic backup."""
plans = tuple(plans)
if not plans:
raise ValueError("Choose at least one duplicate group")
if backup_limit is not None and backup_limit < 1:
raise ValueError("Backup limit must be at least 1, or unlimited")
replaced_paths = tuple(
path for plan in plans for path in plan.replaced
)
if len(set(replaced_paths)) != len(replaced_paths):
raise ValueError("The same file appears in more than one repair choice")
keepers = {plan.keeper for plan in plans}
if keepers.intersection(replaced_paths):
raise ValueError("A selected keeper cannot be removed by another choice")
backup_root = serato_root.expanduser().resolve() / BACKUP_FOLDER
backup = backup_root / _backup_name()
files_root = backup / "files"
metadata_root = backup / "serato-metadata"
backup.mkdir(parents=True)
entries = []
try:
for plan in plans:
for path in plan.replaced:
destination = files_root / _safe_backup_path(path)
destination.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(path, destination)
entries.append(
{
"original": str(path),
"backup": str(destination),
"keeper": str(plan.keeper),
}
)
for path in plans[0].metadata_files:
relative = path.relative_to(serato_root.expanduser().resolve())
destination = metadata_root / relative
destination.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(path, destination)
manifest = {
"created_at": datetime.now(timezone.utc).isoformat(),
"keeper": str(plans[0].keeper),
"keepers": [str(plan.keeper) for plan in plans],
"choice_count": len(plans),
"replaced": entries,
"strategy": "symlink",
}
(backup / "manifest.json").write_text(
json.dumps(manifest, indent=2), encoding="utf-8"
)
for entry in entries:
path = Path(entry["original"])
path.unlink()
path.symlink_to(Path(entry["keeper"]))
except Exception:
_rollback_entries(entries)
shutil.rmtree(backup, ignore_errors=True)
raise
rotate_backups(backup_root, backup_limit)
return RepairReceipt(backup, plans[0].keeper, replaced_paths)
def restore_backup(backup: Path) -> Tuple[Path, ...]:
manifest_path = backup.expanduser().resolve() / "manifest.json"
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
restored = []
for entry in manifest["replaced"]:
original = Path(entry["original"])
saved = Path(entry["backup"])
if original.exists() and not original.is_symlink():
raise ValueError(f"Restore would overwrite a real file: {original}")
if original.is_symlink():
original.unlink()
original.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(saved, original)
restored.append(original)
manifest["restored_at"] = datetime.now(timezone.utc).isoformat()
manifest_path.write_text(json.dumps(manifest, indent=2), encoding="utf-8")
return tuple(restored)
def list_backups(serato_root: Path) -> Tuple[BackupSummary, ...]:
backup_root = serato_root.expanduser().resolve() / BACKUP_FOLDER
if not backup_root.is_dir():
return ()
summaries = []
for backup in backup_root.iterdir():
manifest_path = backup / "manifest.json"
if not manifest_path.is_file():
continue
try:
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
replaced = tuple(
Path(entry["original"]) for entry in manifest["replaced"]
)
size = sum(
path.stat().st_size
for path in backup.rglob("*")
if path.is_file()
)
summaries.append(
BackupSummary(
path=backup.resolve(),
created_at=manifest["created_at"],
keeper=Path(manifest["keeper"]),
replaced=replaced,
size_bytes=size,
restored_at=manifest.get("restored_at"),
)
)
except (KeyError, OSError, TypeError, json.JSONDecodeError):
continue
return tuple(
sorted(summaries, key=lambda item: item.created_at, reverse=True)
)
def rotate_backups(backup_root: Path, limit: Optional[int]) -> None:
if limit is None or not backup_root.is_dir():
return
backups = sorted(
(path for path in backup_root.iterdir() if (path / "manifest.json").is_file()),
key=lambda path: path.name,
reverse=True,
)
for expired in backups[limit:]:
shutil.rmtree(expired)
def _backup_name() -> str:
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
return f"{stamp}-{uuid4().hex[:8]}"
def _safe_backup_path(path: Path) -> Path:
anchorless = path.as_posix().lstrip("/").replace(":", "_")
return Path(anchorless)
def _rollback_entries(entries: Iterable[dict]) -> None:
for entry in entries:
original = Path(entry["original"])
saved = Path(entry["backup"])
if original.is_symlink():
original.unlink()
if not original.exists() and saved.is_file():
original.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(saved, original)
+457 -4
View File
@@ -1,25 +1,264 @@
import argparse
import base64
import binascii
import hmac
import json
import mimetypes
import secrets
import subprocess
from collections import Counter
from dataclasses import asdict
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from importlib import resources
from pathlib import Path
from typing import Iterable
from typing import Iterable, Optional
from urllib.parse import parse_qs, quote, urlsplit
from serato_doctor.crate_parser import load_library_crates
from serato_doctor.database_parser import parse_database
from serato_doctor.duplicates import find_duplicate_groups
from serato_doctor.health import analyze_health
from serato_doctor.matching import MatchingEngine, normalize
from serato_doctor.models.crate import CrateKind
from serato_doctor.models.duplicate import DuplicateKind
from serato_doctor.models.library import Library
from serato_doctor.scanner import scan_filesystem
from serato_doctor.repair import (
BACKUP_FOLDER,
apply_duplicate_repair,
apply_duplicate_repair_batch,
list_backups,
plan_duplicate_repair,
restore_backup,
)
MAX_REQUEST_BYTES = 64 * 1024
DETAIL_LIMIT = 50
FILE_TOKEN_SECRET = secrets.token_bytes(32)
STATIC_FILES = {
"/": ("index.html", "text/html; charset=utf-8"),
"/app.css": ("app.css", "text/css; charset=utf-8"),
"/recovery.css": ("recovery.css", "text/css; charset=utf-8"),
"/layout-fixes.css": ("layout-fixes.css", "text/css; charset=utf-8"),
"/app.js": ("app.js", "text/javascript; charset=utf-8"),
}
def _display_path(path: Path) -> str:
return str(path)
def _file_token(path: Path) -> str:
encoded = base64.urlsafe_b64encode(str(path.resolve()).encode()).decode()
signature = hmac.digest(FILE_TOKEN_SECRET, encoded.encode(), "sha256").hex()
return f"{encoded}.{signature}"
def _verified_audio(token: str) -> Path:
try:
encoded, signature = token.rsplit(".", 1)
expected = hmac.digest(
FILE_TOKEN_SECRET, encoded.encode(), "sha256"
).hex()
if not hmac.compare_digest(signature, expected):
raise ValueError
path = Path(base64.urlsafe_b64decode(encoded.encode()).decode())
except (binascii.Error, ValueError, UnicodeDecodeError):
raise ValueError("Invalid or expired file preview")
if not path.is_file() or path.suffix.casefold() not in {
".mp3",
".m4a",
".wav",
".aif",
".aiff",
".flac",
}:
raise ValueError("Audio file is no longer available")
return path
def _file_preview(path: Path) -> dict:
token = _file_token(path)
return {
"path": _display_path(path),
"audio_url": f"/api/audio?token={quote(token)}",
"reveal_token": token,
}
def _first_reason(match) -> str:
for item in match.evidence:
if item.matched:
return item.explanation
return "Filename-related candidate"
def diagnostic_details(library: Library, limit: int = DETAIL_LIMIT) -> dict:
dynamic_sources = {
crate.path for crate in library.crates if crate.kind is CrateKind.SMART
}
results = tuple(
result
for result in library.reconcile_by_filename()
if result.reference.source not in dynamic_sources
)
missing = [result for result in results if not result.exists_by_filename]
matcher = MatchingEngine(library.tracks)
suggested_matches = []
for result in missing:
candidates = matcher.candidates_for(result.reference)
if not candidates:
continue
best = candidates[0]
suggested_matches.append(
{
"filename": result.reference.filename,
"crate": _display_path(result.reference.source),
"saved_path": _display_path(result.reference.path),
"candidate": _display_path(best.track.path),
"score": f"{best.score_percent}%",
"reason": _first_reason(best),
}
)
duplicate_groups = find_duplicate_groups(library.tracks)
exact_duplicates = [
group
for group in duplicate_groups
if group.kind is DuplicateKind.EXACT_NAME
]
cloud_conflicts = [
group
for group in duplicate_groups
if group.kind is DuplicateKind.CLOUD_CONFLICT
]
database_tracks = library.database.tracks if library.database else ()
missing_database_tracks = [
track for track in database_tracks if not track.path.exists()
]
database_filename_counts = Counter(
normalize(track.filename) for track in missing_database_tracks
)
referenced_names = {reference.filename for reference in library.references}
unused_tracks = [
track for track in library.tracks if track.filename not in referenced_names
]
return {
"database_missing_tracks": {
"title": "Missing tracks in Serato",
"summary": (
"These are Serato database entries whose saved file location "
"does not currently exist on disk."
),
"total": len(missing_database_tracks),
"items": [
{
"filename": track.filename,
"saved_path": _display_path(track.path),
"artist": track.artist or "Unknown artist",
"title": track.title or track.filename,
"repeated_filename": database_filename_counts[
normalize(track.filename)
]
> 1,
}
for track in missing_database_tracks[:limit]
],
},
"old_crate_references": {
"title": "Old crate references",
"summary": (
"These are regular crate appearances whose exact filename was "
"not found in the selected music folder."
),
"total": len(missing),
"items": [
{
"filename": result.reference.filename,
"crate": _display_path(result.reference.source),
"saved_path": _display_path(result.reference.path),
}
for result in missing[:limit]
],
},
"suggested_matches": {
"title": "Suggested matches",
"summary": (
"These are read-only guesses where Serato Doctor found a "
"filename-related candidate on disk."
),
"total": len(suggested_matches),
"items": suggested_matches[:limit],
},
"duplicate_filenames": {
"title": "Duplicate filenames",
"summary": (
"These groups contain different files with the same cleaned-up "
"filename. Review before making any decisions."
),
"total": len(exact_duplicates),
"items": [
{
"filename": group.display_name,
"files": [_display_path(track.path) for track in group.tracks],
"file_previews": [
_file_preview(track.path) for track in group.tracks
],
}
for group in exact_duplicates[:limit]
],
},
"cloud_conflicts": {
"title": "Possible cloud conflicts",
"summary": (
"These filename families look like cloud sync conflict copies, "
"such as a duplicate ending in a number."
),
"total": len(cloud_conflicts),
"items": [
{
"filename": group.display_name,
"files": [_display_path(track.path) for track in group.tracks],
"file_previews": [
_file_preview(track.path) for track in group.tracks
],
}
for group in cloud_conflicts[:limit]
],
},
"broken_symlinks": {
"title": "Broken shortcuts",
"summary": (
"These symbolic links point somewhere that no longer resolves."
),
"total": len(library.broken_symlinks),
"items": [
{
"path": _display_path(link.path),
"target": _display_path(link.target) if link.target else "Unknown",
}
for link in library.broken_symlinks[:limit]
],
},
"unused_tracks": {
"title": "Unused tracks",
"summary": (
"These scanned files were not referenced by any loaded crate. "
"That does not mean they should be deleted."
),
"total": len(unused_tracks),
"items": [
{"filename": track.filename, "path": _display_path(track.path)}
for track in unused_tracks[:limit]
],
},
}
def analyze_paths(
serato: Path, music: Path, reference_roots: Iterable[Path] = ()
) -> dict:
@@ -33,20 +272,149 @@ def analyze_paths(
crates = load_library_crates(serato, reference_roots)
filesystem = scan_filesystem(music)
database_path = serato / "database V2"
database = parse_database(database_path) if database_path.is_file() else None
library = Library.from_crates(
crates,
filesystem.tracks,
filesystem.broken_symlinks,
database,
)
report = analyze_health(library)
result = asdict(report)
result["score_basis"] = report.score_basis
result["details"] = diagnostic_details(library)
return result
def duplicate_repair(
serato: Path,
music: Path,
keeper: Path,
group_files: Iterable[Path],
backup_limit: Optional[int],
apply: bool = False,
) -> dict:
"""Validate a current duplicate group and preview or apply consolidation."""
serato = serato.expanduser().resolve()
music = music.expanduser().resolve()
keeper = keeper.expanduser().resolve()
requested = tuple(path.expanduser().resolve() for path in group_files)
if not serato.is_dir() or not music.is_dir():
raise ValueError("Analyze the library again before repairing duplicates")
tracks = scan_filesystem(music).tracks
groups = find_duplicate_groups(tracks)
valid_groups = [
{track.path.resolve() for track in group.tracks} for group in groups
]
if set(requested) not in valid_groups:
raise ValueError("This duplicate group changed; analyze the library again")
plan = plan_duplicate_repair(keeper, requested, serato)
result = {
"keeper": str(plan.keeper),
"replaced": [str(path) for path in plan.replaced],
"metadata_backups": len(plan.metadata_files),
"strategy": "shortcut",
"database_v2_modified": False,
}
if apply:
receipt = apply_duplicate_repair(plan, serato, backup_limit)
result.update({"applied": True, "backup": str(receipt.backup)})
else:
result["applied"] = False
return result
def duplicate_repair_batch(
serato: Path,
music: Path,
choices: Iterable[dict],
backup_limit: Optional[int],
apply: bool = False,
) -> dict:
"""Validate and preview or apply several keeper choices together."""
serato = serato.expanduser().resolve()
music = music.expanduser().resolve()
if not serato.is_dir() or not music.is_dir():
raise ValueError("Analyze the library again before repairing duplicates")
groups = find_duplicate_groups(scan_filesystem(music).tracks)
valid_groups = [
{track.path.resolve() for track in group.tracks} for group in groups
]
plans = []
selected_groups = set()
for choice in choices:
requested = tuple(Path(value).expanduser().resolve() for value in choice["group_files"])
group_key = frozenset(requested)
if set(requested) not in valid_groups:
raise ValueError("A duplicate group changed; analyze the library again")
if group_key in selected_groups:
raise ValueError("A duplicate group was selected more than once")
selected_groups.add(group_key)
plans.append(
plan_duplicate_repair(Path(choice["keeper"]), requested, serato)
)
if not plans:
raise ValueError("Choose at least one duplicate group")
replaced = [str(path) for plan in plans for path in plan.replaced]
result = {
"choice_count": len(plans),
"replaced": replaced,
"decisions": [
{
"keeper": str(plan.keeper),
"replaced": [str(path) for path in plan.replaced],
}
for plan in plans
],
"metadata_backups": len(plans[0].metadata_files),
"strategy": "shortcut",
"database_v2_modified": False,
}
if apply:
receipt = apply_duplicate_repair_batch(plans, serato, backup_limit)
result.update({"applied": True, "backup": str(receipt.backup)})
else:
result["applied"] = False
return result
def backup_history(serato: Path) -> dict:
serato = serato.expanduser().resolve()
if not serato.is_dir():
raise ValueError(f"Serato folder does not exist: {serato}")
backups = list_backups(serato)
return {
"total": len(backups),
"size_bytes": sum(backup.size_bytes for backup in backups),
"items": [
{
"path": str(backup.path),
"created_at": backup.created_at,
"keeper": str(backup.keeper),
"replaced": [str(path) for path in backup.replaced],
"size_bytes": backup.size_bytes,
"status": backup.status,
"restored_at": backup.restored_at,
}
for backup in backups
],
}
class SeratoDoctorHandler(BaseHTTPRequestHandler):
def do_GET(self) -> None:
asset = STATIC_FILES.get(self.path)
request = urlsplit(self.path)
if request.path == "/api/audio":
try:
token = parse_qs(request.query)["token"][0]
self._audio_response(_verified_audio(token))
except (BrokenPipeError, ConnectionResetError):
return
except (KeyError, IndexError, OSError, ValueError) as error:
self._json_response(404, {"error": str(error)})
return
asset = STATIC_FILES.get(request.path)
if asset is None:
self._json_response(404, {"error": "Not found"})
return
@@ -58,12 +426,23 @@ class SeratoDoctorHandler(BaseHTTPRequestHandler):
)
self.send_response(200)
self.send_header("Content-Type", content_type)
self.send_header("Cache-Control", "no-store")
self.send_header("Content-Length", str(len(content)))
self.end_headers()
self.wfile.write(content)
def do_POST(self) -> None:
if self.path != "/api/analyze":
allowed = {
"/api/analyze",
"/api/duplicates/preview",
"/api/duplicates/apply",
"/api/duplicates/batch/preview",
"/api/duplicates/batch/apply",
"/api/backups/restore",
"/api/backups",
"/api/reveal",
}
if self.path not in allowed:
self._json_response(404, {"error": "Not found"})
return
try:
@@ -73,15 +452,89 @@ class SeratoDoctorHandler(BaseHTTPRequestHandler):
payload = json.loads(self.rfile.read(length))
if not isinstance(payload, dict):
raise ValueError("Request body must be a JSON object")
if self.path == "/api/analyze":
roots = [Path(value) for value in payload.get("reference_roots", [])]
result = analyze_paths(
Path(payload["serato"]), Path(payload["music"]), roots
)
except (KeyError, TypeError, json.JSONDecodeError, ValueError) as error:
elif self.path == "/api/reveal":
path = _verified_audio(payload["token"])
subprocess.run(["open", "-R", str(path)], check=True)
result = {"revealed": str(path)}
elif self.path == "/api/backups":
result = backup_history(Path(payload["serato"]))
elif self.path == "/api/backups/restore":
serato = Path(payload["serato"]).expanduser().resolve()
backup = Path(payload["backup"]).expanduser().resolve()
backup_root = (serato / BACKUP_FOLDER).resolve()
if backup.parent != backup_root:
raise ValueError("That backup does not belong to this library")
restored = restore_backup(backup)
result = {"restored": [str(path) for path in restored]}
elif self.path.startswith("/api/duplicates/batch/"):
raw_limit = payload.get("backup_limit", 10)
backup_limit = None if raw_limit is None else int(raw_limit)
result = duplicate_repair_batch(
Path(payload["serato"]),
Path(payload["music"]),
payload["choices"],
backup_limit,
apply=self.path.endswith("/apply"),
)
else:
raw_limit = payload.get("backup_limit", 10)
backup_limit = None if raw_limit is None else int(raw_limit)
result = duplicate_repair(
Path(payload["serato"]),
Path(payload["music"]),
Path(payload["keeper"]),
(Path(value) for value in payload["group_files"]),
backup_limit,
apply=self.path.endswith("/apply"),
)
except (
KeyError,
TypeError,
OSError,
json.JSONDecodeError,
ValueError,
) as error:
self._json_response(400, {"error": str(error)})
return
self._json_response(200, result)
def _audio_response(self, path: Path) -> None:
size = path.stat().st_size
start, end = 0, size - 1
status = 200
range_header = self.headers.get("Range")
if range_header and range_header.startswith("bytes="):
raw_start, _, raw_end = range_header[6:].partition("-")
start = int(raw_start or 0)
end = min(int(raw_end) if raw_end else end, end)
if start < 0 or start > end:
raise ValueError("Invalid audio range")
status = 206
length = end - start + 1
self.send_response(status)
self.send_header(
"Content-Type", mimetypes.guess_type(path.name)[0] or "audio/mpeg"
)
self.send_header("Accept-Ranges", "bytes")
self.send_header("Content-Length", str(length))
if status == 206:
self.send_header("Content-Range", f"bytes {start}-{end}/{size}")
self.end_headers()
with path.open("rb") as audio:
audio.seek(start)
remaining = length
while remaining:
chunk = audio.read(min(64 * 1024, remaining))
if not chunk:
break
self.wfile.write(chunk)
remaining -= len(chunk)
def _json_response(self, status: int, payload: dict) -> None:
content = json.dumps(payload).encode("utf-8")
self.send_response(status)
File diff suppressed because one or more lines are too long
+319
View File
@@ -2,12 +2,296 @@ const form = document.querySelector('#analysis-form');
const button = document.querySelector('#analyze-button');
const errorBox = document.querySelector('#error-message');
const results = document.querySelector('#dashboard');
const infoButtons = document.querySelectorAll('.info-button');
const drillTriggers = document.querySelectorAll('[data-detail]');
const drilldownTitle = document.querySelector('#drilldown-title');
const drilldownCount = document.querySelector('#drilldown-count');
const drilldownSummary = document.querySelector('#drilldown-summary');
const drilldownList = document.querySelector('#drilldown-list');
const repairPanel = document.querySelector('#duplicate-repair');
const repairChoice = document.querySelector('#repair-choice');
const repairPreview = document.querySelector('#repair-preview');
const repairMessage = document.querySelector('#repair-message');
const previewRepairButton = document.querySelector('#preview-repair');
const applyRepairButton = document.querySelector('#apply-repair');
const restoreRepairButton = document.querySelector('#restore-repair');
const loadBackupsButton = document.querySelector('#load-backups');
const backupSummary = document.querySelector('#backup-summary');
const backupList = document.querySelector('#backup-list');
const audioPreview = document.querySelector('#audio-preview');
const audioPreviewName = document.querySelector('#audio-preview-name');
const audioPlayer = document.querySelector('#audio-player');
const closeAudioPreview = document.querySelector('#close-audio-preview');
const batchPreviewModal = document.querySelector('#batch-preview-modal');
const batchPreviewSummary = document.querySelector('#batch-preview-summary');
const batchPreviewList = document.querySelector('#batch-preview-list');
const batchPreviewSafety = document.querySelector('#batch-preview-safety');
const closeBatchPreview = document.querySelector('#close-batch-preview');
const continueReviewing = document.querySelector('#continue-reviewing');
const acceptPreview = document.querySelector('#accept-preview');
let latestAnalysis = null;
let selectedDuplicateGroup = null;
let previewedRepair = null;
let latestBackup = null;
let reviewState = null;
function expandHome(path) {
return path.trim();
}
function escapeHtml(value) {
return String(value ?? '').replace(/[&<>"']/g, (character) => ({
'&': '&amp;',
'<': '&lt;',
'>': '&gt;',
'"': '&quot;',
"'": '&#039;',
}[character]));
}
function formatBytes(bytes) {
if (!bytes) return '0 B';
const units = ['B', 'KB', 'MB', 'GB', 'TB'];
const unit = Math.min(Math.floor(Math.log(bytes) / Math.log(1024)), units.length - 1);
return `${(bytes / (1024 ** unit)).toFixed(unit ? 1 : 0)} ${units[unit]}`;
}
function formatDate(value) {
const date = new Date(value);
return Number.isNaN(date.valueOf()) ? value : date.toLocaleString([], {dateStyle: 'medium', timeStyle: 'short'});
}
async function loadBackups() {
loadBackupsButton.disabled = true;
backupSummary.textContent = 'Looking for recovery snapshots…';
try {
const response = await fetch('/api/backups', {method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify({serato: expandHome(document.querySelector('#serato-path').value)})});
const data = await response.json();
if (!response.ok) throw new Error(data.error || 'Could not load backups');
backupSummary.textContent = `${data.total} backup${data.total === 1 ? '' : 's'} · ${formatBytes(data.size_bytes)} on disk`;
backupList.innerHTML = data.items.length ? data.items.map((backup) => `
<article class="backup-item">
<div><span class="backup-status ${backup.status}">${backup.status === 'restored' ? 'Restored' : 'Ready to restore'}</span><strong>${formatDate(backup.created_at)}</strong><small>Kept: ${escapeHtml(backup.keeper)}</small><small>${backup.replaced.length} original file${backup.replaced.length === 1 ? '' : 's'} · ${formatBytes(backup.size_bytes)}</small></div>
<button type="button" data-restore-backup="${escapeHtml(backup.path)}" ${backup.status === 'restored' ? 'disabled' : ''}>${backup.status === 'restored' ? 'Already restored' : 'Restore'}</button>
</article>
`).join('') : '<div class="empty-detail">No repair backups found for this library yet.</div>';
} catch (error) {
backupSummary.textContent = error.message;
backupList.innerHTML = '';
} finally { loadBackupsButton.disabled = false; }
}
loadBackupsButton.addEventListener('click', loadBackups);
backupList.addEventListener('click', async (event) => {
const button = event.target.closest('[data-restore-backup]');
if (!button || button.disabled) return;
if (!window.confirm('Restore the original duplicate files from this backup? Existing real files will never be overwritten.')) return;
button.disabled = true; button.textContent = 'Restoring…';
try {
const response = await fetch('/api/backups/restore', {method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify({serato: expandHome(document.querySelector('#serato-path').value), backup: button.dataset.restoreBackup})});
const result = await response.json();
if (!response.ok) throw new Error(result.error || 'Restore failed');
await loadBackups();
} catch (error) { backupSummary.textContent = error.message; button.disabled = false; button.textContent = 'Restore'; }
});
function detailLines(item) {
if (item.file_previews) {
return item.file_previews.map((file) => `<li class="file-compare-row"><span>${escapeHtml(file.path)}</span><div><button type="button" data-audio-url="${escapeHtml(file.audio_url)}" data-audio-name="${escapeHtml(file.path)}">▶ Play preview</button><button type="button" data-reveal-token="${escapeHtml(file.reveal_token)}">Show in Finder</button></div></li>`).join('');
}
const lines = [];
if (item.artist || item.title) lines.push(`${item.artist || 'Unknown artist'}${item.title || item.filename}`);
if (item.crate) lines.push(`Crate: ${item.crate}`);
if (item.saved_path) lines.push(`Saved path: ${item.saved_path}`);
if (item.candidate) lines.push(`Candidate: ${item.candidate}`);
if (item.path) lines.push(`File: ${item.path}`);
if (item.target) lines.push(`Target: ${item.target}`);
if (item.score || item.reason) lines.push(`${item.score || 'Match'} · ${item.reason || 'Candidate found'}`);
if (item.repeated_filename) lines.push('Same filename appears more than once in Seratos missing list.');
return lines.map((line) => `<li>${escapeHtml(line)}</li>`).join('');
}
function renderDetail(key) {
const detail = latestAnalysis?.details?.[key];
if (!detail) return;
drillTriggers.forEach((trigger) => {
trigger.classList.toggle('selected', trigger.dataset.detail === key);
});
drilldownTitle.textContent = detail.title;
drilldownCount.textContent = `${detail.total ?? 0} found`;
drilldownSummary.textContent = detail.summary;
repairPanel.hidden = true;
selectedDuplicateGroup = null;
previewedRepair = null;
latestBackup = null;
restoreRepairButton.hidden = true;
if (!detail.items?.length) {
drilldownList.innerHTML = '<div class="empty-detail">Nothing to review here. Tiny victory parade, very tasteful.</div>';
return;
}
if (detail.items[0].file_previews) {
reviewState = {key, index: 0, choices: new Map()};
repairPanel.hidden = false;
renderReviewGroup();
return;
}
drilldownList.innerHTML = detail.items.map((item, index) => `
<article class="detail-item">
<strong>${escapeHtml(item.filename || item.path || 'Untitled item')}</strong>
<ul>${detailLines(item)}</ul>
</article>
`).join('');
}
function renderReviewGroup() {
const groups = latestAnalysis.details[reviewState.key].items;
const group = groups[reviewState.index];
const chosen = reviewState.choices.get(reviewState.index);
drilldownCount.textContent = `${reviewState.index + 1} of ${groups.length} · ${reviewState.choices.size} approved`;
drilldownSummary.textContent = 'Listen to each candidate, choose the keeper, and well move to the next group. Skip anything uncertain.';
drilldownList.innerHTML = `
<article class="review-workspace">
<div class="review-heading"><div><span>Comparing now</span><strong>${escapeHtml(group.filename)}</strong></div><span>${reviewState.choices.size} selected</span></div>
<div class="review-candidates">${group.file_previews.map((file, index) => `
<section class="review-candidate ${chosen === file.path ? 'winner' : ''}">
<span class="candidate-number">Option ${index + 1}</span>
<strong>${escapeHtml(file.path.split('/').pop())}</strong>
<small>${escapeHtml(file.path)}</small>
<div class="file-actions"><button type="button" data-audio-url="${escapeHtml(file.audio_url)}" data-audio-name="${escapeHtml(file.path)}">▶ Play preview</button><button type="button" data-reveal-token="${escapeHtml(file.reveal_token)}">Show in Finder</button></div>
<button class="choose-winner" type="button" data-choose-winner="${index}">${chosen === file.path ? '✓ Selected keeper' : 'Keep this one →'}</button>
</section>
`).join('')}</div>
<div class="review-navigation"><button type="button" data-review-previous ${reviewState.index === 0 ? 'disabled' : ''}>← Previous</button><button type="button" data-review-skip>Skip for now</button></div>
</article>`;
updateBatchSummary();
}
function updateBatchSummary() {
if (!reviewState) return;
const groups = latestAnalysis.details[reviewState.key].items;
repairChoice.innerHTML = `<div class="batch-summary"><strong>${reviewState.choices.size} group${reviewState.choices.size === 1 ? '' : 's'} approved</strong><span>${groups.length - reviewState.choices.size} skipped or still awaiting a decision</span></div>`;
previewRepairButton.disabled = reviewState.choices.size === 0;
previewedRepair = null;
applyRepairButton.disabled = true;
repairPreview.hidden = true;
}
function repairPayload() {
if (!reviewState?.choices.size) throw new Error('Choose at least one keeper');
const groups = latestAnalysis.details[reviewState.key].items;
const choices = Array.from(reviewState.choices, ([index, keeper]) => ({keeper, group_files: groups[index].files}));
return {serato: expandHome(document.querySelector('#serato-path').value), music: expandHome(document.querySelector('#music-path').value), choices, backup_limit: document.querySelector('#keep-all-backups').checked ? null : Number(document.querySelector('#backup-limit').value)};
}
async function requestRepair(endpoint) {
const response = await fetch(endpoint, {method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify(repairPayload())});
const data = await response.json();
if (!response.ok) throw new Error(data.error || 'Duplicate cleanup failed');
return data;
}
previewRepairButton.addEventListener('click', async () => {
repairMessage.textContent = 'Checking the plan…'; applyRepairButton.disabled = true;
try {
previewedRepair = await requestRepair('/api/duplicates/batch/preview');
batchPreviewSummary.textContent = `${previewedRepair.choice_count} keeper decision(s) · ${previewedRepair.replaced.length} duplicate file(s) consolidated`;
batchPreviewList.innerHTML = previewedRepair.decisions.map((decision, index) => `
<article class="preview-decision"><span>Decision ${index + 1}</span><div class="winner-path"><b>Keep</b><strong>${escapeHtml(decision.keeper.split('/').pop())}</strong><small>${escapeHtml(decision.keeper)}</small></div><div class="replaced-paths"><b>Replace with a shortcut</b>${decision.replaced.map((path) => `<small>${escapeHtml(path)}</small>`).join('')}</div></article>
`).join('');
batchPreviewSafety.textContent = `${previewedRepair.metadata_backups} Serato metadata file(s) and every replaced audio file will be backed up. Database V2 will not be modified.`;
repairPreview.innerHTML = `<strong>Preview approved</strong><p>${previewedRepair.choice_count} keeper decision(s) are ready for one backed-up apply.</p>`;
repairPreview.hidden = false; applyRepairButton.disabled = false;
repairMessage.textContent = 'Preview complete. Nothing has changed yet.';
batchPreviewModal.showModal();
} catch (error) { repairMessage.textContent = error.message; }
});
function dismissBatchPreview() { batchPreviewModal.close(); }
closeBatchPreview.addEventListener('click', dismissBatchPreview);
continueReviewing.addEventListener('click', dismissBatchPreview);
acceptPreview.addEventListener('click', dismissBatchPreview);
applyRepairButton.addEventListener('click', async () => {
if (!previewedRepair) return;
applyRepairButton.disabled = true; repairMessage.textContent = 'Creating the backup before making changes…';
try {
const result = await requestRepair('/api/duplicates/batch/apply');
latestBackup = result.backup;
repairMessage.textContent = `Cleanup complete. Restore backup: ${result.backup}`;
previewRepairButton.disabled = true;
restoreRepairButton.hidden = false;
await loadBackups();
} catch (error) { repairMessage.textContent = error.message; applyRepairButton.disabled = false; }
});
restoreRepairButton.addEventListener('click', async () => {
if (!latestBackup) return;
restoreRepairButton.disabled = true; repairMessage.textContent = 'Restoring the duplicate files…';
try {
const response = await fetch('/api/backups/restore', {method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify({serato: expandHome(document.querySelector('#serato-path').value), backup: latestBackup})});
const result = await response.json();
if (!response.ok) throw new Error(result.error || 'Restore failed');
repairMessage.textContent = `Restore complete. ${result.restored.length} original file(s) returned.`;
restoreRepairButton.hidden = true;
await loadBackups();
} catch (error) { repairMessage.textContent = error.message; restoreRepairButton.disabled = false; }
});
async function handleFileAction(event) {
const previewButton = event.target.closest('[data-audio-url]');
const revealButton = event.target.closest('[data-reveal-token]');
if (!previewButton && !revealButton) return false;
event.preventDefault(); event.stopPropagation();
if (previewButton) {
audioPlayer.src = previewButton.dataset.audioUrl;
audioPreviewName.textContent = previewButton.dataset.audioName.split('/').pop();
audioPreview.hidden = false;
try { await audioPlayer.play(); } catch (_) { /* Native controls remain available. */ }
} else {
const original = revealButton.textContent;
revealButton.disabled = true; revealButton.textContent = 'Opening…';
try {
const response = await fetch('/api/reveal', {method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify({token: revealButton.dataset.revealToken})});
const result = await response.json();
if (!response.ok) throw new Error(result.error || 'Could not open Finder');
revealButton.textContent = 'Shown in Finder';
} catch (error) { revealButton.textContent = error.message; }
setTimeout(() => { revealButton.disabled = false; revealButton.textContent = original; }, 1800);
}
return true;
}
closeAudioPreview.addEventListener('click', () => { audioPlayer.pause(); audioPlayer.removeAttribute('src'); audioPlayer.load(); audioPreview.hidden = true; });
repairChoice.addEventListener('click', handleFileAction);
drilldownList.addEventListener('click', async (event) => {
if (await handleFileAction(event) || !reviewState) return;
const groups = latestAnalysis.details[reviewState.key].items;
const winner = event.target.closest('[data-choose-winner]');
if (winner) {
const fileIndex = Number(winner.dataset.chooseWinner);
reviewState.choices.set(reviewState.index, groups[reviewState.index].files[fileIndex]);
if (reviewState.index < groups.length - 1) reviewState.index += 1;
renderReviewGroup();
return;
}
if (event.target.closest('[data-review-skip]')) {
reviewState.choices.delete(reviewState.index);
if (reviewState.index < groups.length - 1) reviewState.index += 1;
renderReviewGroup();
return;
}
if (event.target.closest('[data-review-previous]') && reviewState.index > 0) {
reviewState.index -= 1;
renderReviewGroup();
}
});
function render(data) {
latestAnalysis = data;
document.querySelectorAll('[data-field]').forEach((element) => {
const value = data[element.dataset.field];
element.textContent = value ?? '—';
@@ -20,10 +304,45 @@ function render(data) {
: score >= 95 ? 'Looking excellent' : score >= 80 ? 'A few things need attention' : 'Review recommended';
document.querySelector('#score-basis').textContent = data.score_basis;
document.querySelector('#analysis-time').textContent = `Completed ${new Date().toLocaleTimeString([], {hour: '2-digit', minute: '2-digit'})}`;
renderDetail(data.database_missing_paths > 0 ? 'database_missing_tracks' : 'old_crate_references');
results.hidden = false;
results.scrollIntoView({behavior: 'smooth', block: 'start'});
}
function closeInfoButtons(except = null) {
infoButtons.forEach((infoButton) => {
if (infoButton !== except) {
infoButton.classList.remove('open');
infoButton.setAttribute('aria-expanded', 'false');
}
});
}
infoButtons.forEach((infoButton) => {
infoButton.addEventListener('click', (event) => {
event.stopPropagation();
const willOpen = !infoButton.classList.contains('open');
closeInfoButtons(infoButton);
infoButton.classList.toggle('open', willOpen);
infoButton.setAttribute('aria-expanded', String(willOpen));
});
});
drillTriggers.forEach((trigger) => {
trigger.addEventListener('click', () => renderDetail(trigger.dataset.detail));
trigger.addEventListener('keydown', (event) => {
if (event.key === 'Enter' || event.key === ' ') {
event.preventDefault();
renderDetail(trigger.dataset.detail);
}
});
});
document.addEventListener('click', () => closeInfoButtons());
document.addEventListener('keydown', (event) => {
if (event.key === 'Escape') closeInfoButtons();
});
form.addEventListener('submit', async (event) => {
event.preventDefault();
errorBox.hidden = true;
+59 -11
View File
@@ -5,7 +5,9 @@
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="color-scheme" content="dark">
<title>Serato Doctor</title>
<link rel="stylesheet" href="/app.css">
<link rel="stylesheet" href="/app.css?v=3">
<link rel="stylesheet" href="/recovery.css?v=3">
<link rel="stylesheet" href="/layout-fixes.css?v=3">
</head>
<body>
<div class="ambient ambient-one"></div>
@@ -20,10 +22,11 @@
<a class="nav-item active" href="#dashboard"><span></span> Dashboard</a>
<a class="nav-item" href="#scan"><span></span> New analysis</a>
<a class="nav-item" href="#diagnostics"><span></span> Diagnostics</a>
<a class="nav-item" href="#recovery"><span></span> Recovery</a>
</nav>
<div class="safety-card">
<span class="safety-icon"></span>
<div><strong>Read-only mode</strong><p>Your library will not be modified.</p></div>
<div><strong>Protected mode</strong><p>Analysis is read-only. Repairs require a preview and backup.</p></div>
</div>
<div class="sidebar-foot">Local interface · v0.1</div>
</aside>
@@ -59,29 +62,74 @@
<div class="section-heading"><div><p class="eyebrow">Latest analysis</p><h2>Library health</h2></div><span id="analysis-time"></span></div>
<div class="hero-grid">
<article class="score-card panel">
<button class="info-button" type="button" aria-label="About library health" aria-expanded="false" data-info="Your health score is the percentage of saved, non-smart crate entries whose filenames were found in the selected music folder. Smart crates are left out because Serato rebuilds them from rules.">i</button>
<div class="score-ring" id="score-ring"><div><strong id="health-score"></strong><span>health</span></div></div>
<div><p class="score-label">Reference integrity</p><h3 id="health-message">Ready to analyze</h3><p id="score-basis">We only score evidence we can defend.</p></div>
</article>
<div class="metrics-grid">
<article class="metric panel"><span>Tracks</span><strong data-field="disk_tracks"></strong><small>audio files found</small></article>
<article class="metric panel warning"><span>Broken references</span><strong data-field="missing_references"></strong><small>static crate entries</small></article>
<article class="metric panel"><span>Unused tracks</span><strong data-field="unused_tracks"></strong><small>not referenced by crates</small></article>
<article class="metric panel"><span>Suggested matches</span><strong data-field="suggested_matches"></strong><small>explainable candidates</small></article>
<article class="metric panel"><button class="info-button" type="button" aria-label="About tracks" aria-expanded="false" data-info="Audio files found inside the music folder you selected. This is the collection Serato Doctor compared with your crates and database.">i</button><span>Tracks scanned</span><strong data-field="disk_tracks"></strong><small>audio files found</small></article>
<article class="metric panel warning drill-trigger" role="button" tabindex="0" data-detail="database_missing_tracks"><button class="info-button" type="button" aria-label="About missing tracks in Serato" aria-expanded="false" data-info="Tracks in Serato's database whose saved file location no longer exists. This should be close to the orange or unmapped track count you see in Serato.">i</button><span>Missing tracks in Serato</span><strong data-field="database_missing_paths"></strong><small><b data-field="database_missing_unique_filenames"></b> unique filenames · click for list</small></article>
<article class="metric panel drill-trigger" role="button" tabindex="0" data-detail="unused_tracks"><button class="info-button" type="button" aria-label="About unused tracks" aria-expanded="false" data-info="Files in the selected music folder whose filename is not used by any loaded crate. They may still be valid library tracks; this is informational, not a deletion recommendation.">i</button><span>Unused tracks</span><strong data-field="unused_tracks"></strong><small>not referenced by crates · click for list</small></article>
<article class="metric panel drill-trigger" role="button" tabindex="0" data-detail="suggested_matches"><button class="info-button" type="button" aria-label="About suggested matches" aria-expanded="false" data-info="Missing crate entries with a filename-related candidate, such as an added OneDrive conflict number. Suggestions are evidence for review, never automatic repairs.">i</button><span>Suggested matches</span><strong data-field="suggested_matches"></strong><small>explainable candidates · click for list</small></article>
</div>
</div>
<div id="diagnostics" class="diagnostics panel">
<div class="section-heading"><div><p class="eyebrow">Full picture</p><h2>Diagnostics</h2></div><span class="read-only-tag">No changes made</span></div>
<div class="diagnostic-list">
<div><span class="diag-icon violet"></span><p><strong>Crates</strong><small><b data-field="static_crates"></b> static · <b data-field="smart_crates"></b> smart · <b data-field="dynamic_references_excluded"></b> dynamic references excluded</small></p></div>
<div><span class="diag-icon amber"></span><p><strong>Duplicate filenames</strong><small><b data-field="duplicate_filename_groups"></b> exact groups · <b data-field="duplicate_files"></b> extra files</small></p></div>
<div><span class="diag-icon blue"></span><p><strong>Cloud conflicts</strong><small><b data-field="suspected_cloud_conflict_groups"></b> suspected groups · <b data-field="suspected_cloud_conflict_files"></b> extra files</small></p></div>
<div><span class="diag-icon red"></span><p><strong>Broken symlinks</strong><small><b data-field="broken_symlinks"></b> unresolved links</small></p></div>
<div><span class="diag-icon violet"></span><p><strong>Crates</strong><small><b data-field="static_crates"></b> regular · <b data-field="smart_crates"></b> smart · <b data-field="smart_crate_containers"></b> dynamic containers</small></p><button class="info-button" type="button" aria-label="About crates" aria-expanded="false" data-info="Regular crates are lists you maintain by hand. Smart crates are rebuilt by Serato from rules, so their generated references are not scored as broken.">i</button></div>
<div class="drill-trigger" role="button" tabindex="0" data-detail="duplicate_filenames"><span class="diag-icon amber"></span><p><strong>Duplicate filenames</strong><small><b data-field="duplicate_filename_groups"></b> exact groups · <b data-field="duplicate_files"></b> extra files · click for list</small></p><button class="info-button" type="button" aria-label="About duplicate filenames" aria-expanded="false" data-info="Different files with the same filename after case and Unicode cleanup. They need review, but matching names alone do not mean either file should be deleted.">i</button></div>
<div class="drill-trigger" role="button" tabindex="0" data-detail="cloud_conflicts"><span class="diag-icon blue"></span><p><strong>Possible cloud conflicts</strong><small><b data-field="suspected_cloud_conflict_groups"></b> groups · <b data-field="suspected_cloud_conflict_files"></b> extra files · click for list</small></p><button class="info-button" type="button" aria-label="About cloud conflicts" aria-expanded="false" data-info="Filename families such as Track.mp3 and Track 2.mp3. OneDrive often creates these during sync conflicts, but numbered song titles can also be legitimate.">i</button></div>
<div class="drill-trigger" role="button" tabindex="0" data-detail="broken_symlinks"><span class="diag-icon red"></span><p><strong>Broken shortcuts</strong><small><b data-field="broken_symlinks"></b> unresolved symbolic links · click for list</small></p><button class="info-button" type="button" aria-label="About broken shortcuts" aria-expanded="false" data-info="Shortcut-style symbolic links whose destination no longer exists. Serato Doctor reports them but never removes or recreates them automatically.">i</button></div>
<div class="drill-trigger" role="button" tabindex="0" data-detail="old_crate_references"><span class="diag-icon violet"></span><p><strong>Old crate references</strong><small><b data-field="missing_references"></b> appearances · <b data-field="unique_missing_filenames"></b> unique filenames · click for list</small></p><button class="info-button" type="button" aria-label="About old crate references" aria-expanded="false" data-info="Saved spots in regular crates whose exact filename was not found in the selected music folder. The same track can appear in several crates, so appearances are higher than unique filenames. These are separate from Serato's unmapped-track count.">i</button></div>
<div><span class="diag-icon amber"></span><p><strong>Database coverage</strong><small><b data-field="database_entries"></b> Serato entries · <b data-field="tracks_missing_from_database"></b> scanned tracks absent</small></p><button class="info-button" type="button" aria-label="About database coverage" aria-expanded="false" data-info="Compares filenames in Serato's database with the selected music folder. A scanned track absent from the database may not have been imported, or may be represented under another filename.">i</button></div>
</div>
</div>
<div id="drilldowns" class="drilldowns panel">
<div class="section-heading"><div><p class="eyebrow">Look closer</p><h2 id="drilldown-title">Choose a diagnostic</h2></div><span id="drilldown-count">Read-only examples</span></div>
<p id="drilldown-summary">Click a metric above to see example files and saved paths behind that number.</p>
<div id="drilldown-list" class="detail-list"></div>
</div>
<div id="duplicate-repair" class="repair-panel panel" hidden>
<div class="section-heading"><div><p class="eyebrow">Batch cleanup</p><h2>Review all selected changes</h2></div><span class="repair-tag">One backup · one apply</span></div>
<p>Approved choices are combined into one plan. Skipped groups remain untouched, and Seratos database V2 is never edited.</p>
<div id="repair-choice" class="repair-choice"></div>
<div class="backup-options">
<label>Backups to keep <input id="backup-limit" type="number" min="1" value="10"></label>
<label class="check-label"><input id="keep-all-backups" type="checkbox"> Keep every backup</label>
</div>
<div id="repair-preview" class="repair-preview" hidden></div>
<div class="repair-actions">
<button id="preview-repair" type="button">Preview all selected changes</button>
<button id="apply-repair" class="danger-action" type="button" disabled>Apply all selected changes</button>
<button id="restore-repair" type="button" hidden>Restore this backup</button>
</div>
<div id="repair-message" class="repair-message" role="status"></div>
</div>
</section>
<section id="recovery" class="recovery-panel panel">
<div class="section-heading"><div><p class="eyebrow">Safety net</p><h2>Backup recovery</h2></div><button id="load-backups" type="button">Check backups</button></div>
<p>Repair backups appear here automatically before any duplicate is consolidated. Review saved snapshots, disk usage, and restore history.</p>
<div id="backup-summary" class="backup-summary">Analyze your library, then choose a duplicate to preview its automatic backup.</div>
<div id="backup-list" class="backup-list"></div>
</section>
<div id="audio-preview" class="audio-preview" hidden>
<div><span>Now previewing</span><strong id="audio-preview-name"></strong></div>
<audio id="audio-player" controls preload="metadata"></audio>
<button id="close-audio-preview" type="button" aria-label="Close audio preview">×</button>
</div>
<dialog id="batch-preview-modal" class="batch-preview-modal">
<div class="modal-heading"><div><p class="eyebrow">Final dry-run</p><h2>Heres exactly what will happen</h2></div><button id="close-batch-preview" type="button" aria-label="Close preview">×</button></div>
<p id="batch-preview-summary"></p>
<div id="batch-preview-list" class="batch-preview-list"></div>
<div class="modal-safety"><span></span><p><strong>Backup happens first</strong><small id="batch-preview-safety"></small></p></div>
<div class="modal-actions"><button id="continue-reviewing" type="button">Close and continue reviewing</button><button id="accept-preview" type="button">Looks right</button></div>
</dialog>
</main>
</div>
<script src="/app.js" defer></script>
<script src="/app.js?v=6" defer></script>
</body>
</html>
+407
View File
@@ -0,0 +1,407 @@
.recovery-panel {
margin-top: 42px;
}
@media (max-width: 680px) {
.recovery-panel {
margin-top: 28px;
}
}
.review-workspace {
padding: clamp(16px, 3vw, 24px);
border: 1px solid rgba(155, 135, 245, .18);
border-radius: 16px;
background: rgba(255, 255, 255, .02);
}
.review-heading,
.review-navigation {
display: flex;
align-items: center;
justify-content: space-between;
gap: 16px;
}
.review-heading span,
.batch-summary span {
color: var(--muted);
font-size: 10px;
}
.review-heading strong {
display: block;
margin-top: 5px;
font-size: 14px;
}
.review-candidates {
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: 12px;
margin: 18px 0;
}
.review-candidate {
display: flex;
flex-direction: column;
gap: 9px;
min-width: 0;
padding: 16px;
border: 1px solid var(--line);
border-radius: 14px;
background: rgba(6, 8, 12, .28);
}
.review-candidate.winner {
border-color: rgba(84, 212, 154, .52);
background: rgba(84, 212, 154, .06);
}
.candidate-number {
color: var(--violet);
font-size: 9px;
font-weight: 750;
text-transform: uppercase;
letter-spacing: .1em;
}
.review-candidate > strong,
.review-candidate > small {
overflow-wrap: anywhere;
}
.review-candidate > strong {
font-size: 12px;
}
.review-candidate > small {
flex: 1;
color: var(--muted);
font-size: 9px;
line-height: 1.5;
}
.choose-winner {
margin-top: 4px;
border: 0;
border-radius: 10px;
padding: 10px 12px;
background: linear-gradient(135deg, #907be9, #6e59cf);
color: white;
font: 750 11px/1 inherit;
cursor: pointer;
}
.winner .choose-winner {
background: rgba(84, 212, 154, .18);
color: #8ce4b8;
}
.review-navigation button {
border: 1px solid var(--line);
border-radius: 9px;
padding: 8px 11px;
background: rgba(255, 255, 255, .04);
color: var(--muted);
font: 700 10px/1 inherit;
cursor: pointer;
}
.review-navigation button:disabled {
opacity: .35;
}
.batch-summary {
display: flex;
align-items: center;
justify-content: space-between;
gap: 12px;
padding: 14px;
border: 1px solid rgba(102, 217, 232, .2);
border-radius: 12px;
background: rgba(102, 217, 232, .05);
}
.batch-summary strong {
font-size: 12px;
}
@media (max-width: 760px) {
.review-candidates {
grid-template-columns: 1fr;
}
.review-heading,
.batch-summary {
align-items: flex-start;
flex-direction: column;
}
}
.batch-preview-modal {
width: min(820px, calc(100vw - 32px));
max-height: min(82vh, 760px);
padding: 24px;
overflow: auto;
border: 1px solid rgba(155, 135, 245, .3);
border-radius: 20px;
background: #161923;
color: var(--text);
box-shadow: 0 30px 100px rgba(0, 0, 0, .65);
}
.batch-preview-modal::backdrop {
background: rgba(4, 5, 8, .78);
backdrop-filter: blur(7px);
}
.modal-heading,
.modal-actions,
.modal-safety {
display: flex;
align-items: center;
justify-content: space-between;
gap: 14px;
}
.modal-heading h2 {
margin: 0;
font-size: 23px;
letter-spacing: -.03em;
}
.modal-heading > button {
border: 0;
background: transparent;
color: var(--muted);
font-size: 28px;
cursor: pointer;
}
#batch-preview-summary {
color: var(--muted);
font-size: 12px;
}
.batch-preview-list {
display: grid;
gap: 10px;
max-height: 390px;
margin: 18px 0;
padding-right: 4px;
overflow: auto;
}
.preview-decision {
display: grid;
grid-template-columns: 80px 1fr 1fr;
gap: 13px;
padding: 14px;
border: 1px solid var(--line);
border-radius: 13px;
background: rgba(255, 255, 255, .025);
}
.preview-decision > span,
.preview-decision b {
color: var(--muted);
font-size: 9px;
text-transform: uppercase;
letter-spacing: .08em;
}
.winner-path,
.replaced-paths {
min-width: 0;
}
.winner-path strong,
.winner-path small,
.replaced-paths small {
display: block;
margin-top: 5px;
overflow-wrap: anywhere;
}
.winner-path strong {
color: #8ce4b8;
font-size: 11px;
}
.winner-path small,
.replaced-paths small {
color: var(--muted);
font-size: 9px;
line-height: 1.45;
}
.modal-safety {
justify-content: flex-start;
padding: 13px;
border: 1px solid rgba(102, 217, 232, .2);
border-radius: 12px;
background: rgba(102, 217, 232, .05);
}
.modal-safety > span {
color: var(--cyan);
}
.modal-safety p,
.modal-safety strong,
.modal-safety small {
display: block;
margin: 0;
}
.modal-safety strong {
font-size: 11px;
}
.modal-safety small {
margin-top: 4px;
color: var(--muted);
font-size: 9px;
line-height: 1.45;
}
.modal-actions {
justify-content: flex-end;
margin-top: 16px;
}
.modal-actions button {
border: 1px solid var(--line);
border-radius: 10px;
padding: 10px 13px;
background: rgba(255, 255, 255, .04);
color: var(--text);
font: 700 11px/1 inherit;
cursor: pointer;
}
#accept-preview {
border-color: transparent;
background: #6f5bd0;
}
@media (max-width: 620px) {
.preview-decision {
grid-template-columns: 1fr;
}
.modal-actions {
align-items: stretch;
flex-direction: column-reverse;
}
}
.file-compare-row {
display: grid;
gap: 8px;
padding: 9px 0;
border-top: 1px solid rgba(255, 255, 255, .05);
}
.file-compare-row:first-child {
border-top: 0;
}
.file-compare-row div,
.file-actions {
display: flex;
flex-wrap: wrap;
gap: 7px;
}
.file-compare-row button,
.file-actions button {
border: 1px solid rgba(155, 135, 245, .25);
border-radius: 8px;
padding: 7px 9px;
background: rgba(155, 135, 245, .08);
color: #d8d2f6;
font: 700 10px/1 inherit;
cursor: pointer;
}
.keeper-option {
justify-content: space-between;
}
.keeper-option > label {
display: flex;
align-items: center;
gap: 12px;
min-width: 0;
cursor: pointer;
}
.audio-preview {
position: fixed;
right: 24px;
bottom: 20px;
display: grid;
grid-template-columns: minmax(150px, .7fr) minmax(240px, 1.3fr) auto;
align-items: center;
gap: 16px;
width: min(720px, calc(100vw - 48px));
padding: 13px 14px;
border: 1px solid rgba(102, 217, 232, .3);
border-radius: 14px;
background: rgba(16, 21, 29, .96);
box-shadow: 0 16px 48px rgba(0, 0, 0, .45);
z-index: 5;
}
.audio-preview[hidden] {
display: none;
}
.audio-preview span,
.audio-preview strong {
display: block;
}
.audio-preview span {
color: var(--cyan);
font-size: 9px;
text-transform: uppercase;
letter-spacing: .1em;
}
.audio-preview strong {
max-width: 360px;
margin-top: 4px;
overflow: hidden;
color: var(--text);
font-size: 11px;
text-overflow: ellipsis;
white-space: nowrap;
}
.audio-preview audio {
width: 100%;
height: 34px;
}
.audio-preview > button {
border: 0;
background: transparent;
color: var(--muted);
font-size: 22px;
cursor: pointer;
}
@media (max-width: 680px) {
.keeper-option,
.audio-preview {
align-items: stretch;
grid-template-columns: 1fr;
}
.keeper-option {
flex-direction: column;
}
}
+1
View File
@@ -0,0 +1 @@
.recovery-panel{margin-top:18px;padding:25px}.recovery-panel>p{color:var(--muted);font-size:12px;line-height:1.6}.recovery-panel .section-heading button{border:1px solid rgba(155,135,245,.35);border-radius:10px;padding:9px 12px;background:rgba(155,135,245,.12);color:#ddd8fa;font:700 11px inherit;cursor:pointer}.recovery-panel button:disabled{opacity:.5;cursor:not-allowed}.backup-summary{padding:11px 13px;border-radius:11px;background:rgba(255,255,255,.03);color:#bfc4cf;font-size:11px;margin:15px 0 10px}.backup-list{display:grid;gap:9px}.backup-item{display:flex;align-items:center;justify-content:space-between;gap:18px;padding:14px;border:1px solid var(--line);border-radius:13px;background:rgba(255,255,255,.025)}.backup-item>div{min-width:0}.backup-item strong,.backup-item small{display:block}.backup-item strong{font-size:12px;margin:7px 0}.backup-item small{font-size:10px;color:var(--muted);line-height:1.5;overflow-wrap:anywhere}.backup-item>button{flex:0 0 auto;border:0;border-radius:10px;padding:10px 13px;background:#6f5bd0;color:#fff;font:700 11px inherit;cursor:pointer}.backup-status{display:inline-block;padding:4px 7px;border-radius:999px;font-size:9px;text-transform:uppercase;letter-spacing:.08em;color:var(--cyan);background:rgba(102,217,232,.1)}.backup-status.restored{color:#aeb4c2;background:rgba(255,255,255,.06)}@media(max-width:520px){.backup-item{align-items:stretch;flex-direction:column}.backup-item>button{width:100%}}
+34 -1
View File
@@ -6,6 +6,7 @@ from serato_doctor.crate_parser import (
classify_crate,
clean_path,
load_crate,
load_library_crates,
parse_crate,
parse_crates,
path_markers,
@@ -92,7 +93,7 @@ def test_parse_crates_reuses_configured_roots_for_every_crate(tmp_path):
("relative_path", "expected"),
[
("_Serato_/Subcrates/House.crate", CrateKind.STATIC),
("_Serato_/Smartcrates/Warmup.crate", CrateKind.SMART),
("_Serato_/SmartCrates/Warmup.scrate", CrateKind.SMART),
("_Serato_/Subcrates/Compatible by key.crate", CrateKind.SMART),
("fixtures/Unknown.crate", CrateKind.UNKNOWN),
],
@@ -101,3 +102,35 @@ def test_classify_crate_uses_provenance_and_known_dynamic_name(
tmp_path, relative_path, expected
):
assert classify_crate(tmp_path / relative_path) is expected
def test_smart_crate_preserves_encoded_hierarchy(tmp_path):
crate_path = (
tmp_path / "_Serato_" / "SmartCrates" / "Compatible by key≫≫10A.scrate"
)
crate_path.parent.mkdir(parents=True)
crate_path.write_bytes(b"")
crate = load_crate(crate_path)
assert crate.kind is CrateKind.SMART
assert crate.hierarchy == ("Compatible by key", "10A")
assert crate.display_name == "10A"
assert crate.is_smart_definition
def test_load_library_crates_discovers_scrate_definitions(tmp_path):
smart_folder = tmp_path / "SmartCrates"
static_folder = tmp_path / "Subcrates"
smart_folder.mkdir()
static_folder.mkdir()
(smart_folder / "New EDM.scrate").write_bytes(b"")
(smart_folder / "Re-Drums.scrate").write_bytes(b"")
(static_folder / "House.crate").write_bytes(b"")
crates = load_library_crates(tmp_path)
assert [(crate.path.name, crate.kind) for crate in crates] == [
("New EDM.scrate", CrateKind.SMART),
("Re-Drums.scrate", CrateKind.SMART),
("House.crate", CrateKind.STATIC),
]
+48
View File
@@ -0,0 +1,48 @@
from pathlib import Path
from serato_doctor.database_parser import iter_records, parse_database
def record(tag, payload):
return tag + len(payload).to_bytes(4, "big") + payload
def text_record(tag, value):
return record(tag, value.encode("utf-16-be"))
def test_parse_database_reads_track_paths_and_metadata(tmp_path):
track = b"".join(
[
text_record(b"pfil", "Users/sample/Music/Track.mp3"),
text_record(b"tsng", "Track title"),
text_record(b"tart", "Test artist"),
text_record(b"talb", "Test album"),
text_record(b"tgen", "House"),
]
)
database_path = tmp_path / "database V2"
database_path.write_bytes(
text_record(b"vrsn", "2.0/Test Database") + record(b"otrk", track)
)
database = parse_database(database_path)
assert database.version == "2.0/Test Database"
assert len(database.tracks) == 1
parsed = database.tracks[0]
assert parsed.path == Path("/Users/sample/Music/Track.mp3")
assert parsed.filename == "Track.mp3"
assert parsed.title == "Track title"
assert parsed.artist == "Test artist"
assert parsed.album == "Test album"
assert parsed.genre == "House"
def test_iter_records_ignores_incomplete_trailing_record():
complete = record(b"vrsn", "2.0".encode("utf-16-be"))
incomplete = b"otrk\x00\x00\x00\x10short"
records = list(iter_records(complete + incomplete))
assert records == [(b"vrsn", "2.0".encode("utf-16-be"))]
+5 -2
View File
@@ -58,19 +58,21 @@ def test_empty_library_has_no_health_score():
assert report.score is None
assert report.total_references == 0
assert report.scored_references == 0
assert not report.database_present
assert report.tracks_missing_from_database == 0
def test_smart_crate_references_are_reported_but_not_scored():
static_reference = reference("Found.mp3")
smart_reference = TrackReference(
Path("Smartcrates/Dynamic.crate"),
Path("SmartCrates/Dynamic.scrate"),
Path("/old/House/Dynamic.mp3"),
"Dynamic.mp3",
)
crates = [
Crate(Path("Subcrates/Static.crate"), (static_reference,), CrateKind.STATIC),
Crate(
Path("Smartcrates/Dynamic.crate"),
Path("SmartCrates/Dynamic.scrate"),
(smart_reference,),
CrateKind.SMART,
),
@@ -85,6 +87,7 @@ def test_smart_crate_references_are_reported_but_not_scored():
assert report.missing_references == 0
assert report.static_crates == 1
assert report.smart_crates == 1
assert report.smart_crate_containers == 0
assert report.dynamic_references_excluded == 1
+101
View File
@@ -0,0 +1,101 @@
from pathlib import Path
import pytest
from serato_doctor.repair import (
BACKUP_FOLDER,
apply_duplicate_repair,
apply_duplicate_repair_batch,
list_backups,
plan_duplicate_repair,
restore_backup,
rotate_backups,
)
def library(tmp_path):
serato = tmp_path / "_Serato_"
crate = serato / "Subcrates" / "House.crate"
crate.parent.mkdir(parents=True)
crate.write_bytes(b"crate-data")
(serato / "database V2").write_bytes(b"database-data")
music = tmp_path / "Music"
keeper = music / "Main" / "Track.mp3"
duplicate = music / "Old" / "Track.mp3"
keeper.parent.mkdir(parents=True)
duplicate.parent.mkdir(parents=True)
keeper.write_bytes(b"keeper")
duplicate.write_bytes(b"duplicate")
return serato, keeper, duplicate
def test_duplicate_repair_backs_up_then_preserves_old_path_as_link(tmp_path):
serato, keeper, duplicate = library(tmp_path)
plan = plan_duplicate_repair(keeper, (keeper, duplicate), serato)
receipt = apply_duplicate_repair(plan, serato, backup_limit=3)
assert duplicate.is_symlink()
assert duplicate.resolve() == keeper.resolve()
assert (receipt.backup / "manifest.json").is_file()
assert any((receipt.backup / "serato-metadata").rglob("House.crate"))
assert any((receipt.backup / "serato-metadata").rglob("database V2"))
history = list_backups(serato)
assert len(history) == 1
assert history[0].status == "ready"
assert history[0].size_bytes > 0
restored = restore_backup(receipt.backup)
assert restored == (duplicate.resolve(),)
assert not duplicate.is_symlink()
assert duplicate.read_bytes() == b"duplicate"
assert list_backups(serato)[0].status == "restored"
def test_plan_rejects_keeper_outside_duplicate_group(tmp_path):
serato, keeper, duplicate = library(tmp_path)
outsider = tmp_path / "outsider.mp3"
outsider.write_bytes(b"other")
with pytest.raises(ValueError, match="must belong"):
plan_duplicate_repair(outsider, (keeper, duplicate), serato)
def test_backup_rotation_can_be_limited_or_unlimited(tmp_path):
root = tmp_path / BACKUP_FOLDER
for name in ("001", "002", "003"):
backup = root / name
backup.mkdir(parents=True)
(backup / "manifest.json").write_text("{}", encoding="utf-8")
rotate_backups(root, None)
assert len(list(root.iterdir())) == 3
rotate_backups(root, 2)
assert {path.name for path in root.iterdir()} == {"002", "003"}
def test_batch_repair_uses_one_backup_and_restores_every_group(tmp_path):
serato, first_keeper, first_duplicate = library(tmp_path)
second_keeper = tmp_path / "Music" / "Main" / "Other.mp3"
second_duplicate = tmp_path / "Music" / "Old" / "Other.mp3"
second_keeper.write_bytes(b"other-keeper")
second_duplicate.write_bytes(b"other-duplicate")
plans = (
plan_duplicate_repair(
first_keeper, (first_keeper, first_duplicate), serato
),
plan_duplicate_repair(
second_keeper, (second_keeper, second_duplicate), serato
),
)
receipt = apply_duplicate_repair_batch(plans, serato, backup_limit=10)
assert first_duplicate.is_symlink()
assert second_duplicate.is_symlink()
assert len(list((serato / BACKUP_FOLDER).iterdir())) == 1
assert len(receipt.replaced) == 2
restore_backup(receipt.backup)
assert first_duplicate.read_bytes() == b"duplicate"
assert second_duplicate.read_bytes() == b"other-duplicate"
+3 -1
View File
@@ -24,7 +24,9 @@ def test_generated_sample_library_has_expected_scenario(tmp_path):
if not result.exists_by_filename
}
assert len(list((sample_root / "Serato").rglob("*.crate"))) == 7
crate_files = list((sample_root / "Serato").rglob("*.crate"))
smart_files = list((sample_root / "Serato").rglob("*.scrate"))
assert len(crate_files) + len(smart_files) == 7
assert len(library.references) == 13
assert len(tracks) == 10
assert missing == {"Missing.mp3", "Old Name.mp3"}
+114 -2
View File
@@ -3,7 +3,14 @@ from pathlib import Path
import pytest
from serato_doctor.web import STATIC_FILES, analyze_paths
from serato_doctor.web import (
STATIC_FILES,
_file_token,
_verified_audio,
analyze_paths,
duplicate_repair,
duplicate_repair_batch,
)
def test_web_analysis_uses_production_health_pipeline(tmp_path):
@@ -21,6 +28,15 @@ def test_web_analysis_uses_production_health_pipeline(tmp_path):
assert result["missing_references"] == 2
assert result["static_crates"] == 5
assert result["smart_crates"] == 2
assert result["database_entries"] == 10
assert result["database_library_matches"] == 10
assert result["database_missing_paths"] == 0
assert result["database_missing_unique_filenames"] == 0
assert result["tracks_missing_from_database"] == 0
assert result["details"]["old_crate_references"]["total"] == 2
assert result["details"]["old_crate_references"]["items"][0]["crate"]
assert result["details"]["suggested_matches"]["total"] == 0
assert result["details"]["unused_tracks"]["total"] == 1
def test_web_analysis_rejects_missing_folders(tmp_path):
@@ -31,5 +47,101 @@ def test_web_analysis_rejects_missing_folders(tmp_path):
def test_web_static_assets_are_declared_and_packaged():
asset_root = Path(__file__).parents[1] / "serato_doctor" / "webui"
assert set(STATIC_FILES) == {"/", "/app.css", "/app.js"}
assert set(STATIC_FILES) == {
"/",
"/app.css",
"/recovery.css",
"/layout-fixes.css",
"/app.js",
}
assert all((asset_root / filename).is_file() for filename, _ in STATIC_FILES.values())
html = (asset_root / "index.html").read_text(encoding="utf-8")
assert "Missing tracks in Serato" in html
assert "Old crate references" in html
assert "Choose a diagnostic" in html
assert "Backup recovery" in html
assert "Heres exactly what will happen" in html
assert 'id="batch-preview-modal"' in html
assert 'data-detail="database_missing_tracks"' in html
assert 'data-detail="old_crate_references"' in html
assert html.count('class="info-button"') >= 10
def test_duplicate_repair_preview_does_not_change_files(tmp_path):
serato = tmp_path / "_Serato_"
serato.mkdir()
music = tmp_path / "Music"
first = music / "A" / "Track.mp3"
second = music / "B" / "Track.mp3"
first.parent.mkdir(parents=True)
second.parent.mkdir(parents=True)
first.write_bytes(b"first")
second.write_bytes(b"second")
result = duplicate_repair(
serato, music, first, (first, second), backup_limit=10
)
assert result["applied"] is False
assert result["database_v2_modified"] is False
assert second.read_bytes() == b"second"
assert not second.is_symlink()
def test_audio_preview_tokens_only_open_signed_audio(tmp_path):
track = tmp_path / "Track.mp3"
track.write_bytes(b"audio")
token = _file_token(track)
assert _verified_audio(token) == track
with pytest.raises(ValueError, match="Invalid or expired"):
_verified_audio(token + "changed")
def test_duplicate_details_include_preview_and_finder_controls(tmp_path):
serato = tmp_path / "_Serato_"
serato.mkdir()
music = tmp_path / "Music"
first = music / "A" / "Track.mp3"
second = music / "B" / "Track.mp3"
first.parent.mkdir(parents=True)
second.parent.mkdir(parents=True)
first.write_bytes(b"first")
second.write_bytes(b"second")
result = analyze_paths(serato, music)
group = result["details"]["duplicate_filenames"]["items"][0]
assert len(group["file_previews"]) == 2
assert group["file_previews"][0]["audio_url"].startswith("/api/audio?")
assert group["file_previews"][0]["reveal_token"]
def test_batch_preview_combines_approved_groups_without_changes(tmp_path):
serato = tmp_path / "_Serato_"
serato.mkdir()
music = tmp_path / "Music"
choices = []
for filename in ("First.mp3", "Second.mp3"):
keeper = music / "A" / filename
duplicate = music / "B" / filename
keeper.parent.mkdir(parents=True, exist_ok=True)
duplicate.parent.mkdir(parents=True, exist_ok=True)
keeper.write_bytes(b"keeper")
duplicate.write_bytes(b"duplicate")
choices.append(
{"keeper": str(keeper), "group_files": [str(keeper), str(duplicate)]}
)
result = duplicate_repair_batch(
serato, music, choices, backup_limit=10
)
assert result["applied"] is False
assert result["choice_count"] == 2
assert len(result["replaced"]) == 2
assert len(result["decisions"]) == 2
assert result["decisions"][0]["keeper"].endswith("First.mp3")
assert len(result["decisions"][0]["replaced"]) == 1
assert all(not Path(choice["group_files"][1]).is_symlink() for choice in choices)