Compare commits
13 Commits
main
..
e4d5a32de2
| Author | SHA1 | Date | |
|---|---|---|---|
| e4d5a32de2 | |||
| ec671079ba | |||
| 4e85b5912b | |||
| d4fe61cf8c | |||
| c4d08bfe27 | |||
| bb74c14742 | |||
| 71fa64a960 | |||
| 09167e44c7 | |||
| 508eecd0bd | |||
| 577fe1f7a7 | |||
| c3bf111107 | |||
| 826428f380 | |||
| 4ad5b8a574 |
@@ -2,7 +2,9 @@ __pycache__/
|
|||||||
*.pyc
|
*.pyc
|
||||||
.env
|
.env
|
||||||
.venv/
|
.venv/
|
||||||
|
*.egg-info/
|
||||||
reports/
|
reports/
|
||||||
|
samples/small-library/generated/
|
||||||
*.sqlite
|
*.sqlite
|
||||||
*.db
|
*.db
|
||||||
*.csv
|
*.csv
|
||||||
@@ -11,3 +13,6 @@ reports/
|
|||||||
# Never commit personal Serato data
|
# Never commit personal Serato data
|
||||||
database V2
|
database V2
|
||||||
*.crate
|
*.crate
|
||||||
|
|
||||||
|
# macOS
|
||||||
|
.DS_Store
|
||||||
|
|||||||
@@ -0,0 +1,19 @@
|
|||||||
|
# Contributing
|
||||||
|
|
||||||
|
## Branching
|
||||||
|
|
||||||
|
- `main` is stable.
|
||||||
|
- `develop` is the integration branch.
|
||||||
|
- Feature branches use: `feature/<name>`.
|
||||||
|
|
||||||
|
## Safety Rules
|
||||||
|
|
||||||
|
Never commit personal music library files, Serato databases, or crates.
|
||||||
|
|
||||||
|
Never write repair code without dry-run mode, backup plan, and rollback log.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
Install development dependencies with `python3 -m pip install -e '.[dev]'`.
|
||||||
|
|
||||||
|
Run `pytest` before committing. Tests and fixtures must use synthetic library data.
|
||||||
+49
@@ -0,0 +1,49 @@
|
|||||||
|
# Serato Doctor Roadmap
|
||||||
|
|
||||||
|
## v0.1 — Library Inspector
|
||||||
|
|
||||||
|
- [x] Project repository
|
||||||
|
- [x] Filesystem scanner
|
||||||
|
- [x] Serato crate parser
|
||||||
|
- [x] Missing reference CSV report
|
||||||
|
- [x] Grouped missing reference report
|
||||||
|
- [ ] HTML health dashboard
|
||||||
|
- [x] Test suite
|
||||||
|
- [x] Sample library fixtures
|
||||||
|
- [ ] Database V2 read-only parser
|
||||||
|
- [x] Configuration
|
||||||
|
|
||||||
|
## v0.2 — Diagnostics
|
||||||
|
|
||||||
|
- [ ] Duplicate filename detection
|
||||||
|
- [ ] Duplicate audio hash detection
|
||||||
|
- [ ] Broken symlink detection
|
||||||
|
- [ ] Orphaned audio detection
|
||||||
|
- [ ] OneDrive rename detection
|
||||||
|
- [ ] Crate classification: static vs smart/dynamic
|
||||||
|
- [ ] Library health score
|
||||||
|
|
||||||
|
## v0.3 — Safe Repair
|
||||||
|
|
||||||
|
- [ ] Dry-run repair plan
|
||||||
|
- [ ] Backup before repair
|
||||||
|
- [ ] Compatibility symlink creation
|
||||||
|
- [ ] Compatibility copy creation
|
||||||
|
- [ ] Rename repair
|
||||||
|
- [ ] Rollback log
|
||||||
|
|
||||||
|
## v0.4 — Migration Wizard
|
||||||
|
|
||||||
|
- [ ] Move library root
|
||||||
|
- [ ] Cloud provider migration
|
||||||
|
- [ ] External drive migration
|
||||||
|
- [ ] Verify moved library
|
||||||
|
- [ ] Update application references
|
||||||
|
|
||||||
|
## v1.0 — DJ Library Doctor
|
||||||
|
|
||||||
|
- [ ] Desktop UI
|
||||||
|
- [ ] Serato support
|
||||||
|
- [ ] Rekordbox support
|
||||||
|
- [ ] VirtualDJ support
|
||||||
|
- [ ] Engine DJ support
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
# Architecture
|
||||||
|
|
||||||
|
Serato Doctor is designed as a DJ library inspection, repair, and migration platform.
|
||||||
|
|
||||||
|
## Design Principles
|
||||||
|
|
||||||
|
1. Read-only by default.
|
||||||
|
2. Every repair must support preview/dry-run.
|
||||||
|
3. Every repair must create a backup or rollback path.
|
||||||
|
4. Application-specific logic lives in engines.
|
||||||
|
5. Core matching and scanning logic should be application-agnostic.
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
# Case Study: OneDrive Mac Migration
|
||||||
|
|
||||||
|
## Scenario
|
||||||
|
|
||||||
|
A large Serato DJ library was migrated from an older Mac to a newer Mac using OneDrive.
|
||||||
|
|
||||||
|
## Symptoms
|
||||||
|
|
||||||
|
- OneDrive client stuck syncing
|
||||||
|
- Duplicate OneDrive folders
|
||||||
|
- Thousands of files renamed with trailing ` 2`
|
||||||
|
- Serato reported many tracks as missing
|
||||||
|
- Some files existed on disk but still appeared orange in Serato
|
||||||
|
|
||||||
|
## Findings
|
||||||
|
|
||||||
|
- OneDrive sync state was rebuilt successfully
|
||||||
|
- Thousands of orphaned filename conflicts were repaired
|
||||||
|
- Some Serato references were stale database objects, not missing files
|
||||||
|
- Smart/dynamic crates should be classified separately from static user crates
|
||||||
|
|
||||||
|
## Lessons
|
||||||
|
|
||||||
|
- Filesystem health and Serato database health are separate problems
|
||||||
|
- Smart crates should not be treated the same as static crates
|
||||||
|
- Repair tools must be read-only by default and generate a plan before changing anything
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
# Scan Configuration
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
The prototype recognized crate paths by matching one developer's historical
|
||||||
|
OneDrive path. That made otherwise valid libraries invisible to the parser.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
`ScanConfig` owns the paths for a read-only scan. The CLI accepts repeatable
|
||||||
|
`--reference-root` options for roots embedded in crates before a migration. The
|
||||||
|
parser uses those roots as record boundaries. Without explicit roots, it recognizes
|
||||||
|
generic macOS home and mounted-volume paths without embedding a username.
|
||||||
|
|
||||||
|
## Edge Cases
|
||||||
|
|
||||||
|
- A library may have references from more than one historical root.
|
||||||
|
- Root paths may contain spaces or have leading/trailing separators.
|
||||||
|
- Existing `/Users/...` and `/Volumes/...` crates must work without new flags.
|
||||||
|
- A record without a recognized terminator remains ignored.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
Tests cover default root discovery, a custom migrated root, the CLI option, and
|
||||||
|
the existing synthetic sample library. The scanner remains read-only.
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
# Synthetic Sample Library
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
Real Serato libraries contain private paths, listening history, and copyrighted
|
||||||
|
music. Contributors still need a repeatable library for tests and demonstrations.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
`samples/small-library/manifest.json` describes a compact migration scenario.
|
||||||
|
`generate.py` turns that manifest into fake audio files and UTF-16-LE crate files
|
||||||
|
inside an ignored `generated` directory. The generated files are disposable and
|
||||||
|
contain no real audio or user library data.
|
||||||
|
|
||||||
|
## Edge Cases
|
||||||
|
|
||||||
|
- Five manually maintained crates and two smart/dynamic crates.
|
||||||
|
- Mixed supported audio extensions.
|
||||||
|
- One reference whose file is absent.
|
||||||
|
- One stale reference whose file has been renamed.
|
||||||
|
- References shared by static and smart crates.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
`pytest` generates the sample in a temporary directory and verifies its crate,
|
||||||
|
track, and missing-reference counts through the production parser and scanner.
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
# Test Foundation
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
The crate parser handles unusual binary text and known extension artifacts. Without
|
||||||
|
automated tests, a small refactor could silently change library counts or reports.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
Tests cover the read-only pipeline from crate and filesystem inputs through the
|
||||||
|
core library model to CSV, text, and CLI output. All test data is synthetic and is
|
||||||
|
created in temporary directories.
|
||||||
|
|
||||||
|
## Edge Cases
|
||||||
|
|
||||||
|
- Known MP3, M4A, WAV, and AIF decode artifacts.
|
||||||
|
- Crate records without a recognized stop marker.
|
||||||
|
- Supported extensions with mixed case and unsupported files.
|
||||||
|
- References that exist by filename and references that remain missing.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
Run `pytest`. No test reads a real Serato library or writes outside pytest's
|
||||||
|
temporary directory.
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
[build-system]
|
||||||
|
requires = ["setuptools>=61"]
|
||||||
|
build-backend = "setuptools.build_meta"
|
||||||
|
|
||||||
|
[project]
|
||||||
|
name = "serato-doctor"
|
||||||
|
version = "0.1.0"
|
||||||
|
description = "Inspect, diagnose, repair, and migrate DJ libraries."
|
||||||
|
requires-python = ">=3.9"
|
||||||
|
dependencies = []
|
||||||
|
|
||||||
|
[project.optional-dependencies]
|
||||||
|
dev = ["pytest>=8,<9"]
|
||||||
|
|
||||||
|
[project.scripts]
|
||||||
|
serato-doctor = "serato_doctor.cli:main"
|
||||||
|
|
||||||
|
[tool.pytest.ini_options]
|
||||||
|
testpaths = ["tests"]
|
||||||
|
|
||||||
|
[tool.setuptools.packages.find]
|
||||||
|
include = ["serato_doctor*"]
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
# Small Synthetic Library
|
||||||
|
|
||||||
|
This fixture models a small Serato migration without containing music or personal
|
||||||
|
library data. It has 10 fake audio files, five static crates, two smart crates,
|
||||||
|
one absent song, and one renamed song.
|
||||||
|
|
||||||
|
Generate it with:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
python3 samples/small-library/generate.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Analyze it with:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
python3 -m serato_doctor.cli \
|
||||||
|
--serato samples/small-library/generated/Serato/_Serato_ \
|
||||||
|
--music samples/small-library/generated/Music \
|
||||||
|
--out samples/small-library/generated/scan.csv \
|
||||||
|
--report samples/small-library/generated/report.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected CLI counts are 13 crate references, 10 disk tracks, and 2 references
|
||||||
|
missing by filename. Everything under `generated/` is disposable and ignored by
|
||||||
|
Git.
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
"""Generate a disposable, synthetic Serato library from the sample manifest."""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
|
||||||
|
SAMPLE_ROOT = Path(__file__).parent
|
||||||
|
SERATO_PATH_PREFIX = "Users/sample-user/OneDrive/Jukebox/"
|
||||||
|
|
||||||
|
|
||||||
|
def load_manifest() -> dict:
|
||||||
|
return json.loads((SAMPLE_ROOT / "manifest.json").read_text(encoding="utf-8"))
|
||||||
|
|
||||||
|
|
||||||
|
def build_sample(output: Optional[Path] = None) -> Path:
|
||||||
|
root = output or SAMPLE_ROOT / "generated"
|
||||||
|
manifest = load_manifest()
|
||||||
|
music_root = root / "Music"
|
||||||
|
crate_root = root / "Serato" / "_Serato_" / "Subcrates"
|
||||||
|
crate_root.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
for relative_path in manifest["tracks"]:
|
||||||
|
track_path = music_root / relative_path
|
||||||
|
track_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
track_path.write_bytes(
|
||||||
|
f"Synthetic Serato Doctor fixture: {relative_path}\n".encode("utf-8")
|
||||||
|
)
|
||||||
|
|
||||||
|
for crate in manifest["crates"]:
|
||||||
|
records = "".join(
|
||||||
|
f"{SERATO_PATH_PREFIX}{relative_path}otrk"
|
||||||
|
for relative_path in crate["references"]
|
||||||
|
)
|
||||||
|
(crate_root / crate["name"]).write_bytes(records.encode("utf-16-le"))
|
||||||
|
|
||||||
|
return root
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("--output", type=Path)
|
||||||
|
args = parser.parse_args()
|
||||||
|
root = build_sample(args.output)
|
||||||
|
print(f"Generated synthetic library: {root}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
{
|
||||||
|
"tracks": [
|
||||||
|
"House/First.mp3",
|
||||||
|
"House/Second.m4a",
|
||||||
|
"Open Format/Third.wav",
|
||||||
|
"Open Format/Fourth.aif",
|
||||||
|
"Classics/Fifth.mp3",
|
||||||
|
"Classics/Sixth.flac",
|
||||||
|
"Warmup/Seventh.mp3",
|
||||||
|
"Warmup/Eighth.mp3",
|
||||||
|
"Renamed/New Name.mp3",
|
||||||
|
"Bonus/Ninth.MP3"
|
||||||
|
],
|
||||||
|
"crates": [
|
||||||
|
{
|
||||||
|
"name": "House.crate",
|
||||||
|
"type": "static",
|
||||||
|
"references": ["House/First.mp3", "House/Second.m4a", "House/Missing.mp3"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Open Format.crate",
|
||||||
|
"type": "static",
|
||||||
|
"references": ["Open Format/Third.wav", "Open Format/Fourth.aif"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Classics.crate",
|
||||||
|
"type": "static",
|
||||||
|
"references": ["Classics/Fifth.mp3", "Classics/Sixth.flac"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Renamed Tracks.crate",
|
||||||
|
"type": "static",
|
||||||
|
"references": ["Renamed/Old Name.mp3"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Bonus.crate",
|
||||||
|
"type": "static",
|
||||||
|
"references": ["Bonus/Ninth.MP3"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Compatible by key.crate",
|
||||||
|
"type": "smart",
|
||||||
|
"references": ["House/First.mp3", "House/Second.m4a"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Smart Warmup.crate",
|
||||||
|
"type": "smart",
|
||||||
|
"references": ["Warmup/Seventh.mp3", "Warmup/Eighth.mp3"]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"expected": {
|
||||||
|
"disk_tracks": 10,
|
||||||
|
"crate_references": 13,
|
||||||
|
"missing_by_filename": ["Missing.mp3", "Old Name.mp3"]
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
from pathlib import Path
|
||||||
|
import argparse
|
||||||
|
|
||||||
|
from serato_doctor.config import ScanConfig
|
||||||
|
from serato_doctor.crate_parser import parse_crates
|
||||||
|
from serato_doctor.models.library import Library
|
||||||
|
from serato_doctor.scanner import scan_audio
|
||||||
|
from serato_doctor.report import write_csv, write_missing_report
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(prog="serato-doctor")
|
||||||
|
parser.add_argument("--serato", default=str(Path.home() / "Music/_Serato_"))
|
||||||
|
parser.add_argument("--music", default=str(Path.home() / "Library/CloudStorage/OneDrive-Personal/Jukebox"))
|
||||||
|
parser.add_argument("--out", default=str(Path.home() / "Desktop/serato_doctor_scan.csv"))
|
||||||
|
parser.add_argument("--report", default=str(Path.home() / "Desktop/serato_doctor_missing_report.txt"))
|
||||||
|
parser.add_argument(
|
||||||
|
"--reference-root",
|
||||||
|
action="append",
|
||||||
|
default=[],
|
||||||
|
help="Old library root stored in crates; may be supplied more than once",
|
||||||
|
)
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
config = ScanConfig.build(
|
||||||
|
serato=Path(args.serato),
|
||||||
|
music=Path(args.music),
|
||||||
|
out=Path(args.out),
|
||||||
|
report=Path(args.report),
|
||||||
|
reference_roots=(Path(root) for root in args.reference_root),
|
||||||
|
)
|
||||||
|
|
||||||
|
library = Library.build(
|
||||||
|
references=parse_crates(
|
||||||
|
config.serato / "Subcrates", config.reference_roots
|
||||||
|
),
|
||||||
|
tracks=scan_audio(config.music),
|
||||||
|
)
|
||||||
|
results = library.reconcile_by_filename()
|
||||||
|
missing_count = sum(1 for result in results if not result.exists_by_filename)
|
||||||
|
|
||||||
|
write_csv(results, config.out)
|
||||||
|
write_missing_report(results, config.report)
|
||||||
|
|
||||||
|
print(f"Crate references: {len(library.references)}")
|
||||||
|
print(f"Disk tracks: {len(library.tracks)}")
|
||||||
|
print(f"Missing by filename: {missing_count}")
|
||||||
|
print(f"CSV: {config.out}")
|
||||||
|
print(f"Report: {config.report}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
|
|||||||
@@ -0,0 +1,25 @@
|
|||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Iterable, Tuple
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ScanConfig:
|
||||||
|
"""Read-only paths used for one library scan."""
|
||||||
|
|
||||||
|
serato: Path
|
||||||
|
music: Path
|
||||||
|
out: Path
|
||||||
|
report: Path
|
||||||
|
reference_roots: Tuple[Path, ...] = ()
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def build(
|
||||||
|
cls,
|
||||||
|
serato: Path,
|
||||||
|
music: Path,
|
||||||
|
out: Path,
|
||||||
|
report: Path,
|
||||||
|
reference_roots: Iterable[Path] = (),
|
||||||
|
) -> "ScanConfig":
|
||||||
|
return cls(serato, music, out, report, tuple(reference_roots))
|
||||||
@@ -0,0 +1,84 @@
|
|||||||
|
from pathlib import Path
|
||||||
|
from typing import Iterable, Tuple
|
||||||
|
|
||||||
|
from serato_doctor.models.crate import Crate
|
||||||
|
from serato_doctor.models.reference import TrackReference
|
||||||
|
|
||||||
|
|
||||||
|
def read_crate_text(crate_path: Path) -> str:
|
||||||
|
return crate_path.read_bytes().decode("utf-16-le", errors="ignore").replace("\x00", "")
|
||||||
|
|
||||||
|
|
||||||
|
def clean_path(raw: str) -> str:
|
||||||
|
# Common Serato decode artifacts where final extension char gets merged.
|
||||||
|
raw = raw.replace(".mp漳", ".mp3")
|
||||||
|
raw = raw.replace(".MP漳", ".MP3")
|
||||||
|
raw = raw.replace(".m4愠", ".m4a")
|
||||||
|
raw = raw.replace(".M4愠", ".M4A")
|
||||||
|
raw = raw.replace(".wa瘠", ".wav")
|
||||||
|
raw = raw.replace(".WA瘠", ".WAV")
|
||||||
|
raw = raw.replace(".ai映", ".aif")
|
||||||
|
raw = raw.replace(".AI映", ".AIF")
|
||||||
|
return raw.strip()
|
||||||
|
|
||||||
|
|
||||||
|
DEFAULT_PATH_MARKERS = ("Users/", "Volumes/")
|
||||||
|
|
||||||
|
|
||||||
|
def path_markers(reference_roots: Iterable[Path]) -> Tuple[str, ...]:
|
||||||
|
configured = tuple(
|
||||||
|
root.expanduser().as_posix().strip("/") + "/" for root in reference_roots
|
||||||
|
)
|
||||||
|
return configured or DEFAULT_PATH_MARKERS
|
||||||
|
|
||||||
|
|
||||||
|
def load_crate(crate_path: Path, reference_roots: Iterable[Path] = ()) -> Crate:
|
||||||
|
text = read_crate_text(crate_path)
|
||||||
|
refs = []
|
||||||
|
|
||||||
|
# Serato record markers seen after paths in UTF-16-LE decoded crate data.
|
||||||
|
stop_markers = ["牴k", "otrk", "ptrk", "tvcn", "ovct"]
|
||||||
|
|
||||||
|
for marker in path_markers(reference_roots):
|
||||||
|
for part in text.split(marker)[1:]:
|
||||||
|
candidate = marker + part
|
||||||
|
|
||||||
|
stops = [
|
||||||
|
candidate.find(stop)
|
||||||
|
for stop in stop_markers
|
||||||
|
if candidate.find(stop) != -1
|
||||||
|
]
|
||||||
|
if not stops:
|
||||||
|
continue
|
||||||
|
|
||||||
|
raw_path = "/" + candidate[: min(stops)]
|
||||||
|
raw_path = clean_path(raw_path)
|
||||||
|
|
||||||
|
path = Path(raw_path)
|
||||||
|
refs.append(
|
||||||
|
TrackReference(
|
||||||
|
source=crate_path,
|
||||||
|
path=path,
|
||||||
|
filename=path.name,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
return Crate(path=crate_path, references=tuple(refs))
|
||||||
|
|
||||||
|
|
||||||
|
def parse_crate(
|
||||||
|
crate_path: Path, reference_roots: Iterable[Path] = ()
|
||||||
|
) -> list[TrackReference]:
|
||||||
|
"""Parse references from one crate, preserving the prototype API."""
|
||||||
|
|
||||||
|
return list(load_crate(crate_path, reference_roots).references)
|
||||||
|
|
||||||
|
|
||||||
|
def parse_crates(
|
||||||
|
root: Path, reference_roots: Iterable[Path] = ()
|
||||||
|
) -> list[TrackReference]:
|
||||||
|
refs = []
|
||||||
|
reference_roots = tuple(reference_roots)
|
||||||
|
for crate in root.rglob("*.crate"):
|
||||||
|
refs.extend(parse_crate(crate, reference_roots))
|
||||||
|
return refs
|
||||||
|
|||||||
@@ -0,0 +1,6 @@
|
|||||||
|
from serato_doctor.models.crate import Crate
|
||||||
|
from serato_doctor.models.library import Library
|
||||||
|
from serato_doctor.models.reference import ReferenceResult, TrackReference
|
||||||
|
from serato_doctor.models.track import DiskTrack
|
||||||
|
|
||||||
|
__all__ = ["Crate", "DiskTrack", "Library", "ReferenceResult", "TrackReference"]
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Tuple
|
||||||
|
|
||||||
|
from serato_doctor.models.reference import TrackReference
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Crate:
|
||||||
|
"""A Serato crate and the track references parsed from it."""
|
||||||
|
|
||||||
|
path: Path
|
||||||
|
references: Tuple[TrackReference, ...]
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Iterable, Tuple
|
||||||
|
|
||||||
|
from serato_doctor.models.reference import ReferenceResult, TrackReference
|
||||||
|
from serato_doctor.models.track import DiskTrack
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Library:
|
||||||
|
"""The read-only view of crate references and audio found on disk."""
|
||||||
|
|
||||||
|
references: Tuple[TrackReference, ...]
|
||||||
|
tracks: Tuple[DiskTrack, ...]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def build(
|
||||||
|
cls,
|
||||||
|
references: Iterable[TrackReference],
|
||||||
|
tracks: Iterable[DiskTrack],
|
||||||
|
) -> "Library":
|
||||||
|
return cls(tuple(references), tuple(tracks))
|
||||||
|
|
||||||
|
def reconcile_by_filename(self) -> Tuple[ReferenceResult, ...]:
|
||||||
|
disk_names = {track.filename for track in self.tracks}
|
||||||
|
return tuple(
|
||||||
|
ReferenceResult(
|
||||||
|
reference=reference,
|
||||||
|
exists_by_filename=reference.filename in disk_names,
|
||||||
|
)
|
||||||
|
for reference in self.references
|
||||||
|
)
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class TrackReference:
|
||||||
|
"""A track path referenced by a Serato crate."""
|
||||||
|
|
||||||
|
source: Path
|
||||||
|
path: Path
|
||||||
|
filename: str
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ReferenceResult:
|
||||||
|
"""The filename-level reconciliation result for a crate reference."""
|
||||||
|
|
||||||
|
reference: TrackReference
|
||||||
|
exists_by_filename: bool
|
||||||
|
|
||||||
|
def as_row(self) -> dict:
|
||||||
|
return {
|
||||||
|
"crate": str(self.reference.source),
|
||||||
|
"serato_path": str(self.reference.path),
|
||||||
|
"filename": self.reference.filename,
|
||||||
|
"exists_by_filename": self.exists_by_filename,
|
||||||
|
}
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class DiskTrack:
|
||||||
|
"""An audio file discovered on disk."""
|
||||||
|
|
||||||
|
path: Path
|
||||||
|
filename: str
|
||||||
|
size: int
|
||||||
|
suffix: str
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
from collections import Counter, defaultdict
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Iterable
|
||||||
|
import csv
|
||||||
|
|
||||||
|
from serato_doctor.models.reference import ReferenceResult
|
||||||
|
|
||||||
|
|
||||||
|
def write_missing_report(results: Iterable[ReferenceResult], out: Path) -> None:
|
||||||
|
rows = [result.as_row() for result in results]
|
||||||
|
missing = [r for r in rows if not r["exists_by_filename"]]
|
||||||
|
|
||||||
|
crate_counts = Counter(r["crate"] for r in missing)
|
||||||
|
filename_counts = Counter(r["filename"] for r in missing)
|
||||||
|
|
||||||
|
with out.open("w", encoding="utf-8") as f:
|
||||||
|
f.write("# Serato Doctor Missing Report\n\n")
|
||||||
|
f.write(f"Total missing references: {len(missing)}\n\n")
|
||||||
|
|
||||||
|
f.write("## Missing by crate\n\n")
|
||||||
|
for crate, count in crate_counts.most_common():
|
||||||
|
f.write(f"{count:5} {crate}\n")
|
||||||
|
|
||||||
|
f.write("\n## Most common missing filenames\n\n")
|
||||||
|
for filename, count in filename_counts.most_common(100):
|
||||||
|
f.write(f"{count:5} {filename}\n")
|
||||||
|
|
||||||
|
f.write("\n## Detail\n\n")
|
||||||
|
by_crate = defaultdict(list)
|
||||||
|
for r in missing:
|
||||||
|
by_crate[r["crate"]].append(r["filename"])
|
||||||
|
|
||||||
|
for crate, names in sorted(by_crate.items()):
|
||||||
|
f.write(f"\n### {crate}\n")
|
||||||
|
for name in sorted(set(names)):
|
||||||
|
f.write(f"- {name}\n")
|
||||||
|
|
||||||
|
|
||||||
|
def write_csv(results: Iterable[ReferenceResult], out: Path) -> None:
|
||||||
|
rows = [result.as_row() for result in results]
|
||||||
|
with out.open("w", newline="", encoding="utf-8") as f:
|
||||||
|
writer = csv.DictWriter(
|
||||||
|
f,
|
||||||
|
fieldnames=["crate", "serato_path", "filename", "exists_by_filename"],
|
||||||
|
)
|
||||||
|
writer.writeheader()
|
||||||
|
writer.writerows(rows)
|
||||||
|
|||||||
@@ -0,0 +1,31 @@
|
|||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from serato_doctor.models.track import DiskTrack
|
||||||
|
|
||||||
|
AUDIO_SUFFIXES = {".mp3", ".m4a", ".wav", ".aif", ".aiff", ".flac"}
|
||||||
|
|
||||||
|
|
||||||
|
def scan_audio(folder: Path) -> list[DiskTrack]:
|
||||||
|
tracks = []
|
||||||
|
|
||||||
|
for path in folder.rglob("*"):
|
||||||
|
if not path.is_file():
|
||||||
|
continue
|
||||||
|
if path.suffix.lower() not in AUDIO_SUFFIXES:
|
||||||
|
continue
|
||||||
|
|
||||||
|
try:
|
||||||
|
stat = path.stat()
|
||||||
|
except OSError:
|
||||||
|
continue
|
||||||
|
|
||||||
|
tracks.append(
|
||||||
|
DiskTrack(
|
||||||
|
path=path,
|
||||||
|
filename=path.name,
|
||||||
|
size=stat.st_size,
|
||||||
|
suffix=path.suffix.lower(),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
return tracks
|
||||||
|
|||||||
@@ -0,0 +1,84 @@
|
|||||||
|
import csv
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from serato_doctor.cli import main
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_writes_reports_and_prints_counts(tmp_path, monkeypatch, capsys):
|
||||||
|
serato = tmp_path / "serato"
|
||||||
|
subcrates = serato / "Subcrates"
|
||||||
|
music = tmp_path / "music"
|
||||||
|
subcrates.mkdir(parents=True)
|
||||||
|
music.mkdir()
|
||||||
|
|
||||||
|
crate_text = (
|
||||||
|
"Users/sample-user/OneDrive/Jukebox/Found.mp漳牴k"
|
||||||
|
"Users/sample-user/OneDrive/Jukebox/Missing.mp漳牴k"
|
||||||
|
)
|
||||||
|
(subcrates / "Test.crate").write_bytes(crate_text.encode("utf-16-le"))
|
||||||
|
(music / "Found.mp3").write_bytes(b"synthetic audio")
|
||||||
|
csv_path = tmp_path / "scan.csv"
|
||||||
|
report_path = tmp_path / "report.txt"
|
||||||
|
monkeypatch.setattr(
|
||||||
|
sys,
|
||||||
|
"argv",
|
||||||
|
[
|
||||||
|
"serato-doctor",
|
||||||
|
"--serato",
|
||||||
|
str(serato),
|
||||||
|
"--music",
|
||||||
|
str(music),
|
||||||
|
"--out",
|
||||||
|
str(csv_path),
|
||||||
|
"--report",
|
||||||
|
str(report_path),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
main()
|
||||||
|
|
||||||
|
output = capsys.readouterr().out
|
||||||
|
assert "Crate references: 2" in output
|
||||||
|
assert "Disk tracks: 1" in output
|
||||||
|
assert "Missing by filename: 1" in output
|
||||||
|
assert f"CSV: {csv_path}" in output
|
||||||
|
assert f"Report: {report_path}" in output
|
||||||
|
with csv_path.open(newline="", encoding="utf-8") as csv_file:
|
||||||
|
rows = list(csv.DictReader(csv_file))
|
||||||
|
assert [row["exists_by_filename"] for row in rows] == ["True", "False"]
|
||||||
|
assert "Total missing references: 1" in report_path.read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_accepts_old_reference_root(tmp_path, monkeypatch, capsys):
|
||||||
|
serato = tmp_path / "serato"
|
||||||
|
subcrates = serato / "Subcrates"
|
||||||
|
music = tmp_path / "music"
|
||||||
|
subcrates.mkdir(parents=True)
|
||||||
|
music.mkdir()
|
||||||
|
(subcrates / "Test.crate").write_bytes(
|
||||||
|
"Archive/Jukebox/Found.mp3otrk".encode("utf-16-le")
|
||||||
|
)
|
||||||
|
(music / "Found.mp3").write_bytes(b"synthetic audio")
|
||||||
|
monkeypatch.setattr(
|
||||||
|
sys,
|
||||||
|
"argv",
|
||||||
|
[
|
||||||
|
"serato-doctor",
|
||||||
|
"--serato",
|
||||||
|
str(serato),
|
||||||
|
"--music",
|
||||||
|
str(music),
|
||||||
|
"--out",
|
||||||
|
str(tmp_path / "scan.csv"),
|
||||||
|
"--report",
|
||||||
|
str(tmp_path / "report.txt"),
|
||||||
|
"--reference-root",
|
||||||
|
"/Archive/Jukebox",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
main()
|
||||||
|
|
||||||
|
output = capsys.readouterr().out
|
||||||
|
assert "Crate references: 1" in output
|
||||||
|
assert "Missing by filename: 0" in output
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from serato_doctor.config import ScanConfig
|
||||||
|
|
||||||
|
|
||||||
|
def test_scan_config_freezes_reference_roots():
|
||||||
|
roots = (Path(root) for root in ["/old/one", "/old/two"])
|
||||||
|
|
||||||
|
config = ScanConfig.build(
|
||||||
|
serato=Path("serato"),
|
||||||
|
music=Path("music"),
|
||||||
|
out=Path("scan.csv"),
|
||||||
|
report=Path("report.txt"),
|
||||||
|
reference_roots=roots,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert config.reference_roots == (Path("/old/one"), Path("/old/two"))
|
||||||
@@ -0,0 +1,86 @@
|
|||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from serato_doctor.crate_parser import (
|
||||||
|
clean_path,
|
||||||
|
load_crate,
|
||||||
|
parse_crate,
|
||||||
|
parse_crates,
|
||||||
|
path_markers,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("artifact", "expected"),
|
||||||
|
[
|
||||||
|
("song.mp漳", "song.mp3"),
|
||||||
|
("song.m4愠", "song.m4a"),
|
||||||
|
("song.wa瘠", "song.wav"),
|
||||||
|
("song.ai映", "song.aif"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_clean_path_repairs_known_extension_artifacts(artifact, expected):
|
||||||
|
assert clean_path(artifact) == expected
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_crate_extracts_references(tmp_path):
|
||||||
|
crate_path = tmp_path / "House.crate"
|
||||||
|
crate_text = (
|
||||||
|
"header"
|
||||||
|
"Users/sample-user/OneDrive/Jukebox/House/First.mp漳牴k"
|
||||||
|
"metadata"
|
||||||
|
"Users/sample-user/OneDrive/Jukebox/House/Second.m4愠otrk"
|
||||||
|
)
|
||||||
|
crate_path.write_bytes(crate_text.encode("utf-16-le"))
|
||||||
|
|
||||||
|
crate = load_crate(crate_path)
|
||||||
|
|
||||||
|
assert crate.path == crate_path
|
||||||
|
assert [reference.filename for reference in crate.references] == [
|
||||||
|
"First.mp3",
|
||||||
|
"Second.m4a",
|
||||||
|
]
|
||||||
|
assert parse_crate(crate_path) == list(crate.references)
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_crate_ignores_record_without_stop_marker(tmp_path):
|
||||||
|
crate_path = Path(tmp_path) / "Incomplete.crate"
|
||||||
|
crate_path.write_bytes(
|
||||||
|
"Users/sample-user/OneDrive/Jukebox/House/Incomplete.mp3".encode(
|
||||||
|
"utf-16-le"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert parse_crate(crate_path) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_crate_uses_configured_reference_root(tmp_path):
|
||||||
|
crate_path = tmp_path / "Migrated.crate"
|
||||||
|
crate_path.write_bytes(
|
||||||
|
"Archive/Old Library/House/Track.mp3otrk".encode("utf-16-le")
|
||||||
|
)
|
||||||
|
|
||||||
|
references = parse_crate(crate_path, [Path("/Archive/Old Library")])
|
||||||
|
|
||||||
|
assert references[0].path == Path("/Archive/Old Library/House/Track.mp3")
|
||||||
|
|
||||||
|
|
||||||
|
def test_default_path_markers_do_not_contain_a_username():
|
||||||
|
assert path_markers([]) == ("Users/", "Volumes/")
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_crates_reuses_configured_roots_for_every_crate(tmp_path):
|
||||||
|
for name in ("First", "Second"):
|
||||||
|
(tmp_path / f"{name}.crate").write_bytes(
|
||||||
|
f"Archive/Jukebox/{name}.mp3otrk".encode("utf-16-le")
|
||||||
|
)
|
||||||
|
|
||||||
|
references = parse_crates(
|
||||||
|
tmp_path, (root for root in [Path("/Archive/Jukebox")])
|
||||||
|
)
|
||||||
|
|
||||||
|
assert {reference.filename for reference in references} == {
|
||||||
|
"First.mp3",
|
||||||
|
"Second.mp3",
|
||||||
|
}
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from serato_doctor.models.library import Library
|
||||||
|
from serato_doctor.models.reference import TrackReference
|
||||||
|
from serato_doctor.models.track import DiskTrack
|
||||||
|
|
||||||
|
|
||||||
|
def test_library_reconciles_references_by_filename():
|
||||||
|
crate = Path("House.crate")
|
||||||
|
references = [
|
||||||
|
TrackReference(crate, Path("/old/Found.mp3"), "Found.mp3"),
|
||||||
|
TrackReference(crate, Path("/old/Missing.mp3"), "Missing.mp3"),
|
||||||
|
]
|
||||||
|
tracks = [DiskTrack(Path("/new/Found.mp3"), "Found.mp3", 10, ".mp3")]
|
||||||
|
|
||||||
|
results = Library.build(references, tracks).reconcile_by_filename()
|
||||||
|
|
||||||
|
assert [result.exists_by_filename for result in results] == [True, False]
|
||||||
|
assert results[1].as_row() == {
|
||||||
|
"crate": "House.crate",
|
||||||
|
"serato_path": "/old/Missing.mp3",
|
||||||
|
"filename": "Missing.mp3",
|
||||||
|
"exists_by_filename": False,
|
||||||
|
}
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
import runpy
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from serato_doctor.crate_parser import parse_crates
|
||||||
|
from serato_doctor.models.library import Library
|
||||||
|
from serato_doctor.scanner import scan_audio
|
||||||
|
|
||||||
|
|
||||||
|
def test_generated_sample_library_has_expected_scenario(tmp_path):
|
||||||
|
generator_path = (
|
||||||
|
Path(__file__).parents[1] / "samples" / "small-library" / "generate.py"
|
||||||
|
)
|
||||||
|
build_sample = runpy.run_path(str(generator_path))["build_sample"]
|
||||||
|
sample_root = build_sample(tmp_path / "sample")
|
||||||
|
|
||||||
|
references = parse_crates(sample_root / "Serato" / "_Serato_" / "Subcrates")
|
||||||
|
tracks = scan_audio(sample_root / "Music")
|
||||||
|
results = Library.build(references, tracks).reconcile_by_filename()
|
||||||
|
missing = {
|
||||||
|
result.reference.filename
|
||||||
|
for result in results
|
||||||
|
if not result.exists_by_filename
|
||||||
|
}
|
||||||
|
|
||||||
|
assert len(list((sample_root / "Serato").rglob("*.crate"))) == 7
|
||||||
|
assert len(references) == 13
|
||||||
|
assert len(tracks) == 10
|
||||||
|
assert missing == {"Missing.mp3", "Old Name.mp3"}
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
from serato_doctor.scanner import scan_audio
|
||||||
|
|
||||||
|
|
||||||
|
def test_scan_audio_finds_supported_files(tmp_path):
|
||||||
|
music = tmp_path / "music"
|
||||||
|
nested = music / "House"
|
||||||
|
nested.mkdir(parents=True)
|
||||||
|
(nested / "First.MP3").write_bytes(b"synthetic audio")
|
||||||
|
(nested / "Second.flac").write_bytes(b"fixture")
|
||||||
|
(nested / "notes.txt").write_text("not audio", encoding="utf-8")
|
||||||
|
|
||||||
|
tracks = scan_audio(music)
|
||||||
|
|
||||||
|
assert {track.filename for track in tracks} == {"First.MP3", "Second.flac"}
|
||||||
|
first = next(track for track in tracks if track.filename == "First.MP3")
|
||||||
|
assert first.suffix == ".mp3"
|
||||||
|
assert first.size == len(b"synthetic audio")
|
||||||
Reference in New Issue
Block a user