Voice Dataset Studio: local-first audio dataset curation

TL;DR

• It is dataset curation tooling, not a training framework: import, inventory, listen, inspect, cut, transcribe, diarize, annotate, review, version and export.
• Everything runs locally on SQLite with no GPU required. Raw files are never modified, and automated results are always reviewable suggestions rather than silent decisions.
• Analysis is a gated progression — Files, Risk, Speech, Release — where each gate exposes one primary check, its result, and what the next gate unlocks.
• Exports cover the formats speech tooling actually consumes: kaldi, rttm, textgrid, lhotse, Hugging Face parquet, webdataset, audformat-style and jsonl, each with a manifest and checksums.

What it is and who it is for

Voice Dataset Studio — published as Audio Dataset Studio — is a local application for curating an audio or speech dataset. The README states the boundary directly: it is a dataset curation studio, not a model-training framework.

It is for the person who already has recordings and needs them to become a training set: dozens of takes that need segmenting, transcripts that need alignment, speakers that need labelling, and a package that a trainer will accept on the first try. It is also for the person who has been editing the dataset by hand in a folder tree and losing track of which files were cut, renamed or half-exported.

Import and provenance

Import takes single files, folders, public links, Hugging Face dataset files, JSONL, CSV, TSV or Parquet manifests, Kaldi wav.scp directories and traversal-safe ZIPs. Every import preserves source paths, original bytes, SHA-256 hashes, probe metadata (codec, sample rate, channels), licenses and provenance — which is what makes the curation honest later, because you can always tell where a file came from.

Listen and review

One Explorer workspace lists every recording with inline playback, waveform, spectrogram and a temporal selection. Review works from the keyboard: j and k navigate, 1–4 decide. Selecting a recording opens an inspector with tabs for transport, decisions, segments, transcript, speakers, issues and annotations, so technical detail stays on demand instead of filling the screen.

The decisions are keep, review, quarantine and reject, and they are reversible. The persistent player keeps the waveform canvas and spectrogram available while you move through the set, which is what makes reviewing two hundred files possible.

Staged analysis

Analyze is one gated progression: Files, Risk, Speech, Release. Each gate shows one primary check, its configuration, the result facts and what the next gate unlocks. Advanced checks sit behind a disclosure and nothing runs before you ask for it.

Quality checks — silence, clipping, low RMS, zero duration — produce issues with evidence rather than automatic rejections. That distinction is the design principle: the tool surfaces the problem, you decide whether the recording is unusable. The same holds for the PII and secret scan on transcripts, exact, re-encoded and acoustic duplicate groups, and leakage checks.

Segmentation, speakers and versions

Segments are cut and clips are derived while the raw file stays untouched. Annotations are temporal rubrics, and speakers are registered with turns, consent and recording sessions. Splits are speaker-aware and deterministic.

Versions keep a checkpoint ledger with lineage and checksums, a diff, and a restore-as-new-checkpoint confirmation — nothing is overwritten. Speaker and audio withdrawal keeps an audit trail, which is the part most dataset tooling leaves out.

Export

Export is a four-step wizard — readiness blockers, package, contents, export — with the final action disabled until every blocker clears. Formats: kaldi, rttm, textgrid, lhotse, hf (parquet), webdataset, audformat-style and jsonl. Each export carries a manifest, checksums and license, consent, quality, provenance and split reports, and strict validation blocks a misleading package.

How to install it

Requirements are Python 3.11+, uv, Node 20+ or Bun, and optionally FFmpeg or ffprobe on the path — plain WAV works with the standard library alone.

git clone https://github.com/artificialguybr/voice-dataset-studio.git
cd voice-dataset-studio
python tools/setup.py
cp .env.example .env       # optional; the app runs without it
python tools/start.py

The app opens at http://127.0.0.1:8767. On Windows PowerShell, use py tools\setup.py and py tools\start.py. setup.py installs backend dependencies with uv and frontend dependencies with Bun or npm; start.py serves the built frontend from the API process and handles Ctrl+C on every OS.

Speech capabilities are optional extras, and the base installation stays lightweight:

uv sync --extra speech        # whisperx + pyannote.audio
uv sync --extra features     # speechbrain embeddings
uv sync --extra augmentation # audiomentations recipes
uv sync --extra export      # native lhotse/audformat writers

A missing provider fails the job with an actionable explanation instead of returning a fake result, and provider availability is always visible at GET /api/providers and in Settings.

Plugins, MCP and the API

Settings exposes the same modular extension seam used by the other Dataset Studios, and the repository ships audio packs: audio-local for RMS, peak and duration features plus acoustic fingerprints, speech-local for WhisperX transcription plus pyannote diarization and VAD, and audio-export for Lhotse and audformat availability checks. Packs build from tracked source with uv run python plugins/build_packs.py --plugin all and install from Settings or through POST /api/plugins/install. Plugins use the ids-plugin.v1 JSONL protocol, persistent processes, least-privilege permissions, path containment, timeouts and idle unloading.

An MCP stdio server ships for Claude Desktop, Cursor and other MCP clients: start the API, copy mcp-config.example.json with your repository path, then run uv run python -m backend.mcp_server. It exposes health, dataset listing, audio search and inspection, segment comparison, issue review, explicit audio decisions, transcript, alignment and diarization proposals, speaker turns, asynchronous jobs, versions with diff, withdrawal with explicit ids, and exports — against the same local REST API, so the same validation rules apply.

The FastAPI service is the full integration surface: GET /api/health and GET /api/openapi.json give a complete contract, and the groups cover datasets, imports and jobs, audio, decisions, issues, segments, transcripts, speakers, annotations, consent, provenance, features, duplicates, leakage, splits, versions, withdrawal, exports and providers.

When to use it, when not to

Use it when you have raw audio and need a training-ready package with a paper trail, when a previous export was rejected by a trainer for a reason you could not see, or when two people have edited the same folder tree and you no longer know what is in it.

Do not use it as a training framework and do not expect it to train anything. The README is explicit: curation, not training. And do not expect the automated checks to make the call for you — silence, clipping, duplicates and PII arrive as suggestions with evidence, and every one of them needs your decision.

Sources

1. Voice Dataset Studio — repository — product tour, feature list, requirements, install commands, provider table, plugin packs, MCP and API groups quoted above.
2. README.md — install and run commands, extras, export formats, license (MIT).

Project page: Voice Dataset Studio

For the image side of the same workflow, see automated dataset creation for LoRA training, and for what a good dataset needs in the first place, building a training dataset.

The repository is MIT-licensed. Raw files are never modified, so a bad decision is always reversible.

FAQ

Does it train a model? No. The README calls it a dataset curation studio, not a model-training framework: it prepares and validates the dataset, and the trainer stays separate.

Does it need a GPU? No. SQLite is the default local database and GPU support is optional, only for selected local ML providers.

Does the quality check reject bad recordings automatically? No. Checks such as silence, clipping, low RMS and zero duration produce issues with evidence; the decision to reject is yours, and it is reversible.

Where does the exported dataset go? Nowhere by itself. Export is a four-step wizard that writes a package with a manifest, checksums and reports, and blocks the final action until every readiness blocker clears.