LLM Dataset Studio: curate SFT, DPO and agent-trace data locally

TL;DR

• It curates text, conversation, SFT, DPO and agent-trace datasets locally on SQLite — and the README is explicit that it is not a model-training framework.
• Source files stay untouched, automated checks produce inspectable evidence, and the final decisions stay explicit and reversible.
• The structured importer recognizes the formats people actually have on disk: OpenAI/TRL chat, ShareGPT, Alpaca, TRL DPO, prompt-completion SFT and agent traces.
• Exports are training-ready: JSONL, Parquet, Hugging Face datasets, manifests, split recipes and audit reports.

What it is and who it is for

LLM Dataset Studio is a local application for the part of fine-tuning nobody documents: turning a pile of text, conversations and traces into a dataset with a paper trail, and knowing which records are junk before the training run finds out the expensive way.

It is for the person assembling an SFT set from conversations, the team building DPO pairs and needing to see chosen against rejected side by side, and the one whose agent traces have steps and tool calls that no generic JSONL viewer shows properly.

Import and provenance

Import takes JSONL, Parquet, TXT, folders, ZIP files and Hugging Face datasets, and the structured importer recognizes and deduplicates the common shapes:

• OpenAI / TRL chat: {"messages": [{"role": "user", "content": "..."}]}
• ShareGPT: {"conversations": [{"from": "human", "value": "..."}]}
• Alpaca: instruction, optional input, and output
• TRL DPO: prompt, chosen, rejected
• Prompt-completion SFT: prompt, completion
• Agent and reasoning traces: trace, steps, tool_calls, trajectory or trace_type

Every import preserves source paths, original bytes, SHA-256 hashes, token estimates, metadata, licenses and provenance. Token estimates at import time are what makes the corpus statistics honest later, because the count comes from the file that was imported rather than from a guess.

Review in one Explorer

One Explorer handles plain text, instruction records, multi-turn conversations, SFT messages, DPO pairs and traces, so search, status, sort and review actions stay in the same place while the reader adapts to the selected record. Read and Rapid review modes cover both the careful pass and the queue that needs a fast keyboard decision.

DPO pairs are reviewed as response A versus response B comparisons without changing the normal record workflow. Trace-like data gets filters such as Has tool calls and Missing final response, which is what makes an agent dataset reviewable at all: without those signals you are reading JSON and hoping.

Statistics, validation and duplicates

Statistics and token metrics are combined into one view: document count, total and average tokens, characters, words, token range, language coverage, formats, sources and decision distribution.

Validation covers language, repetition, size, spam, code, secrets, toxicity, safety, schema and contamination. Duplicates are found in three tiers — exact, near and optional semantic — and decisions are reversible: keep, review, quarantine, reject and restore. The studio also creates labels, tags, preference decisions, derived datasets, samples, mixes and splits.

Contamination is the check that matters most for an evaluation set, and it is also the one most dataset tooling skips: a benchmark question that appears in your training data turns a good score into a meaningless one.

Versions and export

Versions are immutable, with manifests, checksums, diffs and restore support. Export writes JSONL, Parquet and Hugging Face datasets, plus manifests, split recipes and audit reports — so the training run can be reproduced, and an experiment on a subset never overwrites the dataset you had.

The repository also exposes MCP and a REST/OpenAPI surface, so an agent can drive curation with explicit, auditable actions instead of a private script.

How to use it

The workflow is: import, read the statistics, review records in the Explorer, run validation and duplicates, decide, then export and version. The built-in sample-corpus is a one-click way to see the shape of the app before importing anything real.

When to use it, when not to

Use it when an SFT set assembled from three sources needs one passable dataset, when a DPO pair file has never been read pair by pair, when agent traces need filtering by tool calls, or before running an evaluation set that might be contaminated.

Do not use it to train a model, and do not expect it to write your instructions or preferences — the studio curates and validates, and the content decisions stay yours. Do not use it as a labeling platform for a large human team either: it manages a dataset and its versions.

Sources

1. LLM Dataset Studio — repository — product tour, feature list, supported training formats and export scope quoted above.
2. README.md — supported formats, license (MIT).

Project page: LLM Dataset Studio

For what the curated data is used for, see the Llama 3.2 1B Synthia instruction fine-tune; for the image counterpart of the same studio, Images Dataset Studio.

FAQ

Does it train a model? No. The README calls it a dataset curation studio, not a model-training framework; it imports, reviews, validates, versions and exports.

Which chat formats does the importer recognize? OpenAI/TRL chat, ShareGPT, Alpaca, TRL DPO, prompt-completion SFT and agent traces with trace, steps, tool_calls, trajectory or trace_type.

How are DPO pairs reviewed? As response A versus response B comparisons, in the same Explorer, without changing the normal record workflow.

Does it check contamination? Yes, alongside language, repetition, size, spam, code, secrets, toxicity, safety and schema validation.