ArtificialGuyBR

Home / Blog / Fixing things

Whisper Small Portuguese Fine-tune (Common Voice 13)

Fine-tuned Whisper Small for Portuguese ASR with 10.3 WER on Common Voice 13.0. Training details, usage, and evaluation.

By ArtificialGuyBR Published Updated 1 sources cited Fixing things

TL;DR

  • Fine-tuned Whisper Small for Portuguese achieving 10.3 WER on CV13.0 test set
  • Trained on Mozilla Common Voice 13.0 Portuguese (8 hrs on 1×A100 80GB)
  • Ready to use with 🤗 transformers pipeline("automatic-speech-recognition")

What This Model Does

This is a Portuguese-specialized version of OpenAI's Whisper Small (244M parameters). The base Whisper Small was trained on 680k hours of multilingual data with weak supervision. This fine-tune narrows the model's focus to Portuguese using the Mozilla Common Voice 13.0 dataset, yielding a measurable Word Error Rate improvement for Portuguese transcription.

The model accepts 16kHz audio and outputs Portuguese text. It's packaged as a standard 🤗 transformers model with safetensors weights, compatible with the automatic-speech-recognition pipeline.

Training Details

Parameter | Value

|---|---

Base model | openai/whisper-small Dataset | mozilla-foundation/common_voice_13_0 (pt config) Train split | train Eval split | test Learning rate | 1e-05 Train batch size | 64 Eval batch size | 32 Optimizer | Adam (β=(0.9,0.999), ε=1e-08) LR schedule | Linear with 500 warmup steps Training steps | 5,000 Hardware | 1× NVIDIA A100 80GB Training time | ~8 hours WER (test) | 10.30 Training followed the Hugging Face Whisper fine-tuning event recipe.

How to Use

from transformers import pipeline

asr = pipeline(
    "automatic-speech-recognition",
    model="artificialguybr/whisper-small-pt-cv13",
    device=0  # or "cuda"
)

result = asr("path/to/portuguese_audio.wav")
print(result["text"])

For longer audio, use chunked inference:

result = asr("long_audio.wav", chunk_length_s=30, stride_length_s=5)

The model expects 16kHz mono audio. Resample if your source differs.

  • Language: Portuguese (forced via generate_kwargs={"language": "portuguese"})
  • Task: transcribe (not translate)
  • Audio: 16kHz, clear speech, minimal background noise
  • Chunk length: 30s for long-form with 5s stride

Limitations

  • Optimized for Portuguese only; other languages will degrade
  • Performance drops with low-quality audio, heavy accents, or noisy backgrounds
  • Not suitable for real-time streaming without additional optimization
  • WER of 10.3 means ~1 error per 10 words — review critical transcriptions

License

Inherits from base Whisper Small (MIT) and Common Voice 13.0 (CC0). Check the model card for any additional restrictions.

Common Errors and Fixes

Error | Cause | Fix

|---|---|---

Model not found | Wrong repo ID | Use artificialguybr/whisper-small-pt-cv13 exactly Poor accuracy on non-PT audio | Model is PT-specialized | Use base Whisper for multilingual OOM on GPU | Batch size too large | Reduce batch_size or use CPU offload Slow inference | No chunking on long audio | Add chunk_length_s=30 to pipeline call

FAQ

Q: How does this compare to Whisper Large-v3 for Portuguese? A: Whisper Large-v3 will generally be more accurate but 3× larger and slower. This fine-tune targets a size/accuracy sweet spot for Portuguese-only workloads.

Q: Can I fine-tune this further on my domain data? A: Yes. Use this as a starting checkpoint with a lower learning rate (e.g., 5e-06) and your domain audio.

Q: Does it support timestamps? A: Yes, the transformers pipeline returns chunk-level timestamps when return_timestamps=True.

Sources

  • Model card: https://huggingface.co/artificialguybr/whisper-small-pt-cv13
  • Base model: https://huggingface.co/openai/whisper-small
  • Dataset: https://huggingface.co/datasets/mozilla-foundation/common_voice_13_0
  • Training recipe: Hugging Face Whisper fine-tuning event