Home → Blog → How to improve speech-to-text accuracy: eight practical fixes

How to improve speech-to-text accuracy: eight practical fixes

Published 2026-08-05 · 7 min read

“Recognition is bad” is not a diagnosis. In almost every real case the quality problem is one of a handful of concrete causes — and most of them are fixed on your side, before the audio ever leaves your server. Here they are, in the order in which they are worth trying.

First: measure, do not guess

Without a number, every change is a matter of taste. Take 20–30 recordings that look like your production traffic, write down what was actually said, and count the word error rate: substitutions plus deletions plus insertions, divided by the number of words spoken.

pip install jiwer
from jiwer import wer

score = wer(reference_text, recognised_text)
print(f"WER: {score:.1%}")

Two things matter more than the absolute value. First, measure on your audio, not on a public benchmark. Second, weight by what you care about: if the business needs order numbers, a 5% WER with every number wrong is a failure, and a 15% WER with clean numbers is a success.

Fix 1. Give the model clean audio

Recognition models work internally with mono at 16 kHz. Anything above that is discarded, and anything below that is lost forever:

ffmpeg -i input.wav -ac 1 -ar 16000 -b:a 64k clean.mp3

Heavy noise reduction is the exception that usually backfires: aggressive filters chew up consonants and recognition gets worse, not better. Fix noise at the microphone if you can.

Fix 2. Say what language it is

Automatic detection is right almost always on long recordings and noticeably less often on three-second clips. If the stream is single-language, say so:

r = client.audio.transcriptions.create(
    model="whisper-1", file=f, language="en",
)

This removes a whole class of failures where a short phrase is recognised as another language and comes back as nonsense. It also speeds processing up slightly, since the detection pass is skipped.

Fix 3. Feed in the vocabulary through prompt

The single highest-value parameter, and the most often ignored. The prompt does not appear in the response — it biases recognition towards the words you list:

r = client.audio.transcriptions.create(
    model="whisper-1", file=f,
    prompt="Products: Kestrel, Kestrel Pro, Hawknest. Agents: Duarte, Ivanova, Ng.",
)

Keep it short and specific. A list of 20–40 terms that actually occur works; a paragraph of general description does nothing. Rotate the list per context: a support queue about billing needs different terms than one about hardware.

The prompt also carries style. If your transcripts should keep punctuation and capitalisation, write the hint as a properly punctuated sentence — the model tends to continue in the style it was given.

Fix 4. Cut the silence

Long silent stretches are where hallucinations come from: with nothing to recognise, a model can produce filler text. They also cost money, because billing is per minute of audio.

ffmpeg -i input.mp3 -af silenceremove=start_periods=1:stop_periods=-1:stop_duration=2:stop_threshold=-45dB out.mp3

Be careful with the threshold: cutting too aggressively removes quiet speech along with the silence. Check the result on a few files before putting it in a pipeline.

Fix 5. Split on pauses, not on the clock

Long recordings have to be split anyway to fit the 25 MB limit. Where you cut matters: a chunk boundary in the middle of a sentence loses the words on both sides of it.

# split on silence rather than at a fixed minute
ffmpeg -i long.mp3 -af silencedetect=noise=-40dB:d=0.7 -f null - 2> pauses.txt

Use the detected pauses as candidate cut points, and keep chunks in the 5–20 minute range. Shorter chunks lose context between them; much longer ones bring no extra benefit.

Fix 6. Ask for the format that carries confidence

verbose_json returns not only the text but also the segments and the detected language with its probability. That is the raw material for automatic quality control:

r = client.audio.transcriptions.create(
    model="whisper-1", file=f, response_format="verbose_json",
)

if r.language_probability < 0.6:
    queue_for_review(r)          # suspicious recording, look at it by hand

Routing the doubtful few per cent to a human review queue improves the quality of the whole pipeline far more cheaply than trying to squeeze the last percent out of recognition itself.

Fix 7. Post-process instead of re-recognising

Some errors are systematic: the same product name comes back wrong the same way every time. A dictionary of replacements fixes it in one line and costs nothing:

FIXES = {"cast rail": "Kestrel", "hawk nest": "Hawknest"}

for wrong, right in FIXES.items():
    text = text.replace(wrong, right)

Build the dictionary from real mistakes you have seen, not from imagination. Ten entries collected over a week of production traffic typically remove more visible errors than any parameter tuning.

Fix 8. Catch empty and invented output

Two failure modes are worth handling explicitly in code: an empty result, and a result that is suspiciously generic. Hallucination filters cut the classic cases such as closing credits and “thanks for watching”, so an empty string genuinely means there was no speech.

text = r.text.strip()
if not text:
    return None            # silence, not an error — do not retry

Retrying an empty recording only spends minutes. Log it, skip it, and move on.

What will not help

Try it on your own recordings. Sign-up takes a minute, and the free minutes are enough to judge the quality.

Get a free API key

Frequently asked questions

What word error rate should I expect?

On clean speech in a well-supported language, single-digit percentages are normal. Phone audio, background noise, crosstalk and strong accents raise it. Measure on your own recordings — a public benchmark says nothing about your traffic.

Does the prompt parameter really change anything?

Yes, and it is the cheapest improvement available. A short list of the names, products and codes that occur in your audio noticeably reduces errors on exactly the words that matter most.

Should I specify the language or let it be detected?

Specify it when the stream is single-language, especially for short clips. Leave detection on when messages genuinely arrive in different languages.

Is it worth removing noise before sending?

Light normalisation helps; aggressive noise reduction usually hurts, because it damages consonants. If the noise is severe, fixing the recording setup beats any filter.

How large should the chunks of a long recording be?

Five to twenty minutes, cut at pauses rather than at fixed intervals. That keeps sentences intact and stays inside the 25 MB request limit.

Related reading