Why speech recognition invents words in silence, and how to stop it

Speech recognition invents words in silence because the model behind most free tools, OpenAI’s open-source Whisper model, is built to turn every 30-second slice of audio into text, and it does that even when the slice holds nothing but room tone. Given near-silence, it writes the most probable text it learned from internet captions, which is how “Thank you for watching” or a phrase repeated ten times ends up over a pause. That is the Whisper hallucination people search for. The fixes are trimming silences, keeping the level healthy, avoiding music beds, and reading around pauses.

Where the invented words come from

The model’s own paper, “Robust Speech Recognition via Large-Scale Weak Supervision”, describes what it was trained on: 680,000 hours of audio from the internet paired with whatever transcript came with it. Its job in training was to predict the transcript from the audio.

The architecture matters here. It is an encoder-decoder Transformer, a sequence-to-sequence model. The encoder listens to a 30-second window of audio. The decoder then writes the transcript one token at a time, and each token is chosen from two things: what the encoder heard, and what the decoder has already written.

The training set included segments with no speech, but at a reduced rate (“with sub-sampled probability”), and the model was trained to emit a special no-speech token for those. So when a window is quiet, the decoder has a choice. It can say “no speech here”, or it can start writing text.

A real recording is never dead silent. A hum, a chair creak, breathing, a music bed, or a colleague two desks away can tip the decoder down the “there is speech” path. Once it has decided to write, it writes the kind of text that usually goes with audio like that in its training data. The paper doesn’t list which phrases come out, so we won’t guess. Text is the default, and silence is the exception the model has to actively choose.

Why Whisper hallucination repeats the same phrase

Repetition is the second half of the problem, and it comes from the decoder feeding on itself. Each token is chosen partly from the tokens before it. When the audio offers no evidence to anchor the next word, the safest continuation of “thank you for watching” is often “thank you for watching” again. Nothing in the sound contradicts it, so the loop runs until the window ends.

The authors say this in so many words. In the limitations section they put the remaining long-form errors down to failure modes of seq2seq models, language models and text-audio alignment, which “include problems such as getting stuck in repeat loops, not transcribing the first or last few words of an audio segment, or complete hallucination where the model will output a transcript entirely unrelated to the actual audio.”

Section 4.5 describes the counter-measures they built into their own decoder. Four tell you what the model’s makers saw going wrong:

Beam search with five beams, “to reduce repetition looping which happens more frequently in greedy decoding”.
Retrying a window with more randomness when the generated text compresses too well with gzip (a ratio above 2.4). Repeated text compresses extremely well, so this is a repetition detector.
Retrying when the average log probability of the output is low, meaning the model wasn’t confident in what it wrote.
Treating a window as silent only when the no-speech probability is above 0.6 and the confidence is also low, because the no-speech signal on its own was not reliable enough.

Two things follow. Even the authors could not stop the loops with the model alone; they needed checks on the output. And their own silence detector needed a second opinion, so the model saying “this is speech” over a quiet stretch is a known weakness, not a fault in your file.

Symptoms, likely causes and fixes

Symptom	Likely cause	Fix
A stock sign-off such as “Thank you for watching” where nobody spoke	Near-silent window; the decoder chose text over the no-speech token	Trim the silence, or delete the line and check the timestamps on either side
One sentence repeated many times in a row	Decoder loop inside a single window	Cut the file just before the loop and transcribe that piece again
The same phrase drifting across several minutes	A loop that started in one window and kept going	Split the recording at the first repeat and run the parts separately
Text appears over a music intro or outro	Music is not speech, but it isn’t silence either	Cut the music out before transcribing
Words invented in the gaps of a very quiet recording	Speech sits close to the noise floor, so pauses and speech look alike	Raise the recording level or normalise the file first
First few words after a long pause are missing	A separate failure the paper names alongside hallucination	Read around every long pause and type the missing words in

What you can do before you transcribe

Most of the fixes happen in an audio editor, not in the transcription tool, and they take a few minutes.

Cut leading and trailing silence and music. The start and end of a recording are where the longest non-speech stretches live.
Shorten long pauses in the middle. Most editors can shorten or remove silence below a chosen level. A two-second gap is fine. A forty-second one is asking for trouble.
Keep the level healthy. If the loudest speech barely moves the meter, normalise the file first. Quiet speech and quiet noise are hard to tell apart, and the model guesses.
Split at natural breaks. A three-hour recording with a lunch break in the middle is two recordings. Transcribe them separately and a loop that starts in the break can’t run through the afternoon.
Read the output around every pause. Hallucinated text sits over a stretch where the waveform is flat. If you export SRT or VTT, the timestamps show you exactly where to look. Our post on SRT, VTT and plain text explains the trade-off.

None of this changes the model. It changes what the model is asked to listen to, which is the one lever you fully control.

What FreeTranscribe does about it automatically

FreeTranscribe runs OpenAI’s open-source Whisper model (the base English version) on your graphics card through WebGPU, in desktop Chrome or Edge, and nothing is uploaded. Because we control the code around the model, we add checks that catch the common hallucination shapes before they reach your transcript:

Quiet windows are skipped. The audio is cut into roughly one-minute windows, with each cut nudged to the quietest moment nearby so we don’t slice through a word. If a whole window’s loudness is below a fixed threshold, it isn’t sent to the model at all. No audio in, no invented text out.
Exact repeats are dropped. If a segment is the same as the previous one, ignoring capitals and punctuation, it’s discarded.
Loops inside a segment are dropped. If a segment of nine words or more contains any three-word phrase four or more times, the whole segment is discarded.
Mid-sentence fragments are joined. A segment that doesn’t end in a full stop, question mark or exclamation mark is held and glued to the next one if the gap is under a second and a half, so the transcript reads as sentences rather than the model’s 30-second processing boundaries.

These rules reduce the problem. They don’t eliminate it. A window that is quiet but above the threshold still goes to the model. A phrase that repeats three times passes the loop check. A sign-off written once over a pause looks like a normal sentence to every rule above, and only you can tell it doesn’t belong. Read the transcript through before relying on it, especially around pauses. The base model is also weaker than the larger ones on names, accents and noisy rooms.

If you want to try it on a file that gave you trouble elsewhere, drop it on the home page. There is no account and no length cap, so a long recording with long pauses is a fair test.

Frequently asked questions

Does a bigger model fix hallucinations in silence? Not by itself. The paper says larger models made “steady and reliable progress” on perception errors like confusing similar-sounding words, but that the long-form errors, repeat loops and hallucination among them, “seem more stubborn in nature”. The authors’ answer was decoding checks, not just scale.

Why does it write “Thank you for watching” in particular? The paper doesn’t say, and we haven’t measured it, so we won’t claim a reason. What it does say is that the model learned from transcripts of internet audio, and that text is what it produces unless it actively picks the no-speech token.

Can I just delete the repeated lines from the SRT? Yes. Delete the whole cue, including its index number and timestamp line, then renumber if your player is fussy. Our guide to making an SRT file covers the format.

Will this happen with other transcription tools too? The paper describes repeat loops and hallucination as failure modes of the model itself, so what differs between tools is what the surrounding software does about them. We can only speak for ours.

Sources, checked 14 September 2026

https://arxiv.org/abs/2212.04356 - Radford et al., “Robust Speech Recognition via Large-Scale Weak Supervision”, abstract page (title, authors, 680,000 hours)
https://arxiv.org/pdf/2212.04356 - Full paper: section 2.1 (30-second segments, no-speech segments sub-sampled), 2.2 (encoder-decoder Transformer), 4.5 (beam search, temperature fallback, gzip ratio 2.4, no-speech threshold 0.6), 6 (repeat loops and hallucination as seq2seq failure modes)
https://arxiv.org/html/2212.04356 - HTML rendering of the same paper, used to cross-check the quoted passages
https://github.com/openai/whisper - README: MIT licence, model table (base is 74 M parameters with an English-only base.en variant)