Why accents change speech recognition accuracy, and what helps

A speech model’s accuracy changes with accent because the model learned the mapping from sound to text from a fixed pile of recordings, and any way of speaking that turned up less often in that pile gets recognised less reliably. Speech recognition accents are a property of the training data, not of the speaker. Nothing about your voice is wrong. The practical answer is the same for every voice: a closer microphone, one person talking at a time, a quiet room, and a read-through around names and technical terms.

Why speech recognition accents are a training data problem, not a speaker problem

OpenAI’s open-source Whisper model was trained by being shown audio with whatever transcript came with it, until it got good at predicting the second from the first. Its paper, “Robust Speech Recognition via Large-Scale Weak Supervision”, puts the scale at “680,000 hours of multilingual and multitask supervision”.

Where that audio came from matters more than the total. Section 2.1 says the dataset is built “from audio that is paired with transcripts on the Internet”, which gives a broad spread of environments, recording setups, speakers and languages.

Diverse is not the same as balanced. Whatever mix of voices sits on the public internet with a usable transcript is the mix the model learned from. The filtering the paper describes is all about transcript quality: dropping machine-generated captions, checking the spoken language against the transcript language, de-duplicating. None of it balances the data by accent or dialect, and the paper does not claim it does.

So the model ends up with a sharp sense of the vowels, rhythms and swallowed syllables it heard a lot of, and a blurrier sense of the ones it heard less. Given a sound pattern it has seen rarely, it still has to write words, so it writes the closest thing it knows.

What the model’s paper measures, and what it does not

The paper does test the data-quantity idea directly, but across languages rather than accents. Section 3.4 reports “a strong squared correlation coefficient of 0.83 between the log of the word error rate and the log of the amount of training data per language”, and estimates that “WER halves for every 16× increase in training data”. The limitations add that the pre-training data is very English-heavy, because the collection pipeline drew mainly from English-centric parts of the internet (section 6).

That is the clearest published statement of the mechanism, and the paper demonstrates it for languages, not for accents inside English.

The rest of the robustness work is about datasets and noise. Section 3.7 adds white noise and “pub noise” to the audio and finds that all the models tested “quickly degrade as the noise becomes more intensive”.

What is not there is a per-accent English benchmark. Read the paper through and the word “accent” appears exactly once, as the column heading en_accented in the VoxPopuli table in Appendix D.2.3. The authors never discuss that column or draw a conclusion from it. One corpus in the English evaluation was built by other researchers to study demographic bias: the Artie bias corpus, described in the paper’s reference list as “An open dataset for detecting demographic bias in speech applications”. The paper reports a word error rate for it, as it does for every corpus, and stops there, with no breakdown by speaker group.

We have not run an accent benchmark, and most quoted figures are not like-for-like

This is why the post has no chart of accuracy by accent. We have not tested FreeTranscribe across a set of English accents, and we are not going to imply otherwise.

Accent accuracy figures in circulation usually trace back to a model’s own evaluation tables, which are broken down per dataset and per language. Those datasets differ in far more than accent: read audiobook speech into a good microphone, spontaneous interviews, meeting-room audio from a ceiling mic. A gap between two of those numbers is a gap between two recording situations at least as much as between two ways of speaking.

A like-for-like test would hold the script, the room, the microphone and the model constant and vary only the speaker. We have not done that work. If we do, we will publish the audio and the method with the numbers.

Recording quality does more than accent for most files

In everyday files, what wrecks a transcript is rarely a subtle difference in vowels. It is a laptop microphone two metres away, three people talking over each other, a hard room with a second of reverb on every word, or speech sitting barely above an air-conditioning hum. Those fight the model on every word, for every speaker.

Move the microphone within arm’s reach and most of that goes away, because the voice arrives louder relative to the room. Agree at the start that people will not talk over each other, so the model is not untangling two voices from one channel. Shut the window and switch the fan off. Our post on recording audio that transcribes well goes through nine of these habits and why each matters.

What you can change, and what each change is worth

Nothing in that table is accent-specific, which is the point. The levers are the same for everybody, and worth pulling before you conclude anything about a voice.

Read the transcript where the model had the least to go on

Names and technical terms are the weak spot for every speaker, because they are rare in any training data and often cannot be worked out from context. The paper describes exactly this with names: during development the authors saw the models “transcribe plausible but almost always incorrect guesses for the names of speakers”, and fine-tuned them on transcripts without speaker annotations to stop it (section 2.4).

So search the transcript for the handful of names and terms you know were said, rather than reading the whole thing against the audio. Product names, place names, acronyms and numbers are where a few minutes buys the most. Our guide to editing a transcript fast lists the seven mistakes worth hunting for.

A larger model is generally more accurate than a small one

This is documented, and it is a property of the model family, not a claim about accents. OpenAI’s README lists six sizes, from tiny at 39 M parameters to large at 1550 M, with required video memory from about 1 GB to about 10 GB, and notes that the English-only .en models tend to perform better, especially at the tiny.en and base.en sizes. The paper’s scaling analysis is careful about where that holds: performance keeps rising with model size for multilingual recognition, translation and language identification, but English recognition is the exception, where the authors see diminishing returns that “could be due to saturation effects from approaching human-level performance” (section 4.1). Our post on speech model sizes walks through the table.

In a browser, size turns into download size. FreeTranscribe runs the base English model, about 200 MB fetched once and then cached, on your graphics card through WebGPU, in desktop Chrome or Edge. Nothing is uploaded, there is no account and no length cap, and English is the only language for now. Base is a compromise: fast enough on ordinary hardware, weaker than the large models on names, unfamiliar speech and noisy rooms.

To see how a particular recording fares, transcribe it on the home page and judge it on your own audio, not on anybody’s accuracy chart.

Frequently asked questions

Does a bigger model handle accents better? Larger models are more accurate overall on the datasets their authors tested, and OpenAI’s README and paper document that. Whether the gain is bigger or smaller for a particular accent is not something we have seen measured, so we will not put a figure on it.

Is my accent the reason a name came out wrong? Probably not on its own. A rare proper noun is rare in the training data no matter who says it, and the paper documents the model guessing speaker names wrongly often enough that the authors fine-tuned the behaviour out. Check every name, every time.

Should I speak slower or change how I speak? We have no measurement that says performing a different voice helps, so we will not recommend it. The levers we can point at are the recording ones: microphone distance, one speaker at a time, a quiet room and a healthy level. Your normal pace into a close microphone beats speaking oddly into a distant one.

Does FreeTranscribe work for languages other than English? Not yet. It runs the English-only base model, in desktop Chrome or Edge with WebGPU. Other languages, Firefox, Safari and phones are not supported today.

Sources, checked 15 September 2026

• https://arxiv.org/html/2212.04356 - Radford et al., “Robust Speech Recognition via Large-Scale Weak Supervision”: abstract (680,000 hours), section 2.1 (internet audio paired with transcripts, filtering, audio language detector), 2.4 (speaker name guessing), 3.4 (0.83 squared correlation, WER halves per 16x of data), 3.7 (white noise and pub noise), 4.1 (model scaling and saturation for English), 6 (English-heavy pre-training data)
• https://arxiv.org/html/2212.04356v1 - The same paper, used to check every occurrence of the word “accent”: Appendix A.1 (English evaluation datasets, including the Artie bias corpus) and Appendix D.2.3 (the en_accented column in the VoxPopuli table)
• https://github.com/openai/whisper - README: model size table (tiny 39 M to large 1550 M, turbo 809 M, about 1 GB to 10 GB of video memory) and the note that the .en models tend to perform better