microsoft/VibeVoice gained 214 stars in a day on GitHub's trending page this week. It is the open repository for Microsoft Research's voice models: speech recognition that handles a one-hour recording in a single pass, a small realtime text-to-speech model, and a streaming recogniser that transcribes while audio arrives. The repository calls itself "Open-Source Frontier Voice AI".
The project is not new. Its long-form TTS model arrived in August 2025, and Microsoft later pulled that code. What trends now is the broader family, which has grown through 2026. This guide explains what each model does, how to try them, and the limits Microsoft states in its own README.
TL;DR: VibeVoice in one table
| Question | Answer, per the repository |
|---|---|
| What is it? | A family of open voice models: ASR (speech to text) and TTS (text to speech) |
| Who makes it? | Microsoft Research |
| Longest input for ASR? | Up to 60 minutes in one pass, within a 64K token window |
| What does ASR return? | Speaker (who), timestamps (when), content (what), with optional hotwords |
| Languages (ASR)? | Over 50, no language setting needed, per the Hugging Face model card |
| Realtime TTS? | VibeVoice-Realtime-0.5B, about 300 ms to first audible speech, streaming text input |
| Long-form TTS? | 1.5B model, up to 90 minutes and 4 speakers, but the code was removed in 2025 and the quick-try is disabled |
| Licence? | MIT on the VibeVoice-ASR model card |
| Does Microsoft recommend production use? | No. The README says not to use it commercially or in real-world applications without more testing |
What are the VibeVoice models?
The README lists five entries in its table. Three matter most for builders today.
VibeVoice-ASR: one hour, one pass
Most speech recognisers slice audio into short chunks. That loses context, and speaker tracking can drift. VibeVoice-ASR instead takes up to 60 minutes of continuous audio within a 64K token length, so the model sees the whole recording. The README says this keeps speaker tracking and meaning consistent across the hour.
Sound wave turning into tidy lines of text, illustrating VibeVoice long-form speech recognition
The output is structured. The model does recognition, speaker diarisation and timestamping together, so each segment says who spoke, when, and what. You can also pass custom hotwords, such as product names or jargon, to improve accuracy on domain-specific content. The technical report on arXiv was submitted on January 26, 2026 by a Microsoft Research team and revised in March.
The Hugging Face card lists the model at 9B parameters and says it supports over 50 languages and code-switching, with no need to pick a language. The README also lists fine-tuning code, vLLM inference and a Transformers integration (March 6, 2026), plus availability in Azure AI Foundry Labs (March 12, 2026).
VibeVoice-ASR-Streaming and the CPU build
Two later releases extend the ASR line. On September 3, 2026, Microsoft released VibeVoice-ASR-Streaming, which continuously transcribes "who said what" as speech arrives, with hotword support and 10 languages. On July 23, 2026, it released VibeVoice-ASR-BitNet, an edge CPU inference engine. The README says heterogeneous quantisation (I8_S plus I2_S) compresses the model from 4.62 GB to 1.58 GB. It runs through a separate C++ project, VibeASR.cpp, and the README says it reaches real time (real-time factor under 1) on 3 or more CPU threads with no GPU.
For anyone building a meeting recorder, call-centre review tool or lecture indexer, this pairing is the interesting part: a batch model for accuracy on long files, and a streaming one for live captions.
VibeVoice-Realtime-0.5B: small and fast TTS
The realtime model is the one you can run today. The README describes a lightweight 0.5B model with about 300 milliseconds to first audible speech, streaming text input, and robust speech generation for around 10 minutes. Streaming text input matters for voice agents: you can start speaking while a language model is still writing its reply.
Soft waveform with a green peak, illustrating VibeVoice realtime text-to-speech output
Microsoft added experimental speakers on December 16, 2025: multilingual voices in nine languages (German, French, Italian, Japanese, Korean, Dutch, Polish, Portuguese, Spanish) and 11 English style voices. There is a Colab notebook for a quick trial.
What happened to the 90-minute TTS model?
VibeVoice-TTS was released on August 25, 2025. According to the README, it synthesises up to 90 minutes of speech with up to 4 distinct speakers and natural turn-taking, and was accepted as an Oral at ICLR 2026.
Then the README records this note, dated September 5, 2025: after release, Microsoft "discovered instances where the tool was used in ways inconsistent with the stated intent", and so it removed the VibeVoice-TTS code from the repository. In the model table, the 1.5B TTS quick-try now reads "Disabled".
That is a rare, explicit pullback from a big lab for an open voice model. It matches the risk section of the README, which warns about deepfakes and disinformation: high-quality synthetic speech can be used for impersonation, fraud or fake audio. Microsoft expects lawful use and says it is best practice to disclose AI-generated content. For the fraud side, see our deepfake video call scam explainer.
How to try it
The README points to three routes:
- ASR playground. Microsoft hosts a demo at aka.ms/vibevoice-asr for VibeVoice-ASR. Upload audio, no install.
- Streaming TTS on Colab. A notebook in the repository (
demo/vibevoice_realtime_colab.ipynb) runs the 0.5B model. - Local install. Clone the repository, install with pip from
pyproject.toml, and download weights from the Hugging Face collection. vLLM plugin code is in thevllm_pluginfolder, and ASR fine-tuning code is infinetuning-asr.
git clone https://github.com/microsoft/VibeVoice
cd VibeVoice
pip install -e .
Follow docs/vibevoice-asr.md and docs/vibevoice-realtime-0.5b.md for exact commands. I did not run the models for this post, so I report only what the README and model card state.
How does it fit among open voice models?
Open speech models are crowding in, and they split by job.
| Job | VibeVoice | Other open options we covered |
|---|---|---|
| Long-recording transcription with speakers | VibeVoice-ASR, 60 minutes in one pass | Smaller on-device normalisers such as Superwhisper S1 Mini |
| Tiny in-browser TTS | Not the focus (0.5B is the smallest) | Gradium Phonon, 31M parameters |
| Multilingual voice cloning | Not offered in the current repo | VoxCPM2 |
| Local voice studio | Not a studio, a model repo | Voicebox |
| Voice agent pipeline | Realtime TTS plus streaming ASR parts | Hugging Face speech-to-speech guide |
Microsoft does not publish a head-to-head benchmark table in the README, so I make no claim about accuracy against Whisper or other recognisers. The reports linked from the README are the place to check word error rates for your language and audio type.
How it works, briefly
The README names one design idea behind the family: continuous speech tokenizers, acoustic and semantic, running at an ultra-low frame rate of 7.5 Hz. A low frame rate means fewer tokens per second of audio, which is what makes hour-long inputs fit in a language model's context. For generation, VibeVoice uses a next-token diffusion framework: an LLM handles text context and dialogue flow, and a diffusion head produces fine acoustic detail.
The README adds that the models inherit biases and errors from the base language model, named as Qwen2.5 1.5B in the TTS release. That is a useful reminder when you judge outputs: the language model behind the voice shapes pronunciation choices and mistakes.
Limits and risks to weigh
- Not for production, per Microsoft. The README says it does not recommend commercial or real-world use without further testing and development, and calls the release research and development only.
- Deepfake risk. Synthetic voices enable impersonation. The README asks for lawful use and for disclosing AI-generated audio.
- Missing pieces. The long-form multi-speaker TTS code is gone, so you cannot reproduce the 90-minute podcast demo.
- Hardware. The 9B ASR model needs a capable GPU. The BitNet build is the CPU route; the README claims real-time speed on 3 or more threads, which you should verify on your own hardware. The README table calls the model 7B while the Hugging Face card says 9B, so check the card for the build you download.
- Privacy. Meeting audio is sensitive. Running ASR locally avoids sending recordings to a cloud API, but you remain responsible for consent and storage.
- Licence details. The ASR model card states MIT. Check the card of each model you use, since the repository covers several releases.
What this means for builders
If you build transcription, minutes, lecture search or call analytics, the single-pass design removes the chunk-and-stitch step and its speaker-label mismatches. If you build voice agents, the realtime 0.5B model with streaming text input is a small, open option to test against hosted voices, such as the ones in Miso One.
A practical test plan: take three real recordings from your domain, one clean and one noisy, one with overlapping speakers. Run them through the ASR playground, compare speaker labels and timestamps by hand, and add your own hotwords for names. Measure word error rate on a hand-corrected sample before you commit.
Bottom line
VibeVoice is a useful, MIT-licensed set of open voice models from a top lab, strongest on long-form recognition with speakers and timestamps. Its trend spike reflects a growing family rather than a single launch. Treat it as a research release: test on your audio, read the reports, and keep the deepfake warning in mind.
Star counts, model lists and licences are accurate as of November 5, 2026 and may change in the repository.
Related reading
- Gradium Phonon: a 31M-parameter open TTS in your browser
- VoxCPM2: tokenizer-free multilingual TTS and voice cloning
- Miso One: an open real-time TTS voice model
- Voicebox: open-source AI voice studio
- Superwhisper S1 Mini: on-device transcript normaliser
- Hugging Face speech-to-speech voice agent guide
- Deepfake video call scam: how to verify