Haru-4B is a new open voice model that listens and talks at the same time. higurashi-ai published it on Hugging Face under the Apache License 2.0, with code on GitHub and a technical blog. It was trending on Hugging Face with 229 likes when we checked.
The model is a pretrained base for research. It uses a Qwen3.5-4B text backbone, handles nine languages, and fits on a GPU with about 12 GB of memory. The authors report about 65 ms of end-to-end latency on an M1 Max MacBook. They report no benchmark scores, and that gap matters for how you should read the release.
This post explains how full-duplex differs from the usual voice pipeline, how Haru-4B is built, how to run it, and what is still unproven. For other voice releases we have covered, see Fish Audio Live 3 and Retell's Tell-1.
TL;DR
| Question | Answer |
|---|---|
| What is it? | A full-duplex speech-to-speech foundation model, about 5B parameters in BF16 |
| License | Apache 2.0 for the weights |
| Base model | Qwen3.5-4B text backbone, 32 layers, hidden size 2560 |
| Languages | English, Spanish, Portuguese, German, French, Chinese, Japanese, Russian, Korean |
| Hardware | About 12 GB NVIDIA GPU, or a Mac through the MLX build |
| Latency | About 65 ms end-to-end on an M1 Max (authors' figure, excludes up to 133 ms of audio framing) |
| Is it an assistant? | No. Base model, weak on facts and instructions |
| Benchmarks | None published yet |
What does full-duplex mean, and why does it matter?
Most voice assistants work in turns. A voice activity detector decides you have stopped, speech recognition turns your audio into text, a language model writes a reply, and a speech synthesizer reads it aloud. Each step adds delay, and the system cannot listen while it speaks. That is why interruptions feel awkward.
A full-duplex model processes incoming and outgoing audio in the same loop. In Haru-4B, every 133 ms frame takes one frame of the user's audio and produces one frame of the model's own audio and text. That lets the model do what people do on a phone call: say "mm-hm" while you talk, stop when you cut in, or start a reply before you finish.
According to the authors, the goal is conversation that feels less like an agentic assistant and more like talking with a person. That is a different target from tool-using voice agents. Our post on the Japanese full-duplex model Shizuku covers the same shift from another team.
Voice waveform illustrating a full-duplex speech model that listens and speaks at once
How is Haru-4B built?
The model card and technical blog describe a compact design with three ideas.
One position, three streams. User audio, the model's previous text token and its previous speech latent are summed into a single position of the language model. The blog says this cuts context length to one third, compared with interleaving the streams token by token. Listening and speaking never wait for each other.
Continuous audio, no codebooks. Audio is a 64-dimensional latent per frame at 7.5 Hz. A small MLP head produces each latent in a single sampling step from the language model's hidden state. The head is trained with flow matching and then distilled with MeanFlow. There is no separate audio transformer and no discrete audio tokens.
A known codec. The audio codec is the VibeVoice acoustic tokenizer, causal, at 24 kHz with 3,200 samples per latent.
| Part | Detail |
|---|---|
| Language model | Qwen3.5-4B text backbone |
| Audio head | 4-layer MLP, MeanFlow, 1 step, guidance scale 1 to 2 (default 1.5) |
| Frame rate | 7.5 Hz, about 133 ms per frame |
| Weights | One safetensors file in bfloat16, plus config and tokenizer |
The model is controlled by a text card that sets the character and a short reference clip that sets the voice.
What data and compute went into it?
The authors say they trained on more than 40 million hours of speech covering more than 160 languages. They exclude LLM-generated dialogues and TTS-synthesized speech, so the model learns turn-taking, backchannels and interruptions from real speech.
Training had two stages:
- Single-channel adaptation. The model learned speech adaptation through speech recognition, speech and text continuation, and text-only rehearsal. Most data was cleaned with a speech restoration model called Sidon.
- Full-duplex training. A separation model called FastDialogueSidon split mixed recordings into aligned two-channel streams. The authors report more than 13 million hours of separated dialogue, with conversations up to 60 minutes long. Character prompts and same-speaker reference audio supplied persona and voice.
The reported compute is 256 NVIDIA B200 GPUs for about 8 days, using Megatron-LM. These are the authors' numbers and we have not reproduced them.
One caveat sits inside the method. The model was pretrained without native two-channel conversations, because the two-channel data came from separating mixed audio. The blog admits that separation errors and noise can remain, especially in overlapping speech.
How fast is it?
The blog gives a latency breakdown for a native Swift app with custom Metal kernels and int8 weights on an M1 Max. These are median figures.
| Stage | Latency |
|---|---|
| Input, microphone to app | 5.3 ms |
| One model step | 34.2 ms |
| Socket and handoff | 4.1 ms |
| Output wait, margin and slack | 13.3 ms |
| Output, app to headphones | 8.1 ms |
| Total | 65.2 ms |
The total excludes audio framing, which adds up to 133 ms. On an NVIDIA B300 through vLLM-Omni, the authors say audio encoding and decoding plus one language-model step take about 15 ms.
Treat these as vendor-reported lab numbers. Real conversations add network, device and Bluetooth delay. The model card itself says Bluetooth devices add 100 ms or more, and it recommends wired headphones and a wired microphone.
Real-time voice concept for a low-latency open source voice model
How do I try Haru-4B?
The card gives a short path for a machine with an NVIDIA GPU of about 12 GB:
git clone https://github.com/higurashi-ai/Haru && cd Haru
pip install -e ./haru
python -m haru.server --hf-repo higurashi-ai/Haru-4B
Then open localhost on port 8998, pick a character, give the model a voice and talk. The repository also includes an MLX implementation with a native macOS app and a plugin for serving through vLLM-Omni.
The model ships with no voice clips. You supply one: a single speaker, 3 seconds to 3 minutes, clean speech. The card adds a plain warning to use only voices you have the right to use. That matters because the model can imitate a reference voice, and impersonation is a real risk. See our report on AI synthetic voices and 911 swatting for how misuse looks in practice.
For programmatic use, the card shows a DuplexSession class. You prepare a reference voice, start a session with a character card, and call a step function with 3,200 samples of 24 kHz audio per frame. The system prompt uses a fixed character-card layout that the repository README documents.
How does it compare with other open voice models?
Here the honest answer is: we do not know yet. The authors cite Moshi, FLM-Audio and OmniFlatten as related systems, but report no head-to-head numbers. The blog's only evaluation is a qualitative demo, an English investment-advice conversation that they say shows backchannels and overlapping speech.
That puts Haru-4B in a different spot from releases that publish scores. A few reference points from our own coverage:
- Apriel Voice 27B preview is a larger open speech model with tool calls, under an MIT license.
- Zonos 3 Mini is an open text-to-speech model that runs in the browser.
- Confucius4-R2T2 targets streaming speech recognition for Qwen3 voice agents.
Those are different jobs. Haru-4B is a conversation model, not a recognizer or a speaker. If you need a reliable assistant today, a pipeline with a stronger language model will answer questions better. If you want to study natural turn-taking, an open full-duplex base you can run locally is the useful part.
What are the limits?
The model card and blog list these:
- Base model only. It holds a natural conversation but does not follow instructions the way an assistant does.
- Weak on facts. The card says it is not a reliable source of facts. The authors also say the small backbone limits response quality.
- Separation noise. Training data came from separated audio, so overlapping speech can carry errors.
- No published benchmarks. Quality claims rest on one demo.
- Voice rights. You carry the responsibility for any voice you clone.
Planned work, per the blog: a more detailed technical report, training code, fine-tuning on native two-channel recordings, synthetic full-duplex data, models above 100 billion parameters, and omni-modal models that take live video.
What this means for builders
If you build voice products, Haru-4B is worth a test session, not a migration. Three practical uses stand out:
- Research on turn-taking. An Apache-2.0 base with training details lets you probe interruption and backchannel behavior.
- On-device demos. The MLX path and the reported 65 ms figure suggest a local conversational demo on a Mac is feasible. Verify on your own device.
- Fine-tuning candidates. At roughly 5B parameters with an open license, it is a reasonable base for a domain-specific character, once you add your own data and evaluation.
Do not ship it as a customer-facing agent without your own tests for factual errors, safety and voice consent. For commercial voice agent benchmarks with latency claims, compare against vendor posts like Retell's Tell-1, and check how each source measures delay.
Specs, license terms and repository details are accurate as of November 14, 2026 and may change.