Tencent Hy-SimulMT: An Open 30B Model for Real-Time Speech Translation

Tencent Hunyuan has released Hy-SimulMT, an open simultaneous translation model that translates speech while the speaker is still talking. It handles Chinese to English and English to Chinese, can keep the speaker's voice through real-time voice cloning, and is listed under Apache-2.0. The FP8 checkpoint has picked up 194 likes on Hugging Face since its October 22 upload, and the release is dated October 26 on the model card.

The headline claim is latency. On the RealSI long-form audio benchmark, Tencent reports a StreamLAAL of roughly 3 seconds, the lowest of the systems in its comparison, with a quality score (GEMBA) above the open-source models and four of five commercial APIs it tested. Those are Tencent's own numbers, so this post separates what the vendor measured from what you can check yourself.

TL;DR: Hy-SimulMT at a glance

Why simultaneous translation is a harder problem than translation

Ordinary machine translation sees a whole sentence. Simultaneous translation does not: it has to commit to words before the speaker finishes the thought. Wait longer and the meaning is clearer but the listener falls behind. Speak earlier and the system risks mistranslating something that a later clause would have fixed.

Tencent frames Hy-SimulMT around that trade-off. In the model card's words, the aim is to convey information accurately "while reducing the disruption that waiting introduces into a conversation." Human conference interpreters face the same tension and typically trail the speaker by a few seconds, which is why the human comparison in this release matters.

Most deployed live-translation products work as a cascade: speech recognition, then text translation, then speech synthesis, each with its own delay. Hy-SimulMT instead takes audio in and emits translation as the stream arrives, with text-to-speech and voice cloning layered after it. That design is why the speech half depends on an external TTS service while the text half does not.

What the benchmarks show

Simultaneous translation concept: two speech bubbles joined by a bridge

Tencent evaluates on two kinds of data. The first is the long-form RealSI audio benchmark, plotted as StreamLAAL (lag in seconds, lower is better) against GEMBA quality (higher is better). Hy-SimulMT sits at about 3 seconds of lag, the lowest on the chart, with quality above the open-source models and four of the five commercial APIs. Tencent also notes that "StreamLAAL is not the same as the time to first speech output," so do not read 3 seconds as the delay before you hear the first word.

The second is four short-form benchmarks. Here are the scores from the model card:

Source: Tencent Hunyuan model card. Higher is better.

Reading this table honestly: Hy-SimulMT is not the winner everywhere. A closed Gemini live-translation system leads on FLEURS and ACL6060, and a Qwen live-translation system leads on CoVoST2 by a wide margin (82.1 against 75.5). Hy-SimulMT's clear win is RealSI-Sentence at 84.7, and Tencent says it beat the other open-source models in the table on FLEURS, ACL6060 and RealSI-Sentence. The pattern is "competitive across the board, best on latency," not "best at everything."

The product names in the table are as listed by Tencent. Because the comparison is vendor-run, differences of a point or two on a single benchmark should not drive a purchasing decision.

The human interpreter comparison

Hy-SimulMT simultaneous translation latency compared with CATTI interpreters Image: Tencent Hunyuan

The most interesting experiment is also the least quantified. Tencent collected 18 Chinese and English recordings, each about six minutes, that appeared online from September 2026 onward. They span ten domains, including software and AI, medicine, manufacturing and energy, law, education and sports. Six professional interpreters (three CATTI Level 1 and three CATTI Level 2) translated them, and for each level the best of the three was used as the comparator.

Tencent's conclusion is that Hy-SimulMT matches expert interpreters on translation quality and often responds with lower latency. The latency comparison is the chart above. The model card publishes no numeric quality scores for this test, and it adds that the findings "apply only to the samples, participating interpreters, and evaluation settings used in this test."

Treat that as a promising signal rather than a result. Eighteen six-minute clips is small, the recordings were selected by the vendor, and interpreters working from a recording are not in the same situation as interpreters working a live room. Using recent recordings from after September 2026 is a sensible choice because it reduces the chance the model saw the audio in training, but it does not remove the selection question.

Voice cloning and the TTS catch

Hy-SimulMT can keep a speaker's vocal characteristics in the translated speech. The model card offers demos for English to Chinese (three speakers) and Chinese to English, but warns that these are offline replays, so their timing is not a live latency measurement. It also does not assess voice similarity or speech quality in general.

The practical catch: speech output and cloning require a configured TTS service. The model itself produces translated text. If you only need captions, the text path is enough. If you want a translated voice, you must wire in speech synthesis, and the SimulInfer documentation on GitHub covers that setup.

Voice cloning of people who did not consent is a real misuse vector. If you deploy this for meetings, tell participants their voice is being cloned and keep the audio on infrastructure you control.

How to run Hy-SimulMT

There are three documented routes.

SimulInfer on vLLM. This is the full pipeline including voice-activity handling and recovery heuristics. You need Linux, Python 3.12 or newer, Git, an NVIDIA GPU with enough memory and a CUDA-compatible driver. An existing environment needs vLLM 0.31 or later with HYV3ForCausalLM support. The model card's command is:

bash deploy.sh --install-server --model tencent/Hy-SimulMT-30B-A3B-Interleave

The service listens on port 8080 for the web and API layer, with the model on 8800. Files go to /translate/stream, and /docs shows the API.

SGLang-Omni. This covers only the BF16 checkpoint Hy-SimulMT-30B-A3B, returns translated text only, and fits on a single H200. Files are limited to 30 seconds, and live audio uses a WebSocket at /v1/realtime with the --enable-realtime flag. It does not reproduce SimulInfer's voice-activity and recovery heuristics, so quality in long sessions may differ.

llama.cpp. Support is in an unmerged pull request (number 31198) and currently works only through llama-mtmd-cli. Do not plan around it yet.

The Transformers snippet on the card works, but the translation pipeline is no longer supported in Transformers v5, so load it as a causal language model. If you are choosing between serving stacks in general, our notes on SGLang's plugin system and vLLM's disaggregated inference work explain the trade-offs.

Where this fits among live voice models

Real-time speech is a crowded area this month. ElevenLabs shipped a realtime speech-to-speech model with an OpenAI-compatible API, xAI launched Grok Voice 3.0, and ChatGPT's live voice gained new voices in six languages. Mistral's Voxtral 3 targets voice agents. Those are general conversational voice products. Hy-SimulMT is narrower: a translation specialist with open weights you can host.

It is also part of a run of open releases from Tencent Hunyuan, which has recently put out the Hy-Memory-30B-A3B memory writer, the Hy-Nomas interactive video world model and the Youtu-Parsing omni-modal parser. The Hy-Memory model shares the same 30B-A3B shape, which suggests Tencent is standardizing on that size for deployable open models.

What this means for what you build or pay

If you build meeting tools, captioning, or customer-support products for Chinese and English audiences, an Apache-licensed model changes the cost picture: you can host it, inspect it and fine-tune it with SimulTrain instead of paying per minute to a closed API. The trade is operations. You run GPUs, pick a TTS stack, and own latency tuning.

If you need other language pairs, this is not your model yet. Only Chinese and English are supported for speech translation, even though the Hugging Face tags list dozens of languages for the base model family. Check the card before assuming coverage.

For a quick evaluation, run your own audio rather than trusting the chart. A sensible test plan:

1. Collect 10 to 20 clips from your domain, with accents and background noise typical of your users.
2. Run them through the SimulInfer service and measure the time between each source phrase and its translation, not just overall lag.
3. Have a bilingual reviewer score meaning errors, especially on names, numbers and technical terms.
4. Compare against your current vendor on the same clips.

What we could not verify

• The RealSI comparison, the short-form table and the interpreter test are all Tencent's. We did not rerun them.
• The numeric StreamLAAL values for the interpreters are only in a chart image, not in text.
• Download counts were small at the time of writing (93 on the FP8 repository), so there is little independent community testing yet.
• The model card lists the Apache-2.0 license in metadata. Read the LICENSE.txt in the repository before shipping commercially.

We will add an update section if independent latency tests appear.

Frequently asked questions

What is Hy-SimulMT? A Tencent Hunyuan model for live Chinese and English speech translation, with optional voice cloning through an attached TTS service.

Is it really open? The weights and tooling are public and the metadata says Apache-2.0, but check the repository license file for your use.

How does 3 seconds compare to human interpreters? Tencent says it is often lower than expert interpreters in its small test, but gives no numbers in text, so treat it as a claim.

Can I run it locally? On a GPU server, yes, through vLLM or SGLang-Omni. A llama.cpp path is not merged yet.

Related reading

• Tencent Hy-Memory-30B-A3B
• Tencent Hy-Nomas world model
• Tencent Youtu-Parsing
• ElevenLabs Eleven Realtime
• Grok Voice 3.0
• Voxtral 3

Model details, benchmarks and repository status are accurate as of October 28, 2026 and may change as Tencent updates the model card.