AnySpeechRecognizer and AnyDecisionModel: Open Speech and Decision Models for Swift Apps

Hugging Face engineer Mattt has announced two Swift packages for Apple platforms: AnySpeechRecognizer and AnyDecisionModel. Both follow the pattern of the earlier AnyLanguageModel, one familiar Swift API with a choice of model behind it, and both can pull open models from the Hugging Face Hub and run them on the device. The pitch is simple: language models are a small part of machine learning, and for jobs like transcription or routing a support ticket, purpose-built models are faster and cheaper than prompting a chatbot.

This matters to anyone building iOS or macOS apps with on-device AI, and to anyone tracking the growing family of "decision models" we have covered, including Cloudflare's Clef-Omni. The numbers in the announcement are the authors' own, so we flag what they measured and what they did not.

TL;DR: what you get

AnySpeechRecognizer: keep Apple's API, change the engine

Apple introduced SpeechAnalyzer in macOS 26 and iOS 26: you add modules such as SpeechTranscriber and SpeechDetector to an analyzer, feed it audio, and read results from async sequences. The only engine behind it is Apple's. AnySpeechRecognizer mirrors SpeechAnalyzer, SpeechTranscriber, DictationTranscriber, SpeechDetector and AssetInventory, so for code that uses the shared API, the post says changing the import from Speech to AnySpeechRecognizer is enough. To use Parakeet you change only how the transcriber is created, passing a model object, and the steps for installing assets, running the analyzer and reading results stay the same.

The engines

• Parakeet v3 is NVIDIA's Parakeet TDT 0.6B v3: 25 European languages with automatic language detection.
• Parakeet Ultra is Moondream's post-trained take on v3, which the authors found more accurate in most of their measurements.
• Parakeet Unified is NVIDIA's English-only Parakeet.
• Qwen3-ASR (0.6B and 1.7B) covers 30 languages and 22 Chinese dialects, including Mandarin, Cantonese, Japanese and Korean, which Parakeet does not. It returns one final result per segment of up to 40 seconds, with no live partial results, word timing or confidence, and it runs on the GPU more slowly than Parakeet, so it suits files better than live input.

Core ML conversions are published on the Hub under the mattt account, each recording its source checkpoint and a checksum for every file, with the package pinning a fixed revision. Licenses carry over from the originals: CC BY 4.0 for Parakeet v3, Parakeet Ultra and Sortformer v2, the NVIDIA Open Model License for Parakeet Unified and Sortformer v2.1, and Apache 2.0 for Qwen3-ASR. Check these before shipping a commercial app.

Speaker diarization, which Apple does not offer

Apple's Speech framework has no diarization module, so the package adds a SpeakerDiarizer running NVIDIA's Streaming Sortformer on Core ML alongside an Apple or Parakeet transcriber on the same timeline. It tracks up to four speakers, including when they talk over each other, and a SpeakerTimeline attributes each timed word to a speaker so you can print a transcript by turn. Sortformer v2 is the default; v2.1, trained on more meeting data, was more accurate on meetings in the authors' tests.

A laptop drawing with a small chip on its base, picturing Swift speech recognition running locally on Apple silicon

How well does it work?

The authors measured with their hear command-line tool on an M4 Max Mac Studio running macOS 27.0.1. Word error rates on LibriSpeech files of 3 to 8 minutes:

On speed, they report Parakeet transcribing an hour of audio in about 20 to 40 seconds from a file, or roughly 1.5 to 2.5 minutes live, with about 100 MB of memory once Core ML has compiled the model. Qwen3-ASR is much slower and needs a few gigabytes.

Two honest limits from the authors: these come from one machine and a few datasets, and meetings are hard for everyone. On AMI recordings, where people interrupt each other, every engine tested had a word error rate above 23%. LibriSpeech is read book audio, so it flatters every system compared with a noisy call. Our coverage of other open speech models, such as Granite Speech 5.0 and NVIDIA's Nemotron 3.5 Transcribe, shows how much the ranking shifts by dataset, so run hear evaluate on your own files.

AnyDecisionModel: probabilities instead of parsed text

Apps often ask a language model to decide: is this message urgent, which team should handle this ticket, how frustrated is this customer. The usual approach is to prompt and parse generated text, which yields one answer with no view of the alternatives. AnyDecisionModel instead derives each answer from the model's probabilities, returning typed Swift values.

enum Department: String, Choosable { case billing, technical, sales }

let model = MLXDecisionModel(modelID: "mlx-community/Qwen3-4B-Instruct-2507-4bit")
let session = DecisionSession(model: model, state: .text(ticket))

let urgent = try await session.probability(of: "Does this convey urgency?")
let team = try await session.choice("Which team?", from: Department.self)

A Choice holds the selected case, a probability for each option and a confidence value; a Score does the same for rating levels. Because your enum conforms to Choosable, the result works in an exhaustive switch. A session holds a single state, text or JSON, and every question is about that state; questions are independent, and you can batch them with decide(_:). For the MLX backend, the package builds a prompt with a one-token label for each allowed answer and reads the probability of each label from one forward pass.

If this idea sounds familiar, it is the same one behind the Jev family of single-token decision models; see our explainer on how Jev works and the ranked decision models list. The post is wry about it: decision models "aren't new, exactly," but there is "a lot of hype, and a crop of models trained to make decisions," so the authors wanted a Swift API ready for them.

The Emoji Magnet demo

To show the idea, the repository includes Emoji Magnet, a macOS app with 343 emoji. Type "things you wear" and matching emoji rise out of the pile; refine to "things you wear in winter" and non-matching ones fall back. There are no keyword rules: the app asks the model a yes-or-no question per emoji. With the hosted Jev backend, all 343 questions go in one request and settle in about half a second per phrase. Locally, Qwen3 4B on MLX answers 16 questions per forward pass, about 0.45 seconds per group on an M4 Max after loading, so a full pass over 343 emoji takes about 10 seconds.

A small chip inside a handheld device cutaway, picturing on-device AnyDecisionModel inference

Trained decision heads: Cloudflare's Clef

Reading next-token probabilities works with general models, but they were not trained to decide. The post points to Cloudflare's Clef models, which add a trained decision head that takes a state and a schema of typed questions and returns probabilities for all options in one pass. AnyDecisionModel runs Clef-Flash, the 9B variant, via MLXHeadDecisionModel. The 4-bit model is a 6.2 GB download and its model card recommends at least 16 GB of memory. Clef can read images and video, but this backend accepts only text and JSON. For the broader Clef picture, see our Clef-Omni post and the comparison of Clairic, Jev, OpenAI Decisions, Clef and Microsoft Decision 1.

Calibration, and why confidence is not correctness

Raw language-model probabilities tend to be overconfident, so the local backends accept a Calibration, which applies a temperature to every answer and a bias to yes-or-no answers; the post says to fit and evaluate on separate datasets. Models can also favor an option because of its position, so rotationDebiasing asks a choice question once per rotation of the options and averages, at the cost of one prompt evaluation per rotation.

The confidence value is based on the entropy of the probabilities. As the authors stress, it measures how concentrated the probabilities are, not whether the answer is right, so measure your own error rate before choosing a threshold. This lines up with an independent finding we covered: LLMs repeat 96% of confident errors, which is a reminder that high confidence is not accuracy.

Six backends

MLX runs a small model on Apple silicon; MLX decision heads run models with their own heads such as Clef; Core AI does the same as MLX on macOS 27 and iOS 27; llama.cpp reads decisions from a local GGUF model; Jev calls TypeSafe's hosted Jev through the System One API, or any server implementing it such as Kev; and Ollama calls a System One decision model locally or in Ollama's cloud (version 0.35.0 or later, or 0.40.5 or later for cloud models). The next-token backends label answers with yes/no, A to Z or 0 to 9, so a choice has at most 26 options and a score at most 10 levels; decision heads have no such limit.

What this means for what you build

• Replace brittle prompt-and-parse code. If your app classifies text, a typed enum with probabilities is easier to test and route on than free text.
• Choose speech engines per job. Parakeet for fast live English or European languages, Qwen3-ASR for Asian languages and file transcription, Apple's engine where you want zero extra download.
• Plan storage. Downloads run from about 250 MB to 6.2 GB, only when your app asks for them, and go into the standard Hugging Face cache so a Mac app outside the sandbox shares files with Python tools.
• Calibrate before trusting. Treat the confidence number as a signal, not a guarantee.

Setup and caveats

Add the packages to Package.swift:

.package(url: "https://github.com/mattt/AnySpeechRecognizer", from: "0.1.0"),
.package(url: "https://github.com/mattt/AnyDecisionModel", from: "0.6.0", traits: ["MLX"]),

Both need Swift 6.2 or later and macOS 26 or iOS 26 or later; the MLX build tasks use Xcode's Swift 6.4 or later and the Metal toolchain, and Core AI needs Xcode 27 and an OS 27 runtime. Without traits the decision package has no third-party dependencies and builds on Linux with the Jev and Ollama backends. One known conflict: AnyDecisionModel's CoreAI trait cannot be combined with AnySpeechRecognizer for now, because coreai-kit 0.7.3 requires swift-huggingface below 1.0 while AnySpeechRecognizer requires 1.x. Version numbers (0.1.0 and 0.6.0) signal early software, and the authors say they tested only on macOS 27, not iPhone.

What is claimed and what is verified

The package features and requirements are documented in the post and repositories, so you can check them yourself. The accuracy and latency figures are the authors' measurements on one M4 Max Mac Studio; they have published a natural-speech-eval.py tool to reproduce them, but we have not rerun it. The comparison with Apple's transcriber is on LibriSpeech only. Treat these as a starting point.

Related reading

• Cloudflare Clef-Omni open decision model
• Granite Speech 5.0 and open ASR leaderboards
• Nemotron 3.5 Transcribe
• How Jev works
• Top 10 decision models ranked
• IBM Granite Switch 4.1
• Official: Hugging Face announcement, AnySpeechRecognizer, AnyDecisionModel

Versions and measurements are accurate as of October 29, 2026.