KRAFTON Raon-1 mini Preview: A Korean 29B MoE Model Under Apache 2.0

KRAFTON AI has put a new open-weight model on Hugging Face: Raon-1 mini preview, a 28.8B-parameter sparse mixture-of-experts model with about 4.3B parameters active per token, licensed under Apache 2.0. The model card says it was "designed and trained from scratch" and targets Korean conversation and agentic work. The headline numbers are the company's own, so read them as claims until others reproduce them.

It matters for two reasons. It is a rare from-scratch open model out of a game company, and its architecture mixes linear attention with a few full layers, which keeps long-context memory small. If you run local models, the hardware notes are the practical part.

TL;DR

What is inside the model?

The architecture is where Raon-1 mini departs from the usual recipe. According to the model card, it has 25 layers: 18 use linear attention, which KRAFTON calls Raon Delta Attention, and 7 use gated multi-head latent attention. The card describes the linear layers as an improvement on Kimi Delta Attention and a component called Raon-mHC as an improvement on DeepSeek's manifold-constrained hyper-connections.

Other specs from the card: hidden size 2,560, a 259,840-token vocabulary, an untied output head, and no positional encoding (NoPE). Six routed experts and one shared expert fire per token.

Because only the 7 latent-attention layers keep a key-value cache, the card puts the cache cost at about 8 KiB per token in BF16. That is the reason a 262K context is plausible on a single high-end card. For background on how sparse routing works, our explainer on running MoE models larger than RAM covers why total and active parameter counts differ.

A ball routed through one of three chutes, showing how Raon-1 mini activates few experts per token

What do the benchmarks say?

These numbers are KRAFTON's, produced with a judge model named in the card. The comparison models are Qwen3.6-35B-A3B and Gemma 4 26B-A4B.

The pattern is lopsided. Search, instruction following and Korean chat are clear wins. General knowledge (MMLU Pro) is about nine points behind Qwen, and terminal and function-calling tasks trail both rivals. The Korean chat score is an internal benchmark, so it deserves the most caution. The card also lists dense models for reference, but those are not like-for-like with a 4B-active MoE.

Compare that with how other small open models have fared in measured tasks, such as Coinbase's fine-tuned Qwen3.5 9B beating larger models on fraud detection. Task-specific results often matter more than leaderboard averages.

How should you read each benchmark?

Not every row in that table predicts the same thing, and the mix of wins and losses is more informative than any single score.

• MMLU Pro measures broad academic knowledge. Sparse models with few active parameters tend to lose here because facts are stored across many experts and a 4.3B-active model reads only a slice per token. The nine-point gap to Qwen3.6 fits that pattern.
• GPQA Diamond and AIME 2026 test graduate-level science questions and competition math. These reward long reasoning chains, which thinking mode supplies. A lead of one to six points over rivals is meaningful only if it survives repeated runs, because both sets are small.
• IFBench checks whether the model follows precise formatting and constraint instructions. An eighteen-point lead over Qwen3.6 is the biggest gap in the table and the most useful for anyone building structured-output pipelines.
• SWE-Verified is the human-checked subset of real GitHub issues. A 70.8 score is strong for this size, though results depend heavily on the agent harness used, which the card does not fully specify.
• BrowseComp Plus measures search-and-browse agents on a fixed document corpus. The 55.7 score against roughly 33 for both rivals suggests the long-context design helps when an agent must keep many retrieved pages in memory.
• Terminal-Bench Lite and BFCL v4 cover shell tasks and function calling. These are where Raon-1 mini trails, so budget extra validation if your agent writes shell commands or relies on strict tool schemas.

The takeaway: it looks like a search, instruction and Korean-language specialist rather than a general knowledge leader. That is a coherent profile for a model positioned at Korean agentic products.

How do you run it?

The card lists several routes:

• Transformers: load with trust_remote_code enabled; thinking mode is on by default and the card advises against turning it off.
• vLLM (beta): an OpenAI-compatible server with tool calling, using KRAFTON's fork. Tested on NVIDIA B300.
• llama.cpp: a GGUF checkpoint, again needing KRAFTON's branch.
• MLX: supported through mlx-lm for Apple silicon.
• Quantizations: MXFP8 and NVFP4. The NVFP4 build is described as running on a single RTX 5090 with 32 GB, with a DGX Spark profile documented as well.

Recommended sampling is temperature 1.0, top_p 0.95 and top_k 20, with a presence penalty of 1.5 for general use and 0 for coding and math.

There is also a draft model for DFlash2 speculative decoding, which the card says gives up to 2.18 times the output throughput of plain decoding on a B300. Speedups vary by hardware and workload. For the technique itself, see our write-up of Eagle 4 and DFlash 2 on Qwen3 models.

Open weights released from a box toward waiting hands, illustrating the Raon-1 mini Apache 2.0 release

What about agent use?

The card includes configuration examples for the Hermes Agent, Pi and OpenCode harnesses through an OpenAI-compatible endpoint. The mixed BFCL v4 and Terminal-Bench Lite results suggest testing tool-calling reliability on your own workload before committing. The strong BrowseComp Plus and SWE-Verified numbers point at search and code repair as the better fit.

What are the caveats?

1. Preview status. The card warns specs may change before the final release. Korean outlets such as Digital Today report that KRAFTON plans a technical report and fuller release; the card itself says little about data mix, token count or training procedure.
2. Vendor-only numbers. No independent evaluation was found at the time of writing. Cost advantages over competing models are not established by the release.
3. Forks required. Full support currently depends on KRAFTON's vLLM and llama.cpp branches, which may lag upstream.
4. Gated access. The repository asks you to request access, even though approval is described as automatic. The hosted API has separate terms.
5. Judge model. Scores use an LLM judge named in the card, which can favor certain styles.

Where does it sit among open models?

Open-weight MoE releases have been arriving quickly. Compare the approach with Aleph Alpha's Kolibri sovereign MoE, the IQuest Q1 agentic coding model, the Atria Dawn 744B agentic model from a Shanghai lab and DeepSeek V4.1 Flash. Raon-1 mini is smaller than most, which is the point: it targets people who want a capable agent model on one workstation GPU, with Korean as a first-class language rather than an afterthought.

What is a fair evaluation plan before you adopt it?

Vendor tables are a starting point. A short plan gives you evidence of your own in an afternoon:

1. Pick three task families you actually ship, for example Korean customer replies, repository bug fixes and multi-document search. Write 20 prompts for each, with a pass or fail rule you can check automatically where possible.
2. Run the same prompts on Raon-1 mini and one rival, using the sampling settings from the model card for Raon and the rival's recommended settings for the other. Keep thinking enabled for both so you compare like with like.
3. Log tool-call validity separately. Count malformed JSON arguments, wrong function names and missing required fields. This is where BFCL v4 suggests risk.
4. Measure memory and latency at your real context length. The 262K window is native, but your throughput at 100K tokens on your own card is what decides cost. Try the DFlash2 draft model and record the speedup you actually see rather than the 2.18 times quoted for a B300.
5. Check license fit. Apache 2.0 allows commercial use, but confirm that the gated-access click-through and the separate hosted terms match your compliance needs if you plan to call KRAFTON's hosted API instead of self-hosting.

If it passes, you gain a permissively licensed model that fits a single 32 GB card. If it fails on tool calls, the preview label tells you to retest when the final version and technical report land.

Should you try it?

Try it if you build Korean-language products, need a permissively licensed model that fits a 32 GB card, or want long-context search agents. Skip it, for now, if your workload is terminal automation or strict function calling, or if you need independent verification before adopting a model. A sensible first test: run your own ten hardest prompts through Raon-1 mini and Qwen3.6 side by side, with thinking enabled for both, and compare tool-call validity as well as answer quality.

Specs and benchmarks are accurate as of October 27, 2026 and come from the vendor's model card for a preview release.

Related reading

• Running MoE LLMs larger than RAM
• Eagle 4 and DFlash 2 speculative decoding
• Aleph Alpha Kolibri open-weight MoE
• IQuest Q1 open-weight agentic model
• Atria Dawn 744B open weights
• DeepSeek V4.1 Flash open weights
• Coinbase's fine-tuned Qwen3.5 9B