FineDNA and Carbon-X: Hugging Face Open-Sources a DNA Dataset and a 14B Model

Hugging Face Bio has open-sourced the data used to teach a large language model to read DNA. FineDNA, published on the Hub in the last days of October 2026, holds the full training data for all three stages of Carbon-X: mid-training, supervised fine-tuning and reinforcement learning. The model is a 14.8B-parameter chat model built on Qwen3-14B that takes raw DNA in a conversation and answers in English.

On the model card's own five-benchmark table, Carbon-X averages 66.7% against 67.5% for GPT-6.1 Sol, at what the developers estimate is about 1/1,800 of the inference cost per problem. Those are the developers' numbers, not independent ones, and we flag which is which below. We covered the model itself on October 22 in Carbon-X, the 14B open DNA chat model; what is new now is the data behind it. If you saw the earlier Carbon-A gene-finding release, this is the next model in the same family.

TL;DR: FineDNA and Carbon-X at a glance

What is inside FineDNA?

The dataset card organizes FineDNA by training stage. Each stage is a group of configs you can load separately.

Mid-training: DNA, biology text, English and instructions

The 23 mid_* configs hold the sources of the 302B-token mid-training run. The mix, per the card:

The DNA comes from the GENERator and OpenGenome2 datasets after filtering, split into groups such as vertebrate, plant, fungi, prokaryote and mRNA. Biology text covers web pages, papers and forum posts that contain DNA, plus synthetic annotations; English text from FineWeb-Edu, DCLM and FinePDFs-Edu keeps the model's English; instruction data from Nvidia's Nemotron chat sets preserves instruction-following. The card says tokenizing the text column with the Carbon-X tokenizer and using the listed weights reproduces the same data diet. Some biology sources were read more than seven times, while some DNA sources were read less than once, which is how the weights work as a recipe.

SFT: eight task domains

The 203,923 supervised conversations span eight domains: biology Q&A, chat, DNA computation (counting, GC content, melting temperature, translation, restriction sites), instruction following, sequence analysis, sequence classification, sequence continuation and variant effect prediction. The two largest are variant effect prediction (57,254 rows) and sequence continuation (58,727 rows). The card says DNA-computation answers are computed exactly from the sequence, so they are grounded rather than model-written.

RL: prompts a program can check

The 98,433 RL rows are verifiable prompts: each has a gold answer or constraint set that a reward function checks in code. Task groups include short-sequence computations, localization of bases and variants, pathogenic-versus-benign calls, multiple-choice functional genomics, biology MCQs and formatting constraints. The card says the blend was calibrated on rollouts from the SFT model and excludes prompts seen in SFT; it is meant for one epoch.

This matches the pattern behind other recent RL-on-verifiable-rewards work: choose tasks where a script can say right or wrong, so the model cannot just sound confident. That is the dataset-level answer to hallucination. The training recipes for each stage are in the huggingface/carbon repository.

How Carbon-X works

According to the model card, Carbon-X uses a standard decoder with a hybrid tokenizer. Text uses the Qwen3 byte-pair vocabulary. DNA is encoded as non-overlapping 6-mers, so one DNA token covers six base pairs, and there is no separate DNA encoder. Runs of at least six A, C, G or T bases are tagged and tokenized automatically. The context window is 40,960 tokens; the model was post-trained and evaluated with thinking disabled.

A basic call looks like this (from the card; it needs Transformers v5 or later and trust_remote_code=True for the tokenizer):

from datasets import load_dataset

ds = load_dataset("HuggingFaceBio/FineDNA", "sft_mix", split="train")
rl = load_dataset("HuggingFaceBio/FineDNA", "rl_mix", split="train")
mid = load_dataset("HuggingFaceBio/FineDNA", "mid_dna_mrna", split="train", streaming=True)

For inference, the card shows serving with vllm serve HuggingFaceBio/Carbon-X --trust-remote-code and using an OpenAI-compatible client.

How does Carbon-X compare with frontier models?

These are results from the Carbon-X model card. Carbon-X is run with thinking off; the comparison models are each shown at the reasoning effort with their highest average. BioReason and GenomeQA come from the literature; the authors built FrontierVEP and NextBase themselves.

What it shows: a small, specialized, open model is competitive with the best closed model on average, wins on coding variant effects and next-base prediction, and loses on complex variants and single-base variant effects. For general-model context, see our coverage of GPT-6.1 Sol's launch and the Claude Opus 5.5 launch.

The cost claim

The card says Carbon-X costs about 1/1,800 as much per problem as GPT-6.1 Sol, and estimates that analysing the 2.4 million ClinVar variants of uncertain significance would cost about $300 with Carbon-X versus $32,000 to $4 million with the frontier models evaluated. The Carbon-X figure assumes measured throughput per H200-hour at $5 per hour; the frontier figures use standard API pricing. That is a vendor-side estimate: it is plausible directionally, since a 14B model answering without a long reasoning chain is cheap, but treat the exact ratio as unverified until others reproduce it.

What the model cannot do

The card is direct about limits, and this is the section to read before building anything:

• Not clinically validated. It should not be used for medical decisions.
• Genetic data is sensitive. How inputs are handled depends on whether you run it locally, via an API, or in a hosted Space.
• Exact operations. It can miscount bases, confuse coordinates or make incorrect edits, especially in long sequences; pair it with tools.
• Hallucination. It may invent a sequence's identity or function without enough context.
• Uneven coverage. Splice-site and histone-mark questions are weak at 29.8% and 34.6% on GenomeQA.
• Reads, does not design. Quality degrades for long generated sequences; it was not trained to design new ones.
• Long inputs. Mid-training used 8,192-token sequences, up to roughly 49 kbp of DNA; longer inputs are unevaluated.

What this means for builders

1. Open recipes are the news. Few DNA-model releases ship all three stages of data. You can reproduce or adapt the pipeline rather than just download weights.
2. Verifiable rewards generalize. The RL design (answers checked in code) is a template for other domains with exact answers.
3. Specialist beats generalist on cost. For bulk analysis, a small local model can be far cheaper than frontier API calls, if its accuracy suffices.
4. Keep sensitive sequences local. The single-GPU footprint makes on-premises use feasible.
5. Do not use it clinically. Treat outputs as research hints and verify.

For other AI-for-biology releases we have covered, see the Carbon-A candidate-gene release, the Google Antigravity AlphaGenome Atlas skill, OpenAI's GeneBench Pro, and Hugging Face's earlier large-data release, The Stack v3.

What people are asking

Is the 300B-token figure the dataset size? Not exactly. The card lists 514.30B unique mid-training tokens available, and says Carbon-X read 302B of them. The "300B" headline refers to the run, so if you want to reproduce it, plan for about 950 GB of mid-training data and stream it rather than downloading everything.

Why start from Qwen3-14B instead of training from scratch? The card says mid-training starts from an instruct model, which is why it mixes in English text and Nemotron instruction data: to keep the model able to chat and follow formats while it learns DNA. Starting from an existing base saves the cost of general language pretraining. The same pattern shows in how other labs adapt open bases; see our Qwen3.8 27B on Cerebras coverage for the base-model family's speed side.

Can I fine-tune on my own sequences? The SFT and RL configs are laid out so each domain can be loaded alone, for example sft_variant_effect_prediction or sft_sequence_classification. That makes it practical to retrain only the skills you care about, though you should check licenses of third-party mid-training sources first, since the card says they keep their own terms.

What should I test first? Start with the exact-computation tasks, because the card admits the model can miscount bases. Compare its answers to a script on your own data before trusting it for anything else.

What is verified versus claimed

• Verified from the primary cards: dataset structure, sizes, license statement, the benchmark table, the stated limitations.
• Developer claims, not independently reproduced: the benchmark scores, the "strongest open model for DNA understanding" framing, and the cost ratios. Two benchmarks were built by the same authors.
• Early signal: at our check the dataset showed 11 likes and 72 downloads, so there is no community track record yet.

Details reflect the Hugging Face dataset and model cards as of November 1, 2026. Numbers and files can change.

Related reading

• Carbon-X: 14B open DNA chat model, claimed vs verified
• Hugging Face Carbon-A: 566 million candidate genes
• Google Antigravity AlphaGenome Atlas skill
• OpenAI GeneBench Pro and computational biology
• Hugging Face The Stack v3
• GPT-6.1 Sol launch, pricing and benchmarks
• Claude Opus 5.5 launch
• FineDNA dataset card and Carbon-X model card