Base Labs Mammoth-1 and Elephant-1: Open Decision Models Claim #1 (Claimed vs Verified)

Base Labs, the research arm at Baseten, has released two open-weight decision models: mammoth-1, a 27B model, and elephant-1, a 9B sibling. The mammoth-1 card claims first place among 208 models on the Jev Decision Index 0.4 and on a replica of the Artificial Analysis Decision Index. The weights are on Hugging Face under the MIT license, with a technical report and a free hosted trial through October 27.

This post explains what a decision model does, what the card reports, which parts of the "#1" claim you can check, and how to run it. If decision models are new to you, start with our explainer on what decision models are and the ranked top 10 of open and closed decision models.

TL;DR: what is claimed and what is checkable

What is a decision model?

A decision model takes a piece of text, called the state, plus a set of questions, and returns a calibrated probability for each answer option. The mammoth-1 card says it answers all questions from "one shared-prefix prefill," which means the expensive reading of the text happens once, however many questions you ask. Question types in the example code:

• noul: yes or no, returning P(yes), for example "Is the customer asking for a refund?"
• choice: one of named options with descriptions, for example which team (billing, technical, fraud) should handle a ticket, returning the pick and probabilities.
• score: ordered levels from lowest to highest, such as how urgent a ticket is.

That makes it a classifier with calibrated confidence, built for routing, guardrails, moderation and workflow triage, not for open-ended text. Calibration matters because a downstream system can act on thresholds: auto-resolve above 0.95, escalate to a human between 0.6 and 0.95. The card says training combined cross-entropy with a Brier calibration term and a trained "I don't know" option.

How it was trained

Details from the card:

• Data: 6.21 million decision examples, of which 58.3% are core decision skills, 16.1% safety and abstention, 16.1% judging and expert domains and 9.5% public decision data.
• Recipe: one epoch (9,859 steps), peak learning rate 2.7e-6, AdamW, 3% warm-up with cosine decay, bf16.
• Hardware: 18 nodes with 4 GB300 GPUs each, 72 GPUs total.
• Architecture changes: full fine-tune, text only with the vision encoder frozen, and a block-bidirectional attention mask on full-attention layers. Max sequence length is 16,384 tokens.

A launch summary relayed from the company's posts says the team used a "swiss cheese" approach to validate and scale synthetic data, crediting about 20 index points to scaling data 16x and 8 points to cleaning it, and says the training shows an agentic LLM iterating on a simpler model can produce recursive improvement, with human oversight still required. These are the developers' own statements on X, and we have not reviewed the technical report's ablations.

The benchmark claim, examined

The Jev Decision Index table on the card:

And the second table, labeled "our replica" of the Artificial Analysis Decision Index: mammoth-1 65.9, elephant-1 65.2, Liquid's d1-ultra 65.

Points worth keeping in mind:

1. Who runs the benchmark. The Jev index is run by Jev, a competitor in this category, and the card reports results "on the public split." Base Labs ran its own evaluation on a replica of the Artificial Analysis index. Neither is an independent head-to-head run by a neutral third party.
2. The margin is small on the second table. 65.9 against 65 is under a point, and elephant-1 at 9B scores 65.2 on the same replica, which says the index may be near saturation or that the smaller model is unusually strong.
3. Broad area wins. The card says mammoth-1 leads in four of five areas on the public split: knowledge and reasoning, language, tools and automation, retrieval and classification, and arts and human taste. It does not name the one it loses.
4. Compared models. Perplexity Decider and Cloudflare Clef are covered in our posts on Perplexity's Decisions API and Cloudflare Clef Omni. For the other leaderboard claimant, see Liquid d1-ultra, claimed vs verified, which discusses the same Artificial Analysis index.

Bottom line: the numbers are plausible and the tables are internally consistent, but "the top decision model globally" is a vendor-and-index-owner claim. For your own use, test on your own labeled data.

How to run it

Hosted: mammoth-1 is free for one week on Baseten Model APIs with zero data retention on by default, ending October 27, 2026 at 11:59 PM Pacific.

Self-hosted: the card says the setup script installs a pinned serving runtime (OneJev plus a patch). You need one CUDA GPU and the pytorch/pytorch:2.8.0-cuda12.8-cudnn9-devel image. The flow is to download the repository with the Hugging Face CLI and run setup.sh, then call the DecisionEngine class with a state and a dictionary of questions. The response contains the answers: P(yes) for yes or no questions and probabilities per option for choices and scores. Requests longer than the trained limit are refused, not truncated, so check your input lengths.

A worked example from the model card

The card's own example shows the intended workflow. The state is a single customer message: "I was charged twice this month, please refund one of them." Three questions are asked at once. A yes or no question asks whether the customer is requesting a refund. A choice question asks which team should handle it, with criteria for billing (charges, refunds, invoices), technical (app or site faults) and fraud (suspected unauthorised use). A score question rates urgency on three ordered levels: can wait, today, and blocking the customer now.

The point of the design is that you pay for reading the message once and get three calibrated answers back. In a production router you would act on the numbers, for instance sending anything with a fraud probability above a threshold to a human, and logging low-confidence decisions for review. The "I don't know" option trained into the model gives you a built-in abstention signal, which is more useful for safety than a forced pick.

How to evaluate it on your own data

Because the index scores are self-reported, the most valuable thing you can do is a small local bake-off.

1. Collect 200 to 500 labeled examples from your own traffic, including hard and ambiguous cases.
2. Run mammoth-1, elephant-1 and your current classifier (or a prompted general LLM) on the same set.
3. Measure accuracy and calibration, for example expected calibration error and the Brier score, because probabilities are the product here.
4. Check abstention behavior: how often does each model say it does not know, and are those the cases humans also found hard?
5. Measure latency and cost per thousand decisions on your hardware or on Baseten, once the free week ends.

If the 9B model matches the 27B on your data, as the replica table hints it might, the cheaper model is the sensible default.

A noisy signal separating into a clean line, illustrating how a calibrated decision model turns messy text into probabilities

Where this fits in your stack

The 9B elephant-1 is the cheaper option when latency matters, and the card describes it as the fastest. For a broader view of the category's pricing and speed, see our Subq SNAP long-context decision model and IBM Granite Switch posts.

Questions to ask before adopting

• Does calibration hold on your data? A model can top an index and still be overconfident on your domain. Plot reliability curves on a labeled sample.
• Is the free week a pricing signal? After October 27 you will pay Baseten or self-host. Compare cost per thousand decisions with a smaller model.
• License and base. MIT for the weights, derived from Qwen3.5-27B (Apache 2.0). Check any regional policy about model origin.
• Safety use. If you use it as a guardrail, remember a classifier is one layer; pair it with action-level controls such as AgentBeam, the agent security platform from the explainx.ai team, for agents that take real actions.

Related reading

• What are decision models?
• Top 10 decision models ranked
• Liquid d1-ultra claimed vs verified
• Perplexity Decisions API
• Cloudflare Clef Omni
• Clairic vs Jev vs OpenAI Decisions comparison
• Official: mammoth-1 on Hugging Face, elephant-1 and the Base Labs technical article

Details are accurate as of October 22, 2026 and come from the Hugging Face model card; benchmark figures are self-reported.