A 4B Model Trained With RL Flies a Simulated Tesla Roadster Better Than GPT-6 Astra: What SkyPilot Actually Showed

SkyPilot published a write-up claiming that a 4-billion-parameter model, post-trained with reinforcement learning for about four GPU-hours, flies a simulated flying Tesla Roadster better than GPT-6 Astra. The headline is "21% better on storm kickflips." The details are more interesting than the headline, and more modest: the win is in the authors' own simulator, on two stunts, and the biggest factor is not intelligence but latency. This post walks through what SkyPilot's report says, where its own caveats cut the claim down, and what a builder can take from it.

For context on the car itself, see our earlier coverage of the Roadster event recap and how the cold-gas thrusters work.

TL;DR: the questions people ask first

What was the setup?

The inspiration was the Roadster reveal on October 15, when Elon Musk posted on X a clip of the car flipping over a row of parked Teslas. SkyPilot turned that into a task: a physics simulator of a flying Roadster with four air thrusters and gusty wind, with two stunts taken from the reveal, a ramp flip and a kickflip over a Cybertruck.

An LLM is the pilot. Every 0.2 seconds it sends one of five commands: coast, climb, hold, rotate, or land. The simulator keeps running whether or not the model has answered. Reward runs from 0 to 1, with 0.4 for a successful landing and 0.1 for finishing cleanly. Wind comes in three bands, light, gale and storm, and the storm band is where most models fall apart.

This design matters. It is a control task with a hard real-time budget, not a reasoning puzzle. A model that is brilliant but slow loses, which is exactly the situation many agents and robots face.

How was the 4B model trained?

The recipe has two stages, and the first one is what made the second work.

1. Warm start (about 2 hours 20 minutes): supervised fine-tuning on flights that landed. Those flights came from a hand-written scripted pilot in light wind and from DeepSeek V4.1 Flash in gale and storm. (Our coverage of that model: DeepSeek V4.1 Flash 1017 open weights.)
2. Reinforcement learning (about 1 hour 35 minutes): 60 GRPO steps, 16 tasks per step with 8 flights each, with storm flights making up twice as much of the mix.

SkyPilot reports that pure RL from the untrained model never learned the kickflip. Every attempt in a group crashed the same way, so GRPO had no contrast between good and bad samples to learn from. The warm start gave the policy enough successful behavior that some rollouts in each group landed and others did not, and that variance is the learning signal.

That is a practical lesson that transfers: group-relative methods need within-group variance. If every rollout fails identically, your reward is flat and your RL run is wasted GPU time. For another take on cheap RL pipelines, see our post on multi-turn RL on SageMaker for $339 and the TRL async GRPO measurements.

What are the results?

SkyPilot reports these held-out kickflip success rates, on unseen seeds of the same stunts:

On the ramp flip, the trained 4B lands 100%, 100% and 98%, while open models in real time land 3% or less. GPT-6 Astra also landed 0% in real time.

The "21% better than GPT-6 Astra" figure comes from the storm kickflip: 347 landings against 286. Note which Astra number that is. It is Astra with unlimited time to think, which is a generous setup for the frontier model and not how it would run on the real-time clock.

Is it really about latency, not intelligence?

Largely, yes. SkyPilot reports that most open models take one to six seconds per decision, which is five to thirty control ticks. By the time an answer arrives, the car is somewhere else.

The chart plots median decision time against storm success. Only a scripted pilot and the RL-trained 4B answer inside 0.2 seconds, and only the 4B lands most flights. The fair reading of the table is therefore two separate findings:

• Small and fast beats big and slow when the clock is real. That is a systems result, and it will not surprise anyone who has run agents against deadlines.
• When time is unlimited, the 4B model is roughly on par with the strongest teacher. Its storm lead over DeepSeek with unlimited thinking (90% vs 87%) is within noise by the authors' own test.

This echoes other cases where a small, specialized model wins a measured task: Coinbase's fine-tuned Qwen3.5 9B on fraud, Exowatt's fine-tuned Qwen3 on alarm triage, and a CNN beating frontier models on skin cancer diagnosis. In each, the winner is narrow, trained on the task, and compared against general models not built for it.

What are the caveats SkyPilot itself lists?

To its credit, the post is blunt. In the authors' own words and numbers:

• The simulator, vehicle parameters and reward are the authors' assumptions. One environment, two stunts, no real car.
• "Held-out" means unseen seeds of the same stunts, not new stunts. This tests variance within a task, not generalization to other tasks.
• The storm lead over DeepSeek with unlimited time is not significant (p = 0.07, above the authors' 0.05 threshold). Against Astra the storm kickflip result is significant (p below 0.001).
• The comparison is not fully independent. DeepSeek's storm flights were part of the warm-start data.
• Reruns varied. Validation storm results ranged from 85 to 92 of 96 flights, and the reported run was the best of them.
• The 4B does not beat its teachers in light wind, where they were already near perfect.
• A real vehicle would use a conventional controller, and the scripted pilot decides instantly, which no LLM does.

We add one reading of our own. The Astra comparison used no tuning for the task, no examples, and an environment it had never seen. A fine-tuned small model should win that matchup, which is why this is best read as evidence that RL post-training is cheap and effective for narrow control tasks, not as evidence about relative model quality. For how the frontier model's cost profile compares, see GPT-6.1 Sol and Astra cost efficiency.

What does the $13 mean?

SkyPilot puts the final run at about four GPU-hours, roughly $13 at about $3.39 per H100-hour. That is true for the run that produced the model. It excludes the search for the recipe: the post says many more runs came before it, including the failed pure-RL attempts. A fair budget for someone reproducing this is a small multiple of $13, plus time.

Still, the order of magnitude is the point. A LoRA on a 4B model with 60 RL steps fits on a single node. If you have a latency-bound task, a way to score outcomes automatically, and a teacher (a script or a bigger model) that can produce some successful traces, you can try this over a weekend.

What can a builder take from this?

1. Check whether your task is latency-bound before choosing a model. If decisions have a deadline, a frontier model that cannot answer in time scores zero regardless of quality.
2. Warm-start before RL when success is rare. SFT on successful traces, from a script or a stronger model, creates the variance GRPO needs.
3. Design the reward as a ladder. The 0.4 for landing and 0.1 for a clean finish give partial credit, which helps early training.
4. Oversample the hard regime. Storm flights were doubled in the mix.
5. Report the reruns. The spread (85 to 92 of 96) is the honest picture of what a single run tells you.

The training library, SkyRL, is open source. SkyPilot's write-up also shows its own job orchestration running the trainer, inference server and an evaluation watcher together as one job, which is the infrastructure side of the story.

A green ball follows a curved track toward a small cream cup, illustrating reinforcement learning reward

What does it say about the Roadster?

Nothing about the real car. The Roadster reveal is a marketing and engineering story that we covered separately, and SkyPilot's simulator is a plausible toy built from what was shown, not Tesla's flight stack. Treat the post as a clever use of a viral clip to make a point about RL, not as a statement about flying cars.

Bottom line

A 4B model, a LoRA, a scripted-plus-teacher warm start and an hour and a half of GRPO produced a pilot that handles a hard real-time control task its larger rivals cannot even attempt on the clock. The win over GPT-6 Astra is real in the narrow setup described and should be quoted with that setup attached. If you build agents with deadlines, the transferable lessons are about latency, warm starts and reward design, not about who has the best model.

Related reading

• Tesla Roadster event recap: what was and was not announced
• Tesla's flying Roadster: cold-gas thrusters explained
• Coinbase's fine-tuned Qwen3.5 9B vs Opus 4.5 on fraud
• Exowatt: fine-tuned Qwen3 beats Claude Opus, 49x cheaper
• Multi-turn RL on SageMaker for $339
• GPT-6.1 Sol and Astra cost efficiency

Figures are as reported by SkyPilot on October 30, 2026 and may be revised by the authors. We did not rerun the training.