RadixArk Miles v0.1: Production RL Stack for Frontier LLM Post-Training

On August 18, 2026, the same day OpenAI paused its largest frontier RL run over Astra's preliminary cyber-capability rating, the RadixArk team published Miles v0.1 — a production-grade reinforcement-learning stack for frontier LLM post-training. Announced on the LMSYS blog and documented in a companion PyTorch.org write-up, Miles is a fork of THUDM's slime that wires SGLang (rollout), Megatron-LM (training), and Ray (orchestration) into one loop aimed at trillion-parameter MoE models, agentic coding environments, and the low-precision recipes Blackwell-class hardware expects.

If you have been tracking the post-training wave — Intology's Locus beating human-tuned Qwen3 releases, Castform's narrow RL retrieval wins, or the agent-harness RL direction in loop engineering — Miles is the infrastructure layer those stories assume someone will maintain: not a single benchmark claim, but the machinery to run RL at the scale open-weight frontier models now occupy.

TL;DR — what Miles v0.1 actually ships

The Miles RL loop in plain terms

RadixArk describes Miles as optimizing every stage around one principle: verified, clean, and customizable everywhere. In practice that means three repeating stages:

1. Rollout — SGLang engines generate trajectories. For agentic RL, each multi-turn session gets an isolated sandbox (terminal, coding, tool-use) that executes actions and returns a verifier reward.
2. Training — Completed trajectory groups feed Megatron-LM (default) or PyTorch FSDP2, which computes the RL loss and updates weights.
3. Weight update — New weights sync back to the rollout fleet with minimal interruption to in-flight generations.

Ray sits above all three, handling actor lifecycle, scheduling, and fault tolerance across nodes. PyTorch stays the common numerical layer — losses are ordinary autograd graphs, which matters because post-training research moves fast and teams do not want a new abstraction every time they swap a reward or add a router hook.

That architecture parallels what production multi-agent orchestration systems do at inference time — persistent workers, isolated environments, async completion — except Miles applies it to policy improvement rather than one-off task execution.

Why MoE RL needed R3

Mixture-of-experts models broke a quiet assumption in dense-model RL: rollout and training engines could be "close enough." With MoE, a single top-k routing flip changes both which experts fire and which expert weights receive gradients. Tiny numerical drift between SGLang inference and Megatron-LM training therefore shows up as policy divergence, not harmless noise.

Rollout Routing Replay (R3) fixes that by recording SGLang's expert routing during rollout and replaying the same routes during training. RadixArk reports the replay path adds only minimal overhead compared with normal routing — important because MoE RL already pays for long agentic trajectories and straggler-heavy batching.

Miles also ships Token-In-Token-Out (TITO) for multi-turn agentic sessions: the session server preserves exact token IDs the model generated, so chat-template re-serialization between turns cannot silently change what the trainer optimizes. That token-level fidelity is a prerequisite for R3, on-policy distillation, and zero-KL alignment recipes.

Fully async RL and agentic environments

For terminal-use and coding agents, rollout time is dominated by a few long trajectories — sandboxes running tests, tool calls waiting on I/O, compactions in opaque harnesses. Synchronous RL schedules make both sides wait: the trainer idles on stragglers; rollout engines idle during optimizer steps.

Miles v0.1's fully async RL keeps generation continuously in flight while the trainer consumes finished groups. Scheduling is sample-granular: each completed trajectory frees a slot immediately. A bounded buffer decouples rollout throughput from training cadence and gives users a policy boundary for stale-sample rejection without forking the scheduler.

On environments, Miles plugs into Harbor, HUD, NeMo Gym, OpenEnv, Prime Intellect Verifiers, or custom sandboxes backed by AgentENV, Daytona, E2B, or Modal. v0.1 ships maintained end-to-end recipes for agentic coding and terminal tasks — the same workload class as the GLM-5.2 reference run.

Low precision, big models, faster rollouts

Three precision features matter for teams that cannot rent unlimited H100 walls:

Unified FP8 pipeline. Miles supports FP8, MXFP8, and NVFP4 end-to-end — rollout through gradient GEMMs — with bit-exact quantizer contracts so SGLang and Megatron-LM see the same quantized values. RadixArk reports reward curves tracking BF16 baselines while cutting rollout time.

INT4 QAT for 1TB+ models. Quantization-aware training lets very large checkpoints participate in RL on tighter GPU budgets — the same memory pressure RAM and HBM shortages are pushing local builders toward cloud inference for.

Speculative RL with online SFT draft model. Miles trains a small draft model online via supervised fine-tuning and uses it to speculate during rollout. RadixArk claims more than 25% rollout speedup from this path — speculative decoding logic applied to the RL inner loop rather than single-request inference.

Day-0 open-weight frontier coverage

A recurring pain point in open-weight RL is the gap between "weights dropped on Hugging Face" and "we can actually RL this tonight." Miles v0.1 claims day-0 trainability for:

RadixArk brings SGLang inference paths and Miles RL recipes up in parallel rather than sequentially — the difference between "interesting paper" and "we fine-tuned it this week."

Reference run: GLM-5.2 744B-A40B on 64 GB300 GPUs

The blog's concrete proof point is asynchronous RL on GLM-5.2 744B-A40B for terminal-use coding:

• Hardware: 64 NVIDIA GB300 GPUs — 32 for SGLang rollout, 32 for Megatron-LM training
• Parallelism: Training TP 2 / PP 4 / CP 4 / EP 8; inference TP 8 / EP 8 with multi-token prediction (MTP)
• Workload: Multi-turn terminal agents in per-task sandboxes via OpenEnv; 65k max sequence length; batch size 64
• Results: 100 stable rollout steps; ~4.5-minute training steps; rollout weights lagging ~1.7 steps behind trainer; ~96% prefix-cache hit rate

Memory optimizations — NVMe optimizer-state streaming, CPU/NVMe offload, colocated rollout/trainer footprint reductions — are what make 744B async RL fit on 32 training GPUs instead of a much larger slab. That is the production story Miles is selling: not a toy GRPO script, but the plumbing frontier open-weight labs need when RL becomes the default post-release step.

OpenAI paused; RadixArk shipped — what builders should take away

These events landed hours apart but answer different questions:

If you are using frontier APIs, OpenAI's pause is the signal to audit sandbox isolation and credential scope in your agent loops. If you are training open models, Miles is a check whether your stack can keep pace with MoE scale, async agentic rollouts, and FP8 — or whether you are still wedging research-grade scripts into production clusters.

What people are asking

Is Miles ready for production or still researchware?

v0.1 is explicitly positioned as production-level — CI-guarded recipes, fault-tolerant Ray orchestration, three eval modes that do not stall training, AMD ROCm parity alongside NVIDIA. That said, Miles' own roadmap still lists open items like zero mismatch for MoE RL beyond R3, MoE SGLang/Megatron alignment, and elastic rollout/training scheduling. Treat v0.1 as the first broadly usable release, not a finished surface.

Do I need Megatron-LM, or can I start with FSDP?

Megatron-LM is the default and where the largest recipes live (expert/pipeline/context parallelism, NVMe optimizer streaming). FSDP2 loads Hugging Face weights directly — no conversion step — but is data-parallel only. Reasonable for mid-size dense models; insufficient for the 744B reference run.

How does this relate to slime?

Miles forked slime for its clean modular boundaries (rollout functions, losses, rewards behind typed interfaces) and extended it for frontier scale: async scheduling, R3, TITO, P2P/RDMA weight sync, disk-delta updates, and maintained model recipes. If you already run slime experiments, Miles is the "same skeleton, production organs" path.

Who is RadixArk?

RadixArk is the team behind Miles and a major contributor to the SGLang ecosystem. They sit in the same LMSYS-adjacent open infrastructure lane as the teams shipping inference engines and chat arenas — not a consumer product company, but the kind of group that determines whether open models get RL'd or merely downloaded.

Summary

Miles v0.1 is RadixArk's answer to a specific 2026 bottleneck: open-weight frontier models arrive weekly, but production RL loops — async agentic rollouts, MoE-stable training, FP8, trillion-parameter memory planning — did not. By forking slime and composing SGLang, Megatron-LM, and Ray, Miles ships R3 for MoE routing fidelity, speculative RL with online SFT drafts for faster rollouts, INT4 QAT for massive checkpoints, and day-0 recipes for Kimi K3, DeepSeek-V4, Inkling, Qwen3.8, and Nemotron 3 Ultra. The GLM-5.2 744B terminal-use run on 64 GB300 GPUs is the reference proof. It dropped the same day OpenAI gated closed frontier RL — a useful split-screen for anyone deciding whether the action in post-training is safety pauses upstream or open infrastructure downstream.

Related on explainx.ai

• OpenAI pauses frontier RL over Astra cyber-critical risk · same-day contrast
• Intology Locus: automated post-training beats Qwen3
• Castform + Neon: RL post-training on retrieval
• Kimi K3 open weights — 2.8T MoE day-0 context
• Multi-agent orchestration patterns — production guide
• Loop engineering for coding agent harnesses
• RAM prices and local inference economics

Official sources: LMSYS — "Miles v0.1: Production-level Post-training" (August 18, 2026) · PyTorch blog — "Miles: A PyTorch-native stack for large-scale LLM RL post-training" · RadixArk docs at miles.radixark.com · GitHub: radixark/miles

Framework features, model lists, and benchmark numbers reflect RadixArk's August 18, 2026 announcements. Hardware availability, driver stacks, and day-0 model coverage will vary by cluster — validate against your target checkpoint and GPU generation before committing production spend.