“Aligned” often appears in blog titles and keynotes, but the underlying question is old: when you optimize something powerful toward a target, do you get what you meant, or what the target rewards? For frontier language models and agents, that question touches laboratory training, product deployment, and governance—not only a far future AGI milestone.
This article is a map, not a manifesto. It pairs well with scalable oversight and feedback (how we steer systems), specification gaming and Goodhart (how metrics misfire), and interpretability and monitoring (what to watch in practice).
1. Three layers people confuse
- Intent (normative). What should the system do in messy social contexts? This is not fully captured by a loss function. Law, company values, and user harm tradeoffs live here.
- Specification (design). The reward, rubric, constitution, or human-feedback protocol you actually implement. Easy to be crisp on paper, fuzzy at the edge.
- Behavior (empirical). What the system does in logs, red-team tests, and production—including confident mistakes.
Alignment work—industry and academic—tries to close gaps between these layers, especially as capability and autonomy make gaps costlier. Labs publish frameworks for scaling responsibility alongside capability; Anthropic’s Responsible Scaling Policy and its updates (RSP) are examples of institutional alignment: if certain capability thresholds are met, then stronger safeguards apply. That pattern—conditional commitments tied to evals—mirrors how good product orgs treat model releases and risk review.
2. Outer and inner (the short version)
Outer alignment is often described as: did we pick the right objective? (Goodhart, missing externalities, and badly written success metrics live here; see the next post in this series.)
Inner alignment (informally) asks whether the learned policy “wants” what you thought you trained for—e.g. whether under distribution shift or long-horizon power-seeking, it takes shortcuts you would not endorse. This is a research frontier; product teams should not pretend it is “solved.” They can require adversarial testing, monitoring, and constrained tool access when stakes rise.
For day-to-day work, a practical merge is: (a) audit your metrics and task definitions, (b) treat model output as policy subject to review when the impact is high, (c) don’t conflate fluency with safety.
3. What your team can do this quarter
- Write down success measures for each agent: not only latency and token cost, but wrong-action rate, escalation rate, and human override when uncertain.
- Separate “helpful in chat” from “safe in production”—especially for tools and MCP integrations.
- Version prompts, skills, and tool schemas like code; “alignment” in practice is change control.
- Red-team the workflow (not only the model): can an operator game the dashboard or reward the agent for the wrong thing?
Read next: Scalable oversight (RLAIF, constitutions) · Specification gaming · Interpretability and monitoring · Our AGI explainer page
We are an educational and directory product; this is not legal advice or a safety certification. Follow primary sources at labs and regulators for your jurisdiction.