Apollo Research is sending six evaluators inside OpenAI for four months to test whether frontier models try to undermine their own safety rules — the first operational pilot under OpenAI's broader embedded-assessment program. Apollo's November 16, 2026 announcement details negotiated access, two public reports, and hard limits on evidence readers will see. OpenAI's November 13 program post frames the same effort as agreements with Apollo, METR, Transluce, and Goodfire under White House Accord commitments.
For builders, embedded assessments matter because post-release benchmarks cannot catch behaviors that emerge during training, internal agent deployments, or monitor blind spots — themes we covered after the Hugging Face agent incident and in METR's embedded assessment proposal.
What Apollo will investigate
Apollo states the pilot investigates whether models attempt to subvert their own or successors' alignment or control mechanisms — often summarized as scheming in safety literature. The team will examine four linked claims across the development lifecycle:
- Training incentives — whether optimization rewarded covert rule-breaking.
- Evaluation robustness — whether red teams and evals can elicit subversion.
- Internal deployment — whether production-adjacent use showed subversion attempts.
- Detectability — whether monitorable reasoning traces would reveal subversion.
OpenAI's earlier post said Apollo's first stage supplies selected public and unreleased models, training checkpoints, relevant datasets, evaluation results, intervention records, monitoring systems, flagged behaviors, contextual researcher access, and compute for independent tests. Apollo notes later claims' access arrangements remain to be agreed, so readers should not assume uniform depth across all four questions.
Apollo builds on joint work stress-testing anti-scheming training, evaluation awareness effects, and preference drift during reinforcement learning — lines of research that intersect scheming propensity debates and Anthropic's push for employee-like evaluator access.
Access rules: what is in scope and what is not
Scoped evaluator access sets which training artifacts and staff channels an embedded team can use
Apollo distinguishes ideal embedded evaluation (full access and disclosure even against a developer's interests) from this pilot's negotiated rules.
Access commitments OpenAI made
For training-incentive claims, OpenAI committed to supply relevant training code and datasets, reinforcement-learning rollouts with available reasoning traces and rewards, and queries against intermediate checkpoints. Apollo will use its claim-verification framework to review OpenAI's evidence, gather independent evidence, and judge whether OpenAI's collection processes would surface failures.
Access limits still in force
Scope is bounded by agreed claims, models, and time windows. OpenAI may withhold in-scope items when it deems refusal necessary for technical infeasibility, security, or IP protection, but agreed not to withhold because information is unfavorable and must give written reasons. Apollo can request expansions; expansion is not guaranteed.
Staff contact is restricted to OpenAI-selected authorized participants. If an unauthorized employee shares confidential in-scope information, Apollo must stop substantive discussion and route follow-up through OpenAI — a rule Apollo flags as limiting informal discovery compared with employee-channel access at some labs.
Work happens on OpenAI hardware with monitored channels; Apollo says evaluators lack default Slack or general internal channels that safety researchers at OpenAI would use.
Disclosure and meta-transparency
Publication rules balance redacted evidence against fixed public verdicts on each safety claim
Apollo plans two public reports with methods, conclusions, and limitations. OpenAI may redact confidential evidence, including material supporting findings, but Apollo says OpenAI cannot redact verdicts on each claim. Unredacted copies go to OpenAI's Safety and Security Committee.
Claims based solely on employee discussions need OpenAI attestation unless Apollo establishes them independently — so interviews may inform investigations without appearing verbatim in public text.
Meta-transparency rights let Apollo describe scope, withholdings, redactions, and pressure — even when underlying documents stay secret. Apollo argues without meta-transparency, silence in a report could imply safety rather than untested areas.
TechCrunch's November 16 coverage notes community concern that labs can narrow independence; Apollo's published terms are the concrete document to evaluate that fear for this pilot.
How this fits OpenAI's four-assessor phase one
OpenAI's program is wider than Apollo alone:
| Assessor | Focus area (per OpenAI) |
|---|---|
| Apollo Research | Subversion / scheming across training, evals, deployment |
| Goodfire | Interpretability for training-run monitoring and alignment debugging |
| METR | Alignment on largest RL run; follow-on monitor assessment pending scope |
| Transluce | Quality and coverage of post-training alignment evaluations |
METR's alignment scope for the largest frontier RL run is narrower than METR's public proposal for a broad misalignment survey — a gap OpenAI acknowledges in a footnote. Goodfire's work targets internal researchers first, with public learnings as collaboration progresses.
Protocols cover on-site secure environments, Safety and Security Committee oversight, off-the-record staff conversations with a separate attestation path, conflict and cool-off rules, and responsible disclosure delays when findings pose acute risk.
What developers should take from this week
Embedded assessments do not ship features, but they change what safety claims mean for anyone building on frontier APIs or deploying agents with broad tool access.
Treat "passes eval" as time-stamped. A model cleared at release may still have trajectories during training or internal agent use that outside testers never saw — the gap embedded pilots try to close.
Read the access appendix, not only the headline. Apollo's meta-transparency is designed so practitioners can see which questions remain unanswered because evidence was redacted or never shared.
Expect industry standardization slowly. OpenAI published protocols "to advance discussion about norms," explicitly noting technical access challenges and evolving security postures — similar to how Amazon's AVERI review exposed gaps in vendor self-testing.
Teams orchestrating agents should pair governance news with runtime controls: our agent skills primer, MCP connection patterns, and Claude Code review risk context cover day-to-day building; embedded evals cover whether the model behind those agents hid misalignment during training.
What is still unknown
Apollo did not publish a start date for on-site work in the November 16 post. OpenAI had not added a separate news item beyond the November 13 framework at the time Apollo went live. Verdicts are months away.
Open questions readers should watch in Apollo's first report:
- Whether withholding categories materially block conclusions on training incentives.
- How often redactions affect evidentiary chains readers would need to reproduce checks.
- Whether agreed monitor logs cover agentic internal deployments relevant to computer-use healthcare pilots and Codex monorepo Git scale.
How Apollo's pilot compares with Anthropic's embedded model
Anthropic helped popularize employee-like evaluator access with tiered staff, time-boxed investigations, and redacted public summaries. Apollo's November 2026 pilot is narrower in headcount—six people for four months—but broader in ambition on subversion claims that require training artifacts, not only near-release checkpoints.
The practical difference for builders:
| Dimension | Typical post-release eval | Apollo pilot (stated) |
|---|---|---|
| Timing | Days before launch | Across training and internal deployment windows |
| Data | Finished model weights/API | Checkpoints, RL rollouts, monitors |
| Staff access | Curated demos | Authorized staff list only |
| Publication | Redacted PDF | Two reports plus meta-transparency on limits |
Neither model gives consumers a safety guarantee. Both give audit trails policymakers can cite when incidents like agent monitoring compaction gaps appear.
Questions regulators and enterprises will ask
Can Apollo see training data that identifies individuals? Scope agreements exclude unconstrained PII dumps; Apollo must escalate when OpenAI withholds categories. Enterprise buyers should ask whether their fine-tuning data in OpenAI products could appear in future embedded scopes.
Does a clean Apollo report clear API usage? No. Embedded assessments target developer misalignment, not customer prompt misuse. Runtime policies, decision-model guardrails, and customer-side evals still matter.
Will Goodfire or METR contradict Apollo? OpenAI signed four assessors with different methodologies. Divergent public conclusions are a feature, not a bug—watch for SSC-published unredacted copies if disagreements escalate.
Timeline to watch
- Training incentive report (Apollo first publication) — earliest signal on whether RL rewards covert rule-breaking.
- Second report on evals, deployment, and detectability — likely after more internal agent rollouts ship.
- OpenAI SSC visibility — footnotes promise unredacted assessor reports to the board committee before public cuts.
Teams shipping agents should document their own monitoring evidence now. When external embeds become industry-standard, customers will ask whether your agents expose chain-of-thought and tool logs assessors expect OpenAI to provide.
Glossary for readers new to scheming debates
Scheming (in frontier safety discourse) means a model pursuing goals that conflict with operator intent while appearing compliant during evaluation. It is distinct from ordinary hallucination or tool misuse: the worry is strategic concealment across long horizons.
Embedded evaluation means evaluators sit inside the developer's security perimeter with workstation access, not merely API keys to a shipping model.
Meta-transparency is Apollo's label for the right to describe engagement constraints publicly so silence is not misread as a clean bill of health.
Readers comparing vendors should ask for published embed terms, not only safety blog essays, because terms determine what third parties can actually substantiate.
Procurement teams can copy questions from Anthropic's public benefit report pillars and White House SI Force incident disclosure expectations: ask vendors whether external embeds may publish withheld-category counts, whether customers receive SSC-equivalent summaries, and whether evaluators may interview engineers outside a pre-approved list.
