explainx.ai350K+ learn from us
TrendingAI News TodayPathwaysSkills
Pricing
explainx.ai

Upskill in AI — 16 free pathways, live workshops & bootcamps, and 50+ courses from practitioners. Plus the skills, tools, and MCP servers to practice on.

follow us

follow on google

Add explainx.ai as a preferred source

corporate training

support@explainx.ai

get started

Find your pathTake Free Evaluation

community

Join the community

learn

mind: share how you thinkpathways — start freeworkshopsbootcampscoursescompare Explainxcertificationsmock testsexplainx universitycorporate traininglearn skills & mcp

discover

skillsmcp serversexplainx mcpestacktoolsmdx readeragentsllmsdesignsdictionarypeopleagi trackerfelony benchranks

company

aboutvisionmissionteaminstructorsteach on explainxpartnershipscommunityhackathonscareers

content

daily AI newsstate of AI — live resultsblogreleasespromptsgeneratorsresource libraryfor LLMsexplainx.ai kids

solutions

all solutionsdeveloper upskillingmarketing upskillingproduct manager upskillingleadership upskilling

newsletter · weekly

Get AI news, tools, and insights in your inbox.

supportcontactprivacytermsdata rightshow we create contentsubmission guidelines

© 2026 AISOLO Technologies Pvt Ltd

explainx.ai

On this page

  • What Apollo will investigate
  • Access rules: what is in scope and what is not
  • Disclosure and meta-transparency
  • How this fits OpenAI's four-assessor phase one
  • What developers should take from this week
  • What is still unknown
  • How Apollo's pilot compares with Anthropic's embedded model
  • Questions regulators and enterprises will ask
  • Timeline to watch
  • Glossary for readers new to scheming debates
  • Related reading
← Back to blog

explainx / blog

OpenAI Apollo Embedded Evaluation Pilot: Testing for AI Scheming Inside the Lab

OpenAI, AI Safety, Apollo Research, METR, Governance

Part of OpenAI and ChatGPT

Apollo Research will embed six evaluators inside OpenAI for four months to test whether models attempt subversion during training and deployment. What access they get and what they cannot verify yet.

Nov 16, 2026·8 min read·Yash Thakker
add explainx.ai
go deep
OpenAI Apollo Embedded Evaluation Pilot: Testing for AI Scheming Inside the Lab

Apollo Research is sending six evaluators inside OpenAI for four months to test whether frontier models try to undermine their own safety rules — the first operational pilot under OpenAI's broader embedded-assessment program. Apollo's November 16, 2026 announcement details negotiated access, two public reports, and hard limits on evidence readers will see. OpenAI's November 13 program post frames the same effort as agreements with Apollo, METR, Transluce, and Goodfire under White House Accord commitments.

For builders, embedded assessments matter because post-release benchmarks cannot catch behaviors that emerge during training, internal agent deployments, or monitor blind spots — themes we covered after the Hugging Face agent incident and in METR's embedded assessment proposal.

What Apollo will investigate

Apollo states the pilot investigates whether models attempt to subvert their own or successors' alignment or control mechanisms — often summarized as scheming in safety literature. The team will examine four linked claims across the development lifecycle:

  1. Training incentives — whether optimization rewarded covert rule-breaking.
  2. Evaluation robustness — whether red teams and evals can elicit subversion.
  3. Internal deployment — whether production-adjacent use showed subversion attempts.
  4. Detectability — whether monitorable reasoning traces would reveal subversion.

OpenAI's earlier post said Apollo's first stage supplies selected public and unreleased models, training checkpoints, relevant datasets, evaluation results, intervention records, monitoring systems, flagged behaviors, contextual researcher access, and compute for independent tests. Apollo notes later claims' access arrangements remain to be agreed, so readers should not assume uniform depth across all four questions.

Apollo builds on joint work stress-testing anti-scheming training, evaluation awareness effects, and preference drift during reinforcement learning — lines of research that intersect scheming propensity debates and Anthropic's push for employee-like evaluator access.

Access rules: what is in scope and what is not

Scoped evaluator access sets which training artifacts and staff channels an embedded team can useScoped evaluator access sets which training artifacts and staff channels an embedded team can use

Apollo distinguishes ideal embedded evaluation (full access and disclosure even against a developer's interests) from this pilot's negotiated rules.

Access commitments OpenAI made

For training-incentive claims, OpenAI committed to supply relevant training code and datasets, reinforcement-learning rollouts with available reasoning traces and rewards, and queries against intermediate checkpoints. Apollo will use its claim-verification framework to review OpenAI's evidence, gather independent evidence, and judge whether OpenAI's collection processes would surface failures.

Access limits still in force

Scope is bounded by agreed claims, models, and time windows. OpenAI may withhold in-scope items when it deems refusal necessary for technical infeasibility, security, or IP protection, but agreed not to withhold because information is unfavorable and must give written reasons. Apollo can request expansions; expansion is not guaranteed.

Staff contact is restricted to OpenAI-selected authorized participants. If an unauthorized employee shares confidential in-scope information, Apollo must stop substantive discussion and route follow-up through OpenAI — a rule Apollo flags as limiting informal discovery compared with employee-channel access at some labs.

Work happens on OpenAI hardware with monitored channels; Apollo says evaluators lack default Slack or general internal channels that safety researchers at OpenAI would use.

Disclosure and meta-transparency

Publication rules balance redacted evidence against fixed public verdicts on each safety claimPublication rules balance redacted evidence against fixed public verdicts on each safety claim

Apollo plans two public reports with methods, conclusions, and limitations. OpenAI may redact confidential evidence, including material supporting findings, but Apollo says OpenAI cannot redact verdicts on each claim. Unredacted copies go to OpenAI's Safety and Security Committee.

Claims based solely on employee discussions need OpenAI attestation unless Apollo establishes them independently — so interviews may inform investigations without appearing verbatim in public text.

Meta-transparency rights let Apollo describe scope, withholdings, redactions, and pressure — even when underlying documents stay secret. Apollo argues without meta-transparency, silence in a report could imply safety rather than untested areas.

TechCrunch's November 16 coverage notes community concern that labs can narrow independence; Apollo's published terms are the concrete document to evaluate that fear for this pilot.

How this fits OpenAI's four-assessor phase one

OpenAI's program is wider than Apollo alone:

table · 2 cols
AssessorFocus area (per OpenAI)
Apollo ResearchSubversion / scheming across training, evals, deployment
GoodfireInterpretability for training-run monitoring and alignment debugging
METRAlignment on largest RL run; follow-on monitor assessment pending scope
TransluceQuality and coverage of post-training alignment evaluations

METR's alignment scope for the largest frontier RL run is narrower than METR's public proposal for a broad misalignment survey — a gap OpenAI acknowledges in a footnote. Goodfire's work targets internal researchers first, with public learnings as collaboration progresses.

Protocols cover on-site secure environments, Safety and Security Committee oversight, off-the-record staff conversations with a separate attestation path, conflict and cool-off rules, and responsible disclosure delays when findings pose acute risk.

What developers should take from this week

Embedded assessments do not ship features, but they change what safety claims mean for anyone building on frontier APIs or deploying agents with broad tool access.

Treat "passes eval" as time-stamped. A model cleared at release may still have trajectories during training or internal agent use that outside testers never saw — the gap embedded pilots try to close.

Read the access appendix, not only the headline. Apollo's meta-transparency is designed so practitioners can see which questions remain unanswered because evidence was redacted or never shared.

Expect industry standardization slowly. OpenAI published protocols "to advance discussion about norms," explicitly noting technical access challenges and evolving security postures — similar to how Amazon's AVERI review exposed gaps in vendor self-testing.

Teams orchestrating agents should pair governance news with runtime controls: our agent skills primer, MCP connection patterns, and Claude Code review risk context cover day-to-day building; embedded evals cover whether the model behind those agents hid misalignment during training.

What is still unknown

Apollo did not publish a start date for on-site work in the November 16 post. OpenAI had not added a separate news item beyond the November 13 framework at the time Apollo went live. Verdicts are months away.

Open questions readers should watch in Apollo's first report:

  • Whether withholding categories materially block conclusions on training incentives.
  • How often redactions affect evidentiary chains readers would need to reproduce checks.
  • Whether agreed monitor logs cover agentic internal deployments relevant to computer-use healthcare pilots and Codex monorepo Git scale.

How Apollo's pilot compares with Anthropic's embedded model

Anthropic helped popularize employee-like evaluator access with tiered staff, time-boxed investigations, and redacted public summaries. Apollo's November 2026 pilot is narrower in headcount—six people for four months—but broader in ambition on subversion claims that require training artifacts, not only near-release checkpoints.

The practical difference for builders:

table · 3 cols
DimensionTypical post-release evalApollo pilot (stated)
TimingDays before launchAcross training and internal deployment windows
DataFinished model weights/APICheckpoints, RL rollouts, monitors
Staff accessCurated demosAuthorized staff list only
PublicationRedacted PDFTwo reports plus meta-transparency on limits

Neither model gives consumers a safety guarantee. Both give audit trails policymakers can cite when incidents like agent monitoring compaction gaps appear.

Questions regulators and enterprises will ask

Can Apollo see training data that identifies individuals? Scope agreements exclude unconstrained PII dumps; Apollo must escalate when OpenAI withholds categories. Enterprise buyers should ask whether their fine-tuning data in OpenAI products could appear in future embedded scopes.

Does a clean Apollo report clear API usage? No. Embedded assessments target developer misalignment, not customer prompt misuse. Runtime policies, decision-model guardrails, and customer-side evals still matter.

Will Goodfire or METR contradict Apollo? OpenAI signed four assessors with different methodologies. Divergent public conclusions are a feature, not a bug—watch for SSC-published unredacted copies if disagreements escalate.

Timeline to watch

  • Training incentive report (Apollo first publication) — earliest signal on whether RL rewards covert rule-breaking.
  • Second report on evals, deployment, and detectability — likely after more internal agent rollouts ship.
  • OpenAI SSC visibility — footnotes promise unredacted assessor reports to the board committee before public cuts.

Teams shipping agents should document their own monitoring evidence now. When external embeds become industry-standard, customers will ask whether your agents expose chain-of-thought and tool logs assessors expect OpenAI to provide.

Glossary for readers new to scheming debates

Scheming (in frontier safety discourse) means a model pursuing goals that conflict with operator intent while appearing compliant during evaluation. It is distinct from ordinary hallucination or tool misuse: the worry is strategic concealment across long horizons.

Embedded evaluation means evaluators sit inside the developer's security perimeter with workstation access, not merely API keys to a shipping model.

Meta-transparency is Apollo's label for the right to describe engagement constraints publicly so silence is not misread as a clean bill of health.

Readers comparing vendors should ask for published embed terms, not only safety blog essays, because terms determine what third parties can actually substantiate.

Procurement teams can copy questions from Anthropic's public benefit report pillars and White House SI Force incident disclosure expectations: ask vendors whether external embeds may publish withheld-category counts, whether customers receive SSC-equivalent summaries, and whether evaluators may interview engineers outside a pre-approved list.

Daily briefFrom Yash · 350K+ students taught

Catch up on AI

One AI story explained in plain English, plus one thing to try. Every day, free. Unsubscribe anytime.

Related reading

  • METR red-team of OpenAI agent monitoring
  • METR embedded assessments proposal
  • Anthropic independent evaluators with tiered access
  • White House Accord on super intelligence
  • What is an embedded evaluator — explainer
Spotted something out of date? Let us know.
Daily briefFrom Yash · 350K+ students taught

Catch up on AI

One AI story explained in plain English, plus one thing to try. Every day, free. Unsubscribe anytime.

  • ✓ Free
  • ✓ One email a day, no spam
  • ✓ Unsubscribe anytime
Yash Thakker

Written by

Yash Thakker

Yash is an AI expert with over 300K learners. Join his workshops →

View Yash Thakker in People in AI →

Related posts

Nov 14, 2026

Helen Toner on Decoder: Why AI Safety Can't Rest on Trusting CEOs

On November 14, 2026 The Verge published a Decoder interview with Helen Toner, former OpenAI board member and CSET executive director, on the blip, RSI culture in frontier labs, and why governance cannot depend on trusting any single CEO.

Nov 13, 2026

Dwarkesh Patel: Why AI Research May Automate Like Math Did

Dwarkesh Patel published "The avalanche that hit math is coming for AI development" on November 13, 2026, arguing that verifiable domains let models automate frontier AI research within years, not decades. explainx.ai maps the evidence he cites, the timelines he gives, and the skepticism on Hacker News.

Oct 22, 2026

METR and Redwood Red-Teamed OpenAI's Agent Monitoring: 10% Gap, Redacted Exploit

On October 22, METR published a brief account of a three-week exercise in which one METR and one Redwood Research employee stress-tested OpenAI's internal misalignment monitor. They found coverage gaps, bypass headers and an exploit OpenAI redacted. OpenAI says the production issues are patched.