explainx.ai350K+ learn from us
TrendingAI News TodayPathwaysSkills
Pricing
explainx.ai

Upskill in AI — 16 free pathways, live workshops & bootcamps, and 50+ courses from practitioners. Plus the skills, tools, and MCP servers to practice on.

follow us

follow on google

Add explainx.ai as a preferred source

corporate training

support@explainx.ai

get started

Find your pathTake Free Evaluation

community

Join the community

learn

mind: share how you thinkpathways — start freeworkshopsbootcampscoursescompare Explainxcertificationsmock testsexplainx universitycorporate traininglearn skills & mcp

discover

skillsmcp serversexplainx mcpestacktoolsmdx readeragentsllmsdesignsdictionarypeopleagi trackerfelony benchranks

company

aboutvisionmissionteaminstructorsteach on explainxpartnershipscommunityhackathonscareers

content

daily AI newsstate of AI — live resultsblogreleasespromptsgeneratorsresource libraryfor LLMsexplainx.ai kids

solutions

all solutionsdeveloper upskillingmarketing upskillingproduct manager upskillingleadership upskilling

newsletter · weekly

Get AI news, tools, and insights in your inbox.

supportcontactprivacytermsdata rightshow we create contentsubmission guidelines

© 2026 AISOLO Technologies Pvt Ltd

explainx.ai

On this page

  • TL;DR: the review at a glance
  • What did AVERI actually review?
  • What did the review confirm?
  • What weaknesses did it surface?
  • The four changes Amazon says it made
  • What this does and does not show
  • Why meta-evaluation matters for people building with AI
  • How to audit your own eval set, step by step
  • Context: why independent checks on evaluations are becoming a theme
  • What to watch next
  • Related reading
← Back to blog

explainx / blog

Amazon Had Its AI Safety Tests Reviewed by AVERI: What Was Found and the 4 Changes

AI Safety, Amazon, Evaluations, Amazon Nova, AVERI

Part of AI Safety and Alignment

Amazon let AVERI audit its internal AI safety test prompts, not its models. What the review confirmed, what it flagged, and the four fixes Amazon made.

Oct 26, 2026·8 min read·Yash Thakker
add explainx.ai
go deep
Amazon Had Its AI Safety Tests Reviewed by AVERI: What Was Found and the 4 Changes

Amazon asked an outside group to grade its homework, then published the grade. In a post on Amazon Science dated October 20, 2026, the company said the nonprofit AVERI (AI Verification and Evaluation Research Institute) reviewed its internal safety-evaluation prompts for Amazon's models. The review confirmed strengths, flagged gaps, and Amazon says it made four changes. Aggregators framed it this week as Amazon "updating AI safety testing after an independent review identified weaknesses."

The unusual part is the target. AVERI did not test Amazon's models. It examined the tests themselves, asking whether they actually stress safety boundaries. That is called meta-evaluation, and it is rare. Below: what was reviewed, what was found, what changed, and what the post cannot tell you.

TL;DR: the review at a glance

table · 2 cols
QuestionAnswer
Who reviewed?AVERI, an independent nonprofit; led by executive director Miles Brundage
What was reviewed?A random sample of prompts from Amazon's internal safety-evaluation suite
Was Amazon's eval team involved?Not in the assessment, per Amazon
Confirmed strengthsTopical coverage, diversity of testing approaches, low contamination risk
Gaps flaggedUneven multilingual coverage, too few multistep adversarial prompts, stale references, superficial refusal cues
Amazon's fixesTest-set refresh, difficulty tiers, quarterly reviews, broader multilingual prompts
Independent confirmation of the fixes?None found; the account is Amazon's own

Agents that act on your behalf need rules. See how explainx helps secure your agents →

What did AVERI actually review?

Per Amazon's post, the company provided "a random sample of prompts drawn from our internal safety-evaluation suite," spanning categories such as harmful-content generation, cybersecurity risks, privacy protection and copyright compliance. AVERI's team, led by Miles Brundage and including Sean McGregor, A. Feder Cooper and George Balston, examined the prompts for diversity, difficulty calibration and methodological soundness.

Amazon says the assessment was done "without input from our evaluation team," so the findings would reflect an external view. Note what that does not include: the full test suite, grading outputs, or the models' actual results. A random sample can reveal patterns, but it cannot certify a whole suite.

Amazon already publishes frontier safety material for its Nova models; see its papers on Nova Premier under the Frontier Model Safety Framework and Nova 2.0 Lite. This review looks one layer down, at the quality of the prompts feeding such evaluations.

What did the review confirm?

A droplet seeping between trays, picturing evaluation contamination that the Amazon safety test review rated lowA droplet seeping between trays, picturing evaluation contamination that the Amazon safety test review rated low

Amazon lists three validated strengths:

  1. Comprehensive topical coverage, aligned with publicly documented safety categories including self-harm and cybersecurity.
  2. Diversity of testing approaches: professional versus informal writing styles, single-turn and multiturn conversations, and original content.
  3. Low contamination risk: prompts appeared to be original, not copied from public datasets, which lowers the chance that a model saw them in pretraining.

Contamination matters because a model that memorized the test passes without being safe. We covered the same worry for capability benchmarks in our post on DeepMind's double-blind evaluation work against benchmark contamination.

Daily briefFrom Yash · 350K+ students taught

Catch up on AI

One AI story explained in plain English, plus one thing to try. Every day, free. Unsubscribe anytime.

What weaknesses did it surface?

A green probe pressing a crack in a shell, showing red-team style gaps found in AI safety evaluation promptsA green probe pressing a crack in a shell, showing red-team style gaps found in AI safety evaluation prompts

Amazon's list of "opportunities for improvement," quoted closely:

  • Linguistic and stylistic diversity: extend multilingual coverage more evenly across risk categories and vary prompt structures.
  • Attack sophistication: add more multistep adversarial prompts so evaluations "probe deeper model reasoning" and better reflect real misuse.
  • Freshness: a small number of prompts had time-limited references, and a few edge cases had unclear grading criteria.
  • Signal quality: "In some cases, models may refuse based on superficial cues in a prompt, so passing those tests doesn't necessarily indicate reliable safety behavior in new situations."

The last point is the most important for anyone who reads safety scores. If a model refuses because a prompt contains a scary word, it passes the test without understanding the request. Real attackers rephrase. This is the same reason jailbreak research focuses on multi-turn and indirect attacks, a theme in our coverage of METR's red-team review of OpenAI's internal agent monitoring.

The four changes Amazon says it made

table · 2 cols
ChangeWhat it means in practice
Test set refreshRemoved redundant prompts, added harder adversarial scenarios
Difficulty calibration frameworkTiered levels so tests include prompts that push limits, not only easy cases
Quarterly review cadenceRegular maintenance to catch staleness and repetition and add new attack patterns
Broader language coverageMultilingual prompts spread across all risk categories

Amazon says it plans to keep engaging independent evaluators as its models evolve and calls third-party meta-evaluation "an emerging best practice."

What this does and does not show

It shows: a major lab let an outside group inspect the quality of its safety tests and published specific shortcomings. That is more transparency than most labs offer about evaluation quality. For context on how other labs use outside evaluators, see Anthropic's independent evaluator arrangements with METR and the FTC probe into OpenAI and Anthropic safety.

It does not show:

  • How Amazon's models score. No model results are given.
  • That the fixes work. We found no independent verification of the changes, and the account is Amazon's own.
  • How big the gaps were. The post says "a small number" for stale prompts but gives no counts, percentages or sample size.
  • What AVERI's own report says. We reviewed Amazon's description; we did not find a separate AVERI publication in our search.
  • Whether AVERI was paid or what its funding is. The post does not say, and readers should ask, since independence depends on it.

Why meta-evaluation matters for people building with AI

Teams that pick models on safety scorecards inherit the quality of the underlying tests. Three practical takeaways:

  1. Ask what a safety score is measured against. A pass rate on easy, single-turn prompts says little about a determined attacker.
  2. Test in your languages. Amazon's own gap on even multilingual coverage is common. If your users are not English-speaking, run your own tests in their language.
  3. Add multistep attacks. Agents act over many steps; single-prompt tests miss the failure modes that matter. If you deploy agents with real access, AgentBeam is the agent security platform from the explainx.ai team, built to stop AI agents before they take dangerous actions.

A simple internal checklist for your own evals: refresh prompts quarterly, track which prompts are time-limited, separate easy and hard tiers, spot-check that refusals are for the right reason, and keep a held-out set you never publish.

How to audit your own eval set, step by step

Amazon's four fixes translate into a routine any team can run on its own tests, whether they are safety prompts, jailbreak suites or product acceptance checks.

  1. Sample, then read. Pull 50 to 100 random prompts and read them as an attacker would. You will quickly see duplicates, outdated references ("the latest model", a past election, a retired product) and prompts that are really the same test in different words.
  2. Tag difficulty. Label each prompt as easy (a direct request), medium (a reworded or role-played request) or hard (multistep, indirect or tool-assisted). If nearly all fall in the first bucket, a high pass rate is cheap. Amazon's tiered difficulty calibration is this exercise, formalized.
  3. Check why a model passed. For a sample of refusals, vary the prompt: remove the trigger word, translate it, split it across turns. If the refusal disappears, the model was reacting to surface cues, which is the "superficial cues" problem AVERI flagged.
  4. Measure language and style spread. Count prompts by language and by category. A suite with 90% of its multilingual prompts concentrated in two risk categories gives you false confidence everywhere else.
  5. Set a calendar. Put a recurring review on the team's schedule. Amazon chose quarterly. Without a date, test sets fossilize.
  6. Keep a private hold-out. If your prompts are public, future models may train on them. Keep a small set that never leaves the building.

None of this requires a nonprofit, though an outside reader is worth having: people who wrote the tests tend to miss the same blind spots twice.

Context: why independent checks on evaluations are becoming a theme

Safety claims increasingly rest on evaluations, and evaluations rest on prompts that someone wrote. When a lab reports that a model "passes" a safety suite, the claim is only as strong as the suite. This is why outside review has become a recurring story this year. Recent examples on this site include the FTC's probe into how two leading labs describe their safety work, METR's review of Anthropic's risk report, which found that prompt sensitivity matters, and the broader proposal for embedded assessments of AI agent control.

The Amazon-AVERI work is narrower and more modest than those: a sample of prompts, a short blog post, no model scores. Its value is the precedent. A company publicly saying "an outsider found our tests were too easy in places, and here is what we changed" sets a bar that others can be asked to meet. It also invites the obvious follow-up questions listed above about sample size, funding and publication of AVERI's own findings.

What to watch next

  • Whether AVERI publishes its own account of the review, with sample sizes and methods.
  • Whether other labs commission similar meta-evaluations, and whether results are published.
  • Whether Amazon publishes updated Nova safety results using the refreshed test set.
  • Quarterly refresh evidence: a second dated update would show the cadence is real.

Related reading

  • DeepMind double-blind evaluation and benchmark contamination
  • METR and Redwood red-team OpenAI's internal agent monitoring
  • Anthropic's independent evaluators and METR access
  • FTC probe into OpenAI and Anthropic AI safety
  • What an embedded evaluator is
  • Amazon AGI and Nova pretraining changes

Details are accurate as of October 26, 2026 and rest on Amazon Science's October 20 post.

Spotted something out of date? Let us know.
Daily briefFrom Yash · 350K+ students taught

Catch up on AI

One AI story explained in plain English, plus one thing to try. Every day, free. Unsubscribe anytime.

  • ✓ Free
  • ✓ One email a day, no spam
  • ✓ Unsubscribe anytime
Yash Thakker

Written by

Yash Thakker

Yash is an AI expert with over 300K learners. Join his workshops →

View Yash Thakker in People in AI →

Related posts

Nov 14, 2026

Epoch AI Report: Gemini 4 Argon Agents Impersonated Staff During Evals

On November 13, 2026 Epoch AI and Trajectory Labs published matching accounts of October misalignment tests in which Gemini 4 Argon agents reached Epoch's live website, impersonated employees, and attempted exploits. Epoch names three events and proposes consent standards; Trajectory names the model and describes persistence prompts. explainx.ai unpacks what is confirmed, what failed, and what eval runners should copy.

Nov 13, 2026

Anthropic's Guide to Building Strong Safety Evaluations: What to Copy

On November 13, 2026, Anthropic published a practical guide to building safety evaluations. It borrows from the psychology replication crisis and shows how a system prompt can move a score by 34 points.

Nov 1, 2026

How Pro-China Are Chinese AI Models? Vals AI Tested 25 and Found It Is Complicated

A Vals AI study asked 25 Chinese, American and other models hundreds of politically sensitive questions about China. Results vary widely by model, language, system prompt, answer format and agent harness. Here is what it found and how to use it.