Gray Swan AI has announced IPI Black, a $300,000 red teaming challenge aimed at indirect prompt injection, the attack that matters most for AI agents. The company's pinned post on X asks "Is prompt injection solved? Here's $300,000 to find out," and says it was built "in partnership with Anthropic, Meta, Google DeepMind, Reflection AI and our other frontier lab sponsors" as "the largest and most realistic AI red teaming challenge to date." A post a day earlier said IPI Black launches October 15 with "$300K, 3 waves."
We could not load the challenge's own rules page (the Arena returned a rate-limit error), so this post covers what Gray Swan has said publicly, adds context from its earlier round, and flags what remains unverified.
TL;DR
| Item | What is stated | Source |
|---|---|---|
| Name | IPI Black (indirect prompt injection) | Gray Swan on X |
| Prize pool | $300,000, in three waves | Gray Swan on X |
| Modes | White box: you watch the AI respond and adapt. Black box: you go in blind, like a real attacker | Gray Swan on X |
| Partners named | Anthropic, Meta, Google DeepMind, Reflection AI, "other frontier lab sponsors" | Gray Swan on X |
| Who can enter | "Anyone who can talk to AI can win" | Gray Swan on X |
| Models | "The newest, most challenging models" | Gray Swan on X |
| Prize splits, dates, rules, judging | Not verified | Rules page unreadable |
What is indirect prompt injection and why does it matter?
Direct prompt injection is a user typing "ignore your instructions." Indirect prompt injection (IPI) is quieter: the attacker plants instructions in something the agent will read later, a web page, a calendar invite, a PDF, a code comment, a tool result. The user never sees it. If the agent treats that text as a command, it can forward private data, change a file or call a tool.
This is the attack class that grows with agent autonomy. A chatbot that only answers has little to steal. An agent with email, a browser, a terminal and payment access has a lot. We have covered the problem from several angles. Our post on Boris Cherny and GPT-6 Astra's prompt injection benchmark looks at how labs measure it, and AI traffic overtaking humans and prompt injection shows how much web content is now machine-read.
White box versus black box
A decision fork signpost representing the choice between white box and black box modes in IPI Black
Gray Swan's two modes mirror how security people think about attackers:
- White box: you can see the model respond and adapt your attack. This approximates a researcher with access to a model's outputs, probing for weak points iteratively.
- Black box: you go in "blind," like a real attacker who cannot watch the system's internal behavior. This is closer to the field: a criminal who plants a payload on a web page and hopes an agent reads it.
Having both matters. A defense that holds when the attacker gets many tries and sees every response may still fail when the attacker sends one payload. And a defense that looks strong against one-shot attacks can collapse once an attacker can iterate. Reporting results by mode makes the numbers more useful.
How this compares with earlier Gray Swan rounds
Gray Swan has run a series of Arena challenges, and we could not open their rules pages to compare prize pools, so we do not give per-round figures here. What the company has described publicly about its March 2026 indirect prompt injection round is its scale: 13 frontier models, 464 red teamers and more than 272,000 attacks, designed with UK AISI, US CAISI and frontier labs.
Two differences stand out in the new announcement: Reflection AI is named alongside the big three labs, and the framing is three waves rather than a single contest. That suggests iteration: labs can patch between waves and organizers can see whether the second wave's attacks work after the first wave's lessons. Gray Swan calls the $300,000 pool the largest of its challenges to date, which we could not independently check.
Even low per-attempt success rates matter in this setting: a patient attacker with thousands of tries only needs one to get through, which is why contests report success within a fixed number of attempts rather than a simple pass or fail.
What the headline does and does not tell you
It tells you that several frontier labs consider prompt injection unsolved enough to fund a six-figure public contest, and that Gray Swan, which also sells runtime detection, has the lab relationships to run it. Gray Swan raised a $40 million Series A in May 2026 and has announced guardrail integrations with gateways such as Bifrost and TrueFoundry, per its news page. That is context worth keeping in mind: the company is both an independent red-teaming platform and a vendor of defenses.
It does not tell you any model's actual robustness. Results from earlier rounds depend on scenarios, scoring and which models were in scope. A lab's participation is not an endorsement of the scoring, and the competition's outcomes will be published, if at all, by Gray Swan and its sponsors. Until results appear, the news is the contest's existence and size.
Why contests like this exist at all
Prompt injection resists the usual software fix. In classic security, a vulnerability is a bug that a patch removes. In language models, the vulnerability is partly the feature: the model is designed to follow natural-language instructions wherever they appear in its context, and it has no reliable internal marker for which instructions are trusted. Defenses so far, such as training models to resist injected text, filtering inputs, sandboxing tools and adding classifiers, reduce success rates but do not bring them to zero, which is why the numbers are reported as percentages instead of pass or fail.
Open contests fill a gap that internal red teams cannot. Thousands of creative outsiders, paid per success, find attack styles that a lab's own staff would not think of, and the best attacks transfer across models. Labs get a stream of concrete failures to train against. Participants get prize money and, for some, a path into AI security careers. The risk is that publishing winning attacks teaches bad actors too, which is why many contests restrict disclosure for a period.
For buyers of AI agents, the useful takeaway is procurement hygiene. Ask vendors which injection tests their agent has faced, whether they report results by attack mode, and what the agent can do if an injection succeeds. A vendor that can answer the third question, with permission boundaries and approval gates, is safer than one that only quotes a defense success rate.
What this means for what you build
- Treat tool outputs as untrusted input. Anything an agent fetches can contain instructions. Separate the system's instructions from retrieved data in your prompts and permissions.
- Limit blast radius. Give agents the least access they need. A prompt injection that reaches an agent with read-only access is an annoyance; one that reaches an agent with send-email and delete-file access is an incident.
- Require confirmation for risky actions. Human approval for payments, external messages and destructive commands blunts most injections.
- Monitor, do not just prevent. Log tool calls and flag unusual sequences. Our post on Gemini 4 Argon's safety report describes how one lab uses activation probes to monitor behavior.
- Add runtime controls. If you run coding agents or browser agents, a tool like AgentBeam, the agent security platform from the explainx.ai team that stops AI agents before they take dangerous actions, adds a layer outside the model.
For a related look at how agents fail in practice, see our post on how frontier labs skip security basics and the LAST secure code benchmark.
If you want to take part
Gray Swan says the challenge is open to "anyone who can talk to AI," and describes a separate free set of six scenarios that pay $100 each time you trick the AI, no coding needed. Check the Arena page for the rules before you start: eligibility, how attacks are judged, and whether you may publish your findings. Many such contests require disclosure to the organizers before public release, and some require concealment as part of a valid attack. Read those terms carefully, and only test the systems and scenarios the contest authorizes.
Claimed versus verified
| Claim | Source | Status |
|---|---|---|
| $300K prize pool in three waves | Gray Swan on X (pinned post and October 14 post) | Stated by the organizer; rules page not readable by us |
| Partners include Anthropic, Meta, Google DeepMind, Reflection AI | Gray Swan on X | Stated by Gray Swan; no lab statements checked |
| "Largest and most realistic" challenge | Gray Swan | Marketing superlative; not independently checked |
| Launch on October 15 | Gray Swan on X, October 14 post | Stated by Gray Swan |
| Results, winners, per-model success rates | Nobody yet | Pending |
Details are from Gray Swan's public posts on October 6 to 15, 2026 and may change.