Most AI agent failures since 2024 did not happen because the model was too weak. They came from what the agent was allowed to touch: a credential that reached production, an instruction hidden in an email it read, a package it installed, or a purchase it made without asking. Of the 63 public incidents in my AI agent incidents database as of 6 October 2026, 41 caused real-world harm, coding agents account for 49%, and the most common harm is leaked data (37%). The controls that would have stopped most of them are unglamorous: scoped credentials, separate production access, confirmation before anything destructive or financial, spending caps, and an allowlist for what an agent may install.

This post picks the incidents that teach the most, links each one to its original reporting, and turns them into seven lessons. The full table, with severity ratings and a free CSV, lives in the AI agent incidents database. For a formal taxonomy of how agents break, read AI agent failure modes; this is the field record underneath it.

Abstract illustration of a chain of grey blocks with one orange link snapped open in the middle
One weak link, usually an over-broad permission, is enough.

Key takeaways

  • Access, not intelligence, causes the damage. The production-database deletions at SaaStr, DataTalks.Club and PocketOS each needed a credential that reached production. None needed an attacker.
  • Prompt injection is common in research, rarer in the wild. 20 of 63 incidents (32%) involve it, but only 7% of the 41 real-world harm cases.
  • The tooling is a supply chain. A copycat MCP server, a poisoned Amazon Q release and 341 malicious OpenClaw skills all reached users through normal installs.
  • Customer-facing agents make commitments you own. A tribunal ordered Air Canada to honour what its chatbot told a customer.
  • Few failures get a postmortem. Only 37% of logged incidents have a public technical write-up, so most teams learn from press reports.

The record so far: 63 incidents, counted

The database logs every public incident I could verify since January 2024 in which an AI agent took or enabled an action that caused harm. Each row has a date, the agent type, what happened, how it ended, a severity rating and one to three sources, at least one of them original reporting, a vendor advisory or a court document. Here is how the 63 rows break down as of 6 October 2026.

MeasureCountShare
Incidents that caused real-world harm4165%
Flaws in shipped products, confirmed and fixed before known exploitation2235%
Involving coding agents3149%
Main harm: data leaked2337%
Main harm: system compromise1727%
Main harm: data deleted813%
Involving prompt injection2032%
With a public postmortem or technical write-up2337%
Rated severe (many organisations harmed, losses above $1 million, or state-backed)914%

By year, the log holds 3 incidents from 2024, 31 from 2025 and 29 from 2026 up to 6 October. Part of that rise is more agents in production and part is more people looking, so read it as reported incidents, not a true failure rate. The broader record points the same way. Stanford HAI's 2026 AI Index counts 362 documented AI incidents of all kinds in 2025, up from 233 in 2024. It also notes that agents still fail roughly one in three attempts on structured benchmarks, even though OSWorld success rose from about 12% to 66.3%. Gartner expects over 40% of agentic AI projects to be canceled by the end of 2027, due to escalating costs, unclear business value or inadequate risk controls (June 2025).

Lesson 1: credentials that reach production turn a slip into an outage

The most damaging incidents in the log share one shape: an agent working on something minor held a credential that could destroy something major.

Even the disputed case ends in the same place. The Financial Times reported that AWS's Kiro agent decided to "delete and recreate the environment", causing a 13-hour interruption to a customer-facing system (The Decoder, summarising the FT). Amazon replied that the cause was "user error, specifically misconfigured access controls" and added mandatory peer review for production access. Whoever is to blame, the fix is the access boundary.

The control: production credentials never live where an agent works; tokens are scoped to the job they were made for; backups sit outside the blast radius; destructive commands need a human yes. Masad's two fixes, separate environments and a planning-only mode, are a good default for any coding agent.

Lesson 2: agents act on assumptions, so check the result of every step

Agents fail differently from scripts. A script stops when a command fails; an agent often carries on as if it had worked, or improvises around the obstacle.

The control: verify state after every write instead of trusting the agent's summary. Keep task state in a durable store with checkpoints, not only in the context window. Put rules like "confirm before deleting" in configuration or permissions, where compaction cannot drop them. And give every long-running agent a kill switch that does not depend on the agent reading a message. The inbox case is the clearest argument for that last one. More on keeping state in context window management, and on watching outcomes rather than uptime in agent monitoring and observability.

Lesson 3: everything the agent reads is an attack surface

Prompt injection, instructions hidden in content the agent reads, appears in 20 of the 63 incidents. Most are flaws that researchers found and vendors fixed in products such as Microsoft 365 Copilot, Salesforce Agentforce, GitHub Copilot and Cursor. The in-the-wild cases are fewer, and stranger.

OWASP ranks prompt injection as LLM01 in its 2025 Top 10 for LLM applications, with Excessive Agency at LLM06; in December 2025 it added a separate Top 10 for agentic applications. The control: treat everything an agent reads as data, never as instructions. Keep secrets out of reach of any agent that processes public input. Allowlist the URLs an agent may send data to, which is exactly what Salesforce did, and require confirmation for transfers. The defensive patterns are in the prompt injection defence guide.

Lesson 4: the agent's tools are a supply chain

Agents install packages, MCP servers, extensions and skills, often because someone told them to. Each of those is code from a stranger.

The control: pin versions, and review an MCP server or skill before an agent can install it. Keep an allowlist of what agents may install. Never run AI command-line tools with permission-bypass flags in CI, and rotate tokens the day a dependency is flagged. The 47-control AI agent security checklist covers the full set.

Lesson 5: a customer-facing agent's words are your commitments

When an agent talks to customers, its mistakes are public, and sometimes binding.

The control: ground policy answers in your written policy and nothing else, and say clearly that the customer is talking to a bot. Keep a fast route to a person, and require explicit confirmation before any purchase or commitment. The design of that checkpoint is in how to add human-in-the-loop to an agent.

Lesson 6: loops and meters create runaway bills

Two incidents in the log are bills rather than breaches, and both came from pricing that meters the agent's effort.

OWASP files this risk as LLM10, Unbounded Consumption. The control: hard spending caps that stop the run rather than send a warning, a step limit per task, loop detection that breaks on repeated states, and an alert at half the budget. The setup is in how to set AI agent spending limits, and the pricing side in AI agent pricing trends 2026.

Lesson 7: some agents work perfectly, for the wrong person

Four severe rows involve no malfunction at all. In November 2025 Anthropic reported that a group it assessed with high confidence as Chinese state-sponsored used Claude Code to attempt infiltration of roughly thirty targets, succeeding in a small number of cases. The attackers used AI for 80 to 90% of the campaign, with human decisions needed at perhaps four to six points (Anthropic).

The lesson for defenders is speed: an attack run mostly by an agent moves faster than a human team, so detection and credential rotation have to be automatic too. The lesson for anyone shipping an agent is that capability is dual-use, which is why vendors' abuse monitoring belongs on your evaluation list. The 2026 AI agent security breaches roundup covers the defender's side.

A pre-launch checklist from the incident record

Mapped back to the incidents above, seven controls would have stopped or contained most of them.

ControlIncidents it addresses
Separate production credentials; tokens scoped to one job; backups outside the agent's reachReplit and SaaStr, DataTalks.Club, PocketOS, Kiro
Human confirmation before destructive, financial or outbound actionsAntigravity, OpenClaw inbox, Operator, Grok and Bankrbot
Verify each step's result; durable task state; an external kill switchGemini CLI, OpenClaw inbox, PixelLeak
Allowlists for outbound URLs and installs; pinned versionsForcedLeak, postmark-mcp, ClawHub, Nx
No secrets reachable from agents that read untrusted inputCline, Nx, Amazon Q
Hard spending caps and step limitsReplit Agent 3, Cursor 2025
Answers grounded in written policy, a labelled bot and a human fallbackAir Canada, Cursor's Sam, Taco Bell

None of these needs a better model. Before you give an agent any of these permissions, it is worth asking whether the task suits an agent at all; the free Can AI do this task? checker gives a quick first read.

What I take from the record

I built the database because these stories travel for two days and then vanish, and nobody keeps the list. Read in one place, the pattern is plain: an agent with more access than the task needed, and no checkpoint between a wrong decision and a real consequence. Two years of model improvements have not changed that shape, and the 2026 AI Index still puts agent failure on structured benchmarks at roughly one attempt in three.

That is also why every agent on Gravity is built and kept working by its builder rather than assembled by the person using it: someone whose job it is should own these controls. Gravity is in public beta. The database is refreshed weekly, and every new row is checked against its source before it goes in.

Frequently asked questions

What are the most common AI agent failures?

In Gravity's AI agent incidents database, which held 63 verified public incidents as of 6 October 2026, the most common harm is leaked data (23 incidents, 37%), followed by system compromise (17) and deleted data (8). Coding agents are involved in 31 incidents (49%). Among real-world cases, the pattern that repeats is an agent holding broad credentials and running a destructive command nobody asked for.

Has an AI agent ever deleted a production database?

Yes, several times. Replit's agent deleted SaaStr's production database during a code freeze in July 2025. A Claude Code run wiped DataTalks.Club's production infrastructure in February 2026. And in April 2026 a Cursor agent deleted PocketOS's production volume and its backups through an over-scoped Railway token, in 9 seconds.

How many AI agent incidents involve prompt injection?

20 of the 63 incidents logged as of 6 October 2026 (32%) involve prompt injection, meaning instructions hidden in content the agent read. Most are flaws researchers found and vendors fixed. Among the 41 incidents that caused real-world harm, the share is 7%, including the Cline triage bot in February 2026 and the Grok and Bankrbot token transfer in May 2026.

Is a company liable for what its AI agent tells customers?

In the one tribunal ruling in the database, yes. In Moffatt v. Air Canada (February 2024), the British Columbia Civil Resolution Tribunal rejected the airline's argument that its chatbot was a separate legal entity and ordered it to pay the customer $812. Other jurisdictions have not ruled the same way yet, so treat it as a strong signal rather than settled law.

How common is it for AI agent projects to fail?

Gartner predicted in June 2025 that over 40% of agentic AI projects will be canceled by the end of 2027, due to escalating costs, unclear business value or inadequate risk controls. Stanford's 2026 AI Index adds that AI agents still fail roughly one in three attempts on structured benchmarks.

How do you prevent AI agent failures?

Mostly with access controls, not better prompts. Keep production credentials away from agents and scope every token to one job. Require human confirmation before destructive, financial or outbound actions, and verify each step's result. Cap spending with limits that stop the run, allowlist what agents may install, and keep secrets out of reach of any agent that reads untrusted content.

Sources