Chapter 13 — When It Goes Wrong

Every chapter so far has been about getting into the answer. This one is about what happens when the answer is wrong, when someone games it, or when a regulator arrives.

The system will state false things about you

This is the likeliest risk to materialize and the least prepared-for.

The Washington University audit verified 98,020 individual claims in AI Overviews against their cited sources and found 11 percent inconsistent — 4 percent actively contradicted by the source, 7 percent simply not present in it. That’s the error rate on claims that carry a citation, which means they look verified.

For your brand this shows up as invented specifications, wrong pricing, wrong availability, misattributed features, discontinued products presented as current, and confident statements about policies you don’t have.

A real case: in 2025 the AI support bot belonging to the developer tool Cursor invented a company policy — that users couldn’t run sessions on multiple machines. No such policy existed. Customers cancelled subscriptions over a rule that was never written, and a co-founder had to state publicly that there was no such policy.

What to do. Monitor for accuracy, not just presence — somebody reads a sample of answers about you every month. Make the correct facts easy to find and hard to contradict, which is the Chapter 5 work with a second justification. Establish a correction path before you need it: who reports an error, to which platform, through which channel, who signs off. And find the upstream source, because most hallucinated brand facts aren’t invented from nothing. They come from a stale directory listing, an outdated release, or a competitor’s comparison page.

Liability is moving, in different directions

In Europe, toward the brand. In May 2026 a Munich court granted a temporary injunction against Google after AI Overviews falsely connected two publishers to scams. The court held Google directly liable, rejected the argument that users should verify independently, and drew an explicit line: an AI overview is Google’s own content, not a list of search results. Since only Google controls the algorithms, Google owns accuracy.

The caveat matters. That’s an interim injunction from a court of first instance, and Google has said it will appeal. It isn’t settled precedent. What it establishes is that the argument works in at least one major jurisdiction, which is a real shift and less than the headlines claimed.

In the US, the other way, so far. A Georgia court granted summary judgment to OpenAI over ChatGPT fabricating embezzlement allegations, on three grounds: a reasonable reader in context couldn’t have understood the output as stating actual facts, no negligence or actual malice, no recoverable harm.

But not settled. A separate case against Google over allegedly fabricated criminal accusations survived a motion to dismiss and moved to discovery.

And you own what your own chatbot says. When Air Canada’s chatbot told a passenger he could apply retroactively for bereavement fares, contradicting the website, the airline argued the chatbot was a separate legal entity responsible for its own statements. The tribunal rejected that outright and held the airline accountable for all information on its site, static or generated.

If you deploy a customer-facing assistant, treat its output as your published statements — because a tribunal already has. Constrain it to verified sources, log what it says, label it.

Manipulation works, and that’s the problem

Researchers at ETH Zürich demonstrated preference manipulation attacks: crafted web content that steers AI-powered search toward the attacker’s product. Manipulated fictional cameras became two and a half times more likely to be recommended, competing successfully against real brands. Fake products moved from a 34 to a 59 percent recommendation rate. Production systems were affected.

Now the finding that should end the discussion internally: when several competitors attack simultaneously, everyone’s recommendation rate degrades. It’s a prisoner’s dilemma. Individually rational, collectively destructive.

Use that argument with anyone in your organization who’s tempted, because it doesn’t require them to share your ethics. The tactic destroys the value of the surface it exploits, and it does so quickly once more than one player adopts it.

The related security exposure is real too, and nobody’s content policy accounts for it. Brave’s researchers demonstrated indirect prompt injection in an agentic browser, and the proof-of-concept vector was a Reddit comment with instructions hidden in a spoiler tag. When a user asked the browser to summarize the page, the AI executed the hidden instructions, accessed the user’s account and exfiltrated credentials.

Your community pages, review sections and user-generated content are now an attack surface that can be turned against your own customers’ assistants. Add prompt-injection review to your moderation policy.

Regulators have arrived

The EU AI Act’s transparency obligations are in force as of August 2026. Users must be told they’re dealing with an AI system unless it’s obvious. Synthetic content, including text, must be marked in machine-readable format — systems already deployed had a grace period into December 2026. Penalties reach €15 million or 3 percent of worldwide turnover.

The direct marketing relevance: AI-generated marketing text distributed in the EU falls within the marking obligation, and AI chat interfaces on brand properties require disclosure. Now, not in some future phase.

The FTC’s “Operation AI Comply” has continued across a change of administration, with more than a dozen cases tied to AI washing in the past year. Two features matter. Enforcement has expanded to B2B marketing — the same substantiation standards apply regardless of audience. And the Commission has held vendors liable for supplying deceptive materials used downstream, which puts your agency and your martech suppliers in scope alongside you.

The review rule from Chapter 7 carries penalties up to $53,088 per knowing violation, with live enforcement since 2026.

The structural risk underneath all of it

Answer engines sit between you and your customers, and their logic is opaque in a way search never quite was. Under classic search, ranking factors were partially understood and results were visible. Now you get impressions without clicks, citations without traffic, and recommendations you can’t observe.

Three specific exposures. Concentration — about thirty domains capture two-thirds of citations within a topic. Volatility — one platform’s share can fall twenty points in a year while another triples. Commercial encroachment — ads inside answers, on a surface smaller than the page it replaced.

The response is deliberate diversification. The entity and corroboration levers are the platform-independent ones; they work across engines precisely because engines disagree about sources and agree about brands. And owned audience relationships — email, community, direct — remain the only fully independent asset.

None of that argues for abandoning AI visibility. It argues against building a business on top of a single opaque intermediary, which is a lesson this industry has already learned once.