Chapter 3 — What Happens in the Two Seconds Before an Answer

You don’t need to be an engineer to run this well. You do need an accurate picture of what happens between a question and an answer, because nearly every bad tactic in this field comes from an inaccurate one.

Here is the sequence, in plain terms.

Step one: it decides whether to look anything up

Before anything else, the system decides whether to search the web at all.

This is documented rather than inferred. Google’s grounding system assigns each prompt a score between 0 and 1 estimating whether searching would improve the answer, with a default threshold of 0.3. Below that, the model answers from what it already knows. No search, no sources, no citations.

The observed rates are striking. Profound, analyzing roughly 730,000 real ChatGPT conversations, found only 18 percent triggered a web search. Semrush’s clickstream data put web search enabled on 34.5 percent of ChatGPT queries — down from 46 percent a year earlier.

Sit with that for a second.

Somewhere between two-thirds and four-fifths of questions are answered from memory. For those questions, your website is irrelevant. Not underperforming. Irrelevant. What matters is whether the model already knows your company exists, what category it belongs to, and what it’s good at.

That single fact is the reason Chapter 5 comes before Chapter 8. Being known is upstream of being read.

Step two: it breaks the question apart

When the system does search, it usually doesn’t run one search. It splits the question into several sub-questions and runs them in parallel. Google calls this query fan-out, and describes it openly: one engineering director put it as “doing a dozen searches for you in the time it takes to do one.”

Nobody outside Google knows how many sub-queries, and anyone who tells you a specific number is guessing. But the consequence holds regardless.

Our dental-practice question fans out into something like: payroll systems for small businesses, payroll for medical and dental practices, handling tipped employees, part-time employee payroll, QuickBooks payroll integrations, pricing for teams under twenty.

You might win four of those and lose on tipped employees, and lose the answer.

You are competing on questions nobody typed and no keyword tool will ever show you.

Step three: it retrieves passages, not pages

Each sub-question returns candidate passages. Not pages — passages.

Documents get chopped into chunks of a few hundred tokens each, indexed independently. Google’s search chunker caps at 500 tokens. Microsoft recommends starting at 512. Google’s own retrieval product defaults to 1,024 with overlap.

So your 3,000-word guide does not compete as a guide. It competes as roughly eight separate fragments, most of which will never be seen together.

This has a practical edge. A section that opens “as we discussed above” is ambiguous when it arrives alone. The systems do some work to mitigate this — headings can be attached to chunks, and neighboring chunks can be pulled in alongside a match — so the situation is less dire than some presentations suggest. But if a section can’t stand up by itself, don’t count on it.

Step four: it compares passages against each other

Surviving passages get re-ranked before they reach the model, and there is good evidence the comparison is head-to-head rather than absolute.

Google Research published work showing that asking a model to compare two documents at a time — a much simpler task than scoring one in isolation — lets a modest open model match GPT-4’s re-ranking quality. Google then patented the method.

That is a methods paper, not a description of what runs in production, and it’s worth being precise about the difference. But it points at something that matches everything else we observe: your passage doesn’t need to be good. It needs to beat the specific passage it’s compared against.

Which reframes the whole content question. A verifiable number, a named source, a real price, a dated measurement, a specific example — these give a comparison something to prefer. A well-written paraphrase of common knowledge gives it nothing.

Practitioners call the difference information gain. It is the most transferable idea in this entire field.

Step five: it checks the claims

Passages that survive get checked against the answer being drafted. This is documented in detail.

Google’s grounding check produces a support score from 0 to 1 for how well an answer is grounded in the retrieved facts, with a default citation confidence threshold of 0.6. Claims get mapped to source chunks at roughly sentence granularity.

Google’s granted patent on generative summaries goes further, and this is the part with strategic consequences. Confidence for each portion of the summary is derived from three things: the model’s own confidence, the trustworthiness of the supporting documents, and how many documents verify that portion. Low-confidence summaries can be suppressed entirely.

Read that last mechanism again.

How many independent sources agree with a claim is an input to whether the claim survives.

A fact stated only on your website is a single-source claim. The same fact appearing on your site, in a trade publication, in a directory listing, and in a structured knowledge base is a multi-source claim. The architecture prefers the second, mechanically, not as a matter of taste.

That is the entire argument of Chapter 7, and it is not a marketing metaphor. It is a scoring function.

Step six: it writes, differently every time

Validated passages go into the model’s context with source identifiers attached, which is how specific sentences link back to specific documents.

The output is probabilistic. Run the same prompt twice and you get different answers, and the variance is larger than most people assume. Researchers at the University of St. Gallen found that identical prompts, run on the same day, produced source overlap of only 0.34 to 0.42 and brand-mention overlap of 0.45 to 0.59. The instability held inside 24-hour windows, ruling out news cycles.

Their recommendation: at least seven runs of a prompt per day before you believe a number.

Almost no commercial dashboard does that. Chapter 11 deals with the consequences.

The six things to remember