The slide last month said the brand showed up in two answers out of five. The slide for the Thursday readout says one in eight. Nobody touched the website, nothing shipped, no competitor launched anything, and the only honest sentence available right now is that the number moved.
Two days is not enough time to rebuild a measurement program. It is enough time to work out whether the first number was ever worth reporting, which is a different and more useful question.
Nobody Publishes The Number You Are Reporting
Start with the part the dashboards do not say out loud. No engine reports how often it names your brand. There is no ChatGPT equivalent of an impressions column, and there is no feed to subscribe to.
The sharpest version of this turned up in r/SEO in September, in the top comment on a thread from somebody asking the same question:
“no one can tell you how much you show up in ChatGPT, for example, since they don’t publish any statistics. the tools are guessing for the sole purpose of charging people money.” — u/VillageHomeF, r/SEO, Sep 2026
He is right about the premise and wrong about what follows from it. Every visibility figure is produced by asking an engine a set of questions and counting the answers. That is a sample statistic, the same shape of thing as a poll.
A poll is not a guess. A poll with a badly drawn sample is.
Which Makes The Sample The Instrument
So the engine is not the only thing that moved between those two slides. The questions did the measuring, and a set of questions nobody designed will drift on its own, because the answers are probabilistic and the queries were never built to hold still against that.
.png)
Practitioners who have tried this arrive at the same place from the other direction, and one in that same thread set the minimum bar:
“two things separate a measurement from a demo: (1) repeated sampling — the run-to-run variance in AI answers is huge, so anything single-shot is unusable; (2) source attribution — knowing you’re invisible is useless unless you know which sources the engine pulled from instead.” — u/nordafriq, r/SEO, Sep 2026
Repeated sampling is the floor rather than the fix. Repeating a bad question gives a stable bad number.
Four Kinds Of Query That Move Your Number On Their Own
Before anything is rebuilt, read your own list. Four kinds of query change what gets measured without anything about your brand changing, and striking them is an afternoon’s work.
Strike those four and a number can move several points on its own, with nothing changed anywhere else. That alone explains some share of a swing like Thursday’s.
We Graded A Set That Had Already Passed The Script
Here is the part of this we can only know because we run it. Empact Partners builds query sets as the instrument for every visibility study inside Generative Engine Optimization, the workstream that works on whether an engine recommends you at all when a buyer asks it for options.
On one seven-thousand-query set, the deterministic checks had already passed: the duplicates were gone, the banned phrasings were gone, the length and opener distributions were inside their bands. By every automated measure the set was finished.
What the second pass found
Then a reader went through it query by query. Roughly a third of the set was rewritten and under one percent was thrown away. The problem was almost never a query that was obviously wrong. It was a query that measured something a little to the side of what the set was built to measure.
Afterwards we re-ran the same deterministic checks against the corrected set. They rejected nothing, exactly as they had rejected nothing before the corrections. The script passed both versions, which is the clearest statement available of what a script is able to see. On our reckoning it catches about one percent of what is wrong with a query set.
The Answer Is One To Three Thousand, And That Is The Smaller Half
The number people want is real, so here it is. A brand visibility study we would stand behind runs between one and three thousand queries, spread across the full surface of use cases rather than deepened on a few.
The reason is unglamorous. Concentration falls as specificity rises, so a set weighted toward broad questions and a set weighted toward narrow ones will report different visibility for the same brand in the same week, both correctly.
.png)
Which is also why a short set gets repaired the wrong way. The instinct is to top it up with whatever queries are easiest to add, and that quietly changes what the study is about. A refill belongs against the specific use cases that came up short, never across the set.
The Twin Almost Nobody Builds
One more thing decides what a set can see, and it comes from Qvery, our sister company, which measures what the engines answer and what they cite. In August it ran paired versions of the same buying questions, once bare and once with a single real constraint attached.
| Bare question | One constraint added | |
|---|---|---|
| How it reads | best ecommerce platform | best ecommerce platform for a store doing under $10k a month |
| What answered it | Review and roundup publishers | The vendors’ own documentation |
| Shared with the other’s top ten | Three sources | Three sources |
| Its most-present source appeared in | 12% of its answers | 10% of its answers |
The rates barely moved. Across the twenty most-present sources the median gap between the two phrasings was under three points, and not one moved ten points or more.
The roster moved almost entirely. Each phrasing’s top ten shared only three sources with the other, which means a set built from bare head terms never sees the vendor layer that answers constrained questions at all.
Qvery is ours, so treat the finding as an interested party’s and ask us for the runs behind it. Qvery states its own limit plainly, and it holds. The pattern tells you how to design a set. It does not prove that redesigning one moves anybody’s visibility.
What We Would Do Before Thursday
Not a rebuild. Read the existing list, strike the four kinds above, and count what is left. Then write the mix of what remains at the bottom of the slide: how many questions, how many broad against narrow, which markets.
The two numbers probably came from sets that were not the same shape, and saying so upstairs is a stronger position than defending either figure.
The half that is not an afternoon
Then the longer half, honestly. Building a set worth trusting means mapping the use cases before the phrasings, weighting them by how central each one is, generating at the scale above, and having somebody read every line of it. The reading is the part that cannot be bought, and it is the part our own numbers say the scripts will not do for you.
Most teams stop at the dashboard they do not trust, because the dashboard arrives and the set does not. If your own visibility number moved and you cannot tell anybody why, book time with me and we can work out whether the instrument is the problem.
