12 AI Visibility Sample Size Statistics for SaaS
How question count, repeated runs and wording affect AI visibility reports, and when a change is worth acting on.
On this page
- Highlights
- Why Repeating One Question Gives Different Brands
- How Accurate a 20-Question Visibility Check Is
- How Much Daily Repeats Steady a Visibility Number
- How the Choice and Wording of Questions Change Your Visibility
- How to Tell Whether Your Visibility Number Can Support a Decision
- Frequently Asked Questions
Empact Newsletter
One email a month on what is working in AI search and SaaS go-to-market.
In one B2B software category, an AI visibility score built from 20 buyer questions drawn at random from a set of 1,000 landed within five percentage points of the full set’s score in about 48% of draws. At 200 questions it did in about 98%.1
The score is the share of AI answers that name a brand, so on 20 questions a single answer moves it by five points. Two quick checks can disagree while nothing in the market changed, and moving budget on that gap means moving it on luck. A handful of questions picked by hand adds a second problem, because it can leave out whole parts of what buyers ask.
The draws above come from questions Empact Partners asks AI engines each week for a SaaS company we work with. We get SaaS products recommended in AI answers, and a score that jumps on its own would hide whether that work did anything.
Highlights
- 5.3 points was how far a 20-question reading typically landed from the full 1,000-question score in one B2B software category we measure, and 1.5 points at 200. A quick check’s number can easily sit five points from the full set’s score.1
- Fewer than 1 in 100 pairs of ChatGPT or Google AI answers to the same brand-recommendation prompt named the same set of brands, so one answer is one draw from a larger pool.2
- About 5 brands on average came back in 80% or more of 100 ChatGPT answers to a B2B software prompt, out of about 44 named at least once. A brand outside that handful shows up in some answers and not in others.3
- 41% of ChatGPT prompts that named brands regularly never gave an identical brand set on any two days in four weeks of daily tracking, so the brands named change from one day to the next as a matter of course.4
- 39% to 71%: the range covering nine in ten of one brand’s readings at 10 prompts in OtterlyAI’s test, against 51% to 60% at 100, so the band tightens fast as prompts are added.6
- 0.288 was the overlap, on a scale where 1 means the same brands, between the brands recommended for two rewordings of one buying question. The same question asked again scored 0.50 to 0.61, so changing the wording changes what you measure.8
Why Repeating One Question Gives Different Brands
Every visibility number starts with one question asked once, and an AI engine writes a fresh answer each time. Four outside tests asked the same question again and again, and three of them measured how often two answers match:
- Fewer than 1 in 100 pairs of responses from ChatGPT or Google’s AI to the same brand-recommendation prompt named the same brands, and matching order was nearer one pair in a thousand. SparkToro had 600 volunteers run 12 prompts 2,961 times.2
- 0.3% of day pairs gave an identical ChatGPT brand set, and 41% of prompts never repeated a set in four weeks, in Detailed.com’s daily tracking of 1,300+ prompts that named a brand on at least a quarter of days.4
- 9.2% of cited URLs matched across three same-day runs of 10,000 searches in Google AI Mode, in SE Ranking’s June 2025 test. It counted cited pages rather than brands, so the pages one run cites are a weak guide to the next.5
The fourth, by Mike Sonders, ran 12 B2B software prompts 100 times each on ChatGPT and counted the brands across the answers:3
Brands Across 100 ChatGPT Answers to One B2B Software Prompt
Average across 12 prompts run 100 times each, Mike Sonders, Search Engine Land, 2026
View the data
| Label | Value |
|---|---|
| Named at all | 44 |
| In one answer | 10 |
| In 80%+ of answers | 5 |
Across 100 runs of one B2B software prompt, ChatGPT named about 44 brands on average, while any single answer shows about 10. Only about five of them came back in most answers. A brand outside that handful appears in some runs and not others, so one answer says little about how often it gets named.
More runs of the same prompt estimate that rate better, and more questions cover more of what buyers ask, which are two different fixes. A swing between two checks, the kind teams read as AI search volatility, can be nothing more than a fresh draw. Vlad Shvets, our founder, turns that into a rule for anyone who reports the number:
Report a rate across many questions and many runs, and treat the first brand an answer names with the same caution, because the first name changes from one run to the next as well.
Vlad Shvets on LinkedInOpens in a new tabHow Accurate a 20-Question Visibility Check Is
To see how far a 20-question check can be trusted, we drew sets of 10 to 200 questions from those weekly answers. They come from the Empact Panel, our own software for measuring AI search visibility. It saves every answer ChatGPT, Google AI Mode and Claude give to the buyer questions of each SaaS company we work with, so the full set’s score is always there to check a smaller draw against.
For the company in our opening example, the Panel holds 1,000 unbranded buyer questions per product line. We took the ChatGPT and Google AI Mode answers from 23 and 30 September 2026 and left out the company’s own brand, so every figure describes the other brands in its category. A reading is a brand’s share of answers that name it on a random subset of those questions, set against its share on all 1,000.
The first two figures are medians across every brand named in 5% or more of a product line’s answers, with both engines and both weeks counted together. The leader figure counts only product lines with a clear leader, and the weekly move compares the same brands one week apart:1
The share of readings within five points climbs steeply to 88% at 100 questions and keeps rising, more slowly, to 98% at 200, so the first questions you add buy far more accuracy than the last ones:
Bigger Question Sets Land Closer to the Full Reading
Readings within 5 points of the 1,000-question line, one B2B software category
View the data
| Label | Value |
|---|---|
| 10 | 33% |
| 20 | 48% |
| 25 | 53% |
| 50 | 71% |
| 100 | 88% |
| 200 | 98% |
The typical miss halves from 20 to 100 questions, 5.3 points to 2.2, and falls by about a third again by 200, to 1.5. So in this test, most of the gain in precision had arrived by about 100 questions:
The Typical Miss Shrinks as the Question Set Grows
Median gap to the full 1,000-question reading, percentage points, same category
View the data
| Label | Value |
|---|---|
| 10 questions | 8.2 pts |
| 20 questions | 5.3 pts |
| 50 questions | 3.2 pts |
| 100 questions | 2.2 pts |
| 200 questions | 1.5 pts |
Where one brand led its product line by two points or more, a 20-question draw picked that leader 82% of the time, so a small set is better at finding a clear leader than at measuring a share:
Even a Small Set Usually Finds a Clear Leader
Draws naming the line's leader, where it led by 2 points or more
View the data
| Label | Value |
|---|---|
| 10 | 66% |
| 20 | 82% |
| 50 | 95% |
| 100 | 99% |
A small set usually finds a clear leader, then, though it picked the wrong brand in nearly one draw in five. Our test did not measure close races, but a typical miss of 5.3 points per brand is far wider than a one-point gap, so a small set is no basis for ranking you against a rival that close. And a question where you were not named tells you less than it seems to:
Twenty questions show where the engines named you on those twenty questions. A question where you were not named is one to test further with more questions, and on its own it does not prove you are missing from that topic.
Vlad Shvets on LinkedInOpens in a new tabA vendor’s test on its own data shows the same shape on a different scale. OtterlyAI drew random subsets of a 320-prompt, 30-day dataset for one brand. At 10 prompts, 90% of its coverage readings fell between 39% and 71%, against 49% to 62% at 50 and 51% to 60% at 100.6
One Brand’s Coverage Band Narrows With More Prompts
Range holding 90% of one brand's coverage results, 320-prompt dataset
View the data
| Label | Range |
|---|---|
| 10 prompts | 39% to 71% |
| 50 prompts | 49% to 62% |
| 100 prompts | 51% to 60% |
OtterlyAI’s test is one brand on one vendor’s scale, so compare the way its band narrows with ours and leave its percentages aside. The band tightens fast at first and slowly after, as ours did.
Our test found most of the precision by about 100 questions, so if you have never measured your own visibility, start at that size. Our free AI Visibility Report works at it. From your website it drafts 100 questions your buyers might ask, puts every one of them to ChatGPT and to Google AI Mode, and counts the answers that name you and those that name your rivals. Treat it as a first map, and settle a close race with more questions:
How Much Daily Repeats Steady a Visibility Number
Asking the same questions every day is the usual fix for a number that jumps. Averaging more runs makes the estimate for those questions more certain, but the gain comes slowly.
A preprint on Swiss-German campaigns measured how fast repeats help. It tracked how uncertain each brand’s detection rate was, measured as a standard error. A rough 95% range is twice that number on either side of the rate. At 7 same-day runs the standard error fell below 0.10, an interval of about ±16 percentage points. With daily data it fell below 0.05 at 24 days.7
In those campaigns, seven runs in a day still left a brand’s rate uncertain by about 16 points either way, so a rate built on a handful of runs of one question is unlikely to settle a five-point change. Pooling many questions narrows that band, by an amount the preprint did not measure. And however often a question runs, it only ever measures that question:
No number of daily repeats measures a topic the question set never asks about.
In the category we tested, a brand’s figure on an unchanged set of 1,000 questions moved a median 1.4 percentage points from one week to the next, across 279 brand and product-line pairs.1 That number mixes real changes with measurement variation, and we have no matched smaller-set series beside it.
What it does show is scale. A five-point move on the full set would be several times the one-week change we saw. A 20-question reading misses the full figure by 5.3 points on its own, so the same five points on 20 questions sit inside ordinary sampling error.
Practitioners who track this weekly draw the same line between more runs and more questions:
Tips from Reddit
How practitioners split repeat runs from question count
With a yes or no outcome ten runs leaves your share uncertain by something like thirty points either way, so 8 of 10 against 6 of 10 isn’t a change you can act on.
prompt count matters more than run count. 20 prompts run 3 times tells you far more than 5 prompts run 10 times, because you’re sampling how buyers actually ask instead of sampling the model’s dice
Reruns smooth out model randomness. But variants, which are different ways someone might ask the same question, tell you if your brand shows up regardless of how it’s phrased. Both matter but they do different jobs.
How the Choice and Wording of Questions Change Your Visibility
Drawing questions at random from a fixed set does not make that set representative of everything buyers ask. In the same category, across 40 brands averaging 5% or more on 30 September, the median gap between a brand’s best and worst product line was 40 percentage points, so which product lines a set covers can change a brand’s number by tens of points.1
median gap between a brand’s best and worst product line
If your questions all come from your strongest product line, the score describes that line, and anyone who reads it as your overall visibility will overrate you.
The same split shows up for a single rival on the Empact Panel of another SaaS company we work with, in a different category. Over 90 days on ChatGPT, RingCentral led one of that company’s product lines and fell to fourth in another, more than 40 points lower. In both cards the company’s name is replaced with “Partner”, our word for the companies we work with:


Wording moves the number too. A preprint tested rewordings on OpenAI and Anthropic models, comparing the brands recommended for two phrasings of one buying question. It scored the overlap with Jaccard similarity:
Two rewordings scored 0.288, and a rewording that added a constraint 0.135, against 0.50 to 0.61 for the same question asked again.8 A reworded question shares roughly half the overlap a repeat does, so changing the wording between two reports changes the measure.
Pollsters met this problem long before AI answers, and their rule fits a question set well. The American Association for Public Opinion Research (AAPOR) asks probability surveys to report their sampling error, because each respondent had a known chance of being picked. A non-probability sample may report a precision figure only alongside how its model was specified and its assumptions validated.9
A question set a team writes is chosen by judgment, so as a whole it is a non-probability sample of all buyer questions. Drawing random subsets from it, as we did here, tells you how precise a smaller set is compared with the full one. It tells you nothing about the questions the full set never asked, and no amount of rerunning changes that:

So before a question set’s number reaches a board, check five things about the set itself:
- It covers every part of the category a buyer asks about, with enough questions in each topic
- Its questions are unbranded, in the words a buyer types
- Each engine and each market is read separately
- The wording is frozen between reports
- Its size and repeats were tested against a larger fixed set
If you are writing that set yourself, our playbook starts you at a hundred buyer questions, with a scoring sheet for the answers and a test for whether a later change in the number is real:
How to Tell Whether Your Visibility Number Can Support a Decision
Those checks are for a set you build. When someone else shows you a visibility number, ask for six things to go with it:
Compare a number only with one taken on the same engine, market, window and question set. The number of questions and runs decides how far either reading could be off, so a smaller set needs a wider margin. If a number moves less than it does in an ordinary week with nothing changed, treat the move as noise. If it moves more, it is worth looking into. The figures in this post have limits of their own:
Most of what this post found can be checked on the questions a team already tracks, so that is where we would begin with yours. We would look at which parts of your category they cover, whether any of them name you, and whether their size holds up against a larger set.
You would see which of your numbers rest on enough questions to compare from week to week, and which product lines the engines barely name you in. Getting you named in those lines is the rest of our work on AI search visibility, and it covers:
- An audit and a roadmap for the work
- Your own pages, made easy for ChatGPT, Google AI Mode and Claude to read and quote
- Mentions on the listicles and other pages those answers cite
- Threads on Reddit where people who use your product recommend it
- Standing reports on visibility, citations and Reddit, each recommendation with its first step
Frequently Asked Questions
How many questions do we need to measure AI visibility?
No study sets a universal minimum. In our Panel data, random readings landed within five points of the 1,000-question figure 48% of the time at 20 questions, 88% at 100 and 98% at 200. Decide what the number has to support, then test smaller question sets against a larger fixed one in your own category before you settle on a size. Our own playbook starts at 100 buyer questions, which is also about where most of the precision in our tests had arrived.
Is a 20-question check worth running at all?
Yes, as a first reading of where the engines name you on those questions. In our Panel data, where one brand led by two points or more, 82% of 20-question draws picked that leader.
What such a check cannot settle is a close race, because its typical miss of 5.3 points is wider than a gap of a point or two. A question where you were not named is a reason to test further, not proof that you are absent from the topic. Use it to choose where a larger set of questions should look first, and which topics need more coverage.
How many times should we run each question?
Runs help less than they seem to. In the Swiss-German preprint, seven same-day runs still left a brand’s rate uncertain by about 16 points either way, and the standard error only fell below 0.05 after 24 days of daily data. Our view is to add questions before adding runs, then repeat the larger set on a fixed schedule so the readings can be compared. Spreading the runs across days also catches engine changes that a single session would miss.
Can we trust a five-point increase in our AI visibility?
Check it against the spread you see when nothing changed. Across our 1,000-question sets, a brand’s figure moved a median 1.4 points a week, so five points there deserves a look. On 20 questions, where a reading typically misses the full figure by 5.3 points, the same increase can be pure chance. Before anyone reports it as progress, rerun the larger set and check that the engine, market and wording all match.
Does asking the same questions every day fix a narrow question set?
No. Daily runs steady the estimate for the questions in the set and do nothing for the questions left out of it. A narrow set run every day gives a precise reading of a narrow slice of your market, which is why coverage comes first. Add questions for the topics your sales team hears before you add more runs of the ones you already ask.
Should our tracking questions include our brand name?
Keep them unbranded. Questions in the buyer’s own words, naming no vendor, measure whether the engines recommend you to someone who has not decided yet. Questions with your name in them test something else, whether the engine can describe you once asked, so track the two separately. A rise in branded readings tells you little about whether new buyers are being sent your way.
How much does rewording a question change the brands AI recommends?
A lot. In the paraphrase preprint, two rewordings of one buying question shared brands at a Jaccard similarity of 0.288, and 0.135 when the rewording added a constraint, against 0.50 to 0.61 for the same question asked twice. Freeze the wording between reports, or the comparison measures your edits.
If a question has to change, start a new series and keep the old one running for a few weeks beside it. Running both on the same weeks lets you compare the two series, though some of the difference will still be ordinary week-to-week variation.
Why does our visibility differ so much between product lines?
Because each part of a category draws its own set of rivals and sources. In our Panel data the median gap between a brand’s best and worst product line was 40 points. Measure each part of the category separately, and never let your strongest line stand in for the whole company. A report that averages every line together can hide a line where rivals dominate.
Is a 1,000-question panel the true picture of our market?
No. It is a reference set that people chose, large enough to test smaller sets against and to track change on the same questions. It shares the blind spots of whoever wrote it, so its coverage of buyer topics deserves the same scrutiny as its size. Review the topics it covers each quarter against what your sales and support teams hear.
What should an AI visibility report disclose beside the number?
Enough to tell whether two numbers are comparable. A different engine, market, window or question set means they measure different things. Drawn from one fixed set with only the number of questions or runs changed, they measure the same thing at different precision. AAPOR asks a non-probability sample to say how it was built before quoting precision, and a hand-written question set is one, so its selection method belongs beside the number.
Sources (9)
- Empact Partners, "The Empact Panel, a B2B software category we measure for a partner" (2026)Empact Partners' own data↩1↩2↩3↩4↩5↩6↩7↩8↩9↩10Unbranded buyer questions, 1,000 per product line, ChatGPT and Google AI Mode, 23 and 30 September 2026; the partner’s brand excluded; random subsets against the full line. Re-read 12 November 2026.
- SparkToro, "NEW Research: AIs are highly inconsistent when recommending brands or products; marketers should take care when tracking AI visibility" (2026)↩1↩2600 volunteers, 12 prompts, 2,961 runs. Read 12 November 2026.
- Search Engine Land, "What repeated ChatGPT runs reveal about brand visibility" (2026)↩1↩2↩3Mike Sonders; 12 B2B software prompts, 100 runs each. Read 12 November 2026.
- Detailed.com, "September 2026: I Compared Prompt Tracking Across 8 AI Visibility Tools" (2026)↩1↩21,300+ prompts daily, four weeks. Read 12 November 2026.
- SE Ranking, "AI Mode research: Volatility, source patterns, and differences from AIO and organic results" (2025)↩110,000 keywords, June 2025. Read 12 November 2026.
- OtterlyAI, "AI Search Visibility: How Stable Are Brand Mentions and Citations?" (2026)↩1↩2↩3One brand, a 320-prompt, 30-day dataset. Read 12 November 2026.
- Schulte, Bleeker and Kaufmann, "Don’t Measure Once: Measuring Visibility in AI Search (GEO)" (2026)↩1arXiv preprint 2604.07585. Read 12 November 2026.
- Jack, Lehman, Maloney and Xu, "Paraphrase Brittleness in Production Retrieval-Augmented Commercial Recommendation" (2026)↩1↩2arXiv preprint 2605.27440. Read 12 November 2026.
- American Association for Public Opinion Research, "Disclosure Standards" (2021)↩1The AAPOR Code, revised April 2021. Read 12 November 2026.
Keep reading
Statistics 45 SaaS SEO Statistics
45 figures on what organic search is worth to a SaaS company, and where AI answers about software get their sources.
Statistics 45 Gemini Citation Statistics for B2B Software Teams
Gemini citation data for SaaS teams on which pages it cites, how its answers shift and how to count your category before funding work.
Statistics 13 AI Answer Brand Mention Statistics: What Shortlists Show
How many brands appear in AI answers, what buyer surveys show, and how to measure your SaaS company's visibility.


