Skip to content

12 AI Visibility Sample Size Statistics for SaaS

How question count, repeated runs and wording affect AI visibility reports, and when a change is worth acting on.

On this page
  1. Highlights
  2. Why Repeating One Question Gives Different Brands
  3. How Accurate a 20-Question Visibility Check Is
  4. How Much Daily Repeats Steady a Visibility Number
  5. How the Choice and Wording of Questions Change Your Visibility
  6. How to Tell Whether Your Visibility Number Can Support a Decision
  7. Frequently Asked Questions

Empact Newsletter

One email a month on what is working in AI search and SaaS go-to-market.

Check your email for a link to confirm your subscription.

You are already subscribed.

Check your email address and try again.

We could not save your subscription just now; please try again.

In one B2B software category, an AI visibility score built from 20 buyer questions drawn at random from a set of 1,000 landed within five percentage points of the full set’s score in about 48% of draws. At 200 questions it did in about 98%.1

The score is the share of AI answers that name a brand, so on 20 questions a single answer moves it by five points. Two quick checks can disagree while nothing in the market changed, and moving budget on that gap means moving it on luck. A handful of questions picked by hand adds a second problem, because it can leave out whole parts of what buyers ask.

The draws above come from questions Empact Partners asks AI engines each week for a SaaS company we work with. We get SaaS products recommended in AI answers, and a score that jumps on its own would hide whether that work did anything.

Highlights

  • 5.3 points was how far a 20-question reading typically landed from the full 1,000-question score in one B2B software category we measure, and 1.5 points at 200. A quick check’s number can easily sit five points from the full set’s score.1
  • Fewer than 1 in 100 pairs of ChatGPT or Google AI answers to the same brand-recommendation prompt named the same set of brands, so one answer is one draw from a larger pool.2
  • About 5 brands on average came back in 80% or more of 100 ChatGPT answers to a B2B software prompt, out of about 44 named at least once. A brand outside that handful shows up in some answers and not in others.3
  • 41% of ChatGPT prompts that named brands regularly never gave an identical brand set on any two days in four weeks of daily tracking, so the brands named change from one day to the next as a matter of course.4
  • 39% to 71%: the range covering nine in ten of one brand’s readings at 10 prompts in OtterlyAI’s test, against 51% to 60% at 100, so the band tightens fast as prompts are added.6
  • 0.288 was the overlap, on a scale where 1 means the same brands, between the brands recommended for two rewordings of one buying question. The same question asked again scored 0.50 to 0.61, so changing the wording changes what you measure.8

Why Repeating One Question Gives Different Brands

Every visibility number starts with one question asked once, and an AI engine writes a fresh answer each time. Four outside tests asked the same question again and again, and three of them measured how often two answers match:

  • Fewer than 1 in 100 pairs of responses from ChatGPT or Google’s AI to the same brand-recommendation prompt named the same brands, and matching order was nearer one pair in a thousand. SparkToro had 600 volunteers run 12 prompts 2,961 times.2
  • 0.3% of day pairs gave an identical ChatGPT brand set, and 41% of prompts never repeated a set in four weeks, in Detailed.com’s daily tracking of 1,300+ prompts that named a brand on at least a quarter of days.4
  • 9.2% of cited URLs matched across three same-day runs of 10,000 searches in Google AI Mode, in SE Ranking’s June 2025 test. It counted cited pages rather than brands, so the pages one run cites are a weak guide to the next.5

The fourth, by Mike Sonders, ran 12 B2B software prompts 100 times each on ChatGPT and counted the brands across the answers:3

Brands Across 100 ChatGPT Answers to One B2B Software Prompt

Average across 12 prompts run 100 times each, Mike Sonders, Search Engine Land, 2026

View the data
LabelValue
Named at all44
In one answer10
In 80%+ of answers5
One answer shows about 10 brands, but 100 runs of the same prompt surface about 44.Source: Search Engine Land

Across 100 runs of one B2B software prompt, ChatGPT named about 44 brands on average, while any single answer shows about 10. Only about five of them came back in most answers. A brand outside that handful appears in some runs and not others, so one answer says little about how often it gets named.

More runs of the same prompt estimate that rate better, and more questions cover more of what buyers ask, which are two different fixes. A swing between two checks, the kind teams read as AI search volatility, can be nothing more than a fresh draw. Vlad Shvets, our founder, turns that into a rule for anyone who reports the number:

Report a rate across many questions and many runs, and treat the first brand an answer names with the same caution, because the first name changes from one run to the next as well.

Vlad ShvetsFounder & CEO @ Empact PartnersVlad Shvets on LinkedIn

How Accurate a 20-Question Visibility Check Is

To see how far a 20-question check can be trusted, we drew sets of 10 to 200 questions from those weekly answers. They come from the Empact Panel, our own software for measuring AI search visibility. It saves every answer ChatGPT, Google AI Mode and Claude give to the buyer questions of each SaaS company we work with, so the full set’s score is always there to check a smaller draw against.

For the company in our opening example, the Panel holds 1,000 unbranded buyer questions per product line. We took the ChatGPT and Google AI Mode answers from 23 and 30 September 2026 and left out the company’s own brand, so every figure describes the other brands in its category. A reading is a brand’s share of answers that name it on a random subset of those questions, set against its share on all 1,000.

The first two figures are medians across every brand named in 5% or more of a product line’s answers, with both engines and both weeks counted together. The leader figure counts only product lines with a clear leader, and the weekly move compares the same brands one week apart:1

48%of 20-question readings landed within five points of the full 1,000
98%of 200-question readings did
82%of 20-question draws named the right leader where one led by two points or more
1.4 pointswas a brand’s median weekly move on the full 1,000 questions
Source: Empact Partners

The share of readings within five points climbs steeply to 88% at 100 questions and keeps rising, more slowly, to 98% at 200, so the first questions you add buy far more accuracy than the last ones:

Bigger Question Sets Land Closer to the Full Reading

Readings within 5 points of the 1,000-question line, one B2B software category

Within 5 points of the full line
View the data
LabelValue
1033%
2048%
2553%
5071%
10088%
20098%
At 20 questions, fewer than half the readings land within five points of the full 1,000-question figure.Source: Empact Partners

The typical miss halves from 20 to 100 questions, 5.3 points to 2.2, and falls by about a third again by 200, to 1.5. So in this test, most of the gain in precision had arrived by about 100 questions:

The Typical Miss Shrinks as the Question Set Grows

Median gap to the full 1,000-question reading, percentage points, same category

View the data
LabelValue
10 questions8.2 pts
20 questions5.3 pts
50 questions3.2 pts
100 questions2.2 pts
200 questions1.5 pts
A 10-question set typically misses the full reading by 8.2 points, and the miss shrinks with every step up in size.Source: Empact Partners

Where one brand led its product line by two points or more, a 20-question draw picked that leader 82% of the time, so a small set is better at finding a clear leader than at measuring a share:

Even a Small Set Usually Finds a Clear Leader

Draws naming the line's leader, where it led by 2 points or more

View the data
LabelValue
1066%
2082%
5095%
10099%
By 50 questions the clear leader came out on top in 95% of draws. Brands closer than two points apart are outside this test.Source: Empact Partners

A small set usually finds a clear leader, then, though it picked the wrong brand in nearly one draw in five. Our test did not measure close races, but a typical miss of 5.3 points per brand is far wider than a one-point gap, so a small set is no basis for ranking you against a rival that close. And a question where you were not named tells you less than it seems to:

Twenty questions show where the engines named you on those twenty questions. A question where you were not named is one to test further with more questions, and on its own it does not prove you are missing from that topic.

Vlad ShvetsFounder & CEO @ Empact PartnersVlad Shvets on LinkedIn

A vendor’s test on its own data shows the same shape on a different scale. OtterlyAI drew random subsets of a 320-prompt, 30-day dataset for one brand. At 10 prompts, 90% of its coverage readings fell between 39% and 71%, against 49% to 62% at 50 and 51% to 60% at 100.6

One Brand’s Coverage Band Narrows With More Prompts

Range holding 90% of one brand's coverage results, 320-prompt dataset

View the data
LabelRange
10 prompts39% to 71%
50 prompts49% to 62%
100 prompts51% to 60%
Ten prompts leave a band so wide that one brand’s coverage depends heavily on which prompts were drawn.Source: OtterlyAI

OtterlyAI’s test is one brand on one vendor’s scale, so compare the way its band narrows with ours and leave its percentages aside. The band tightens fast at first and slowly after, as ours did.

Our test found most of the precision by about 100 questions, so if you have never measured your own visibility, start at that size. Our free AI Visibility Report works at it. From your website it drafts 100 questions your buyers might ask, puts every one of them to ChatGPT and to Google AI Mode, and counts the answers that name you and those that name your rivals. Treat it as a first map, and settle a close race with more questions:

How Much Daily Repeats Steady a Visibility Number

Asking the same questions every day is the usual fix for a number that jumps. Averaging more runs makes the estimate for those questions more certain, but the gain comes slowly.

A preprint on Swiss-German campaigns measured how fast repeats help. It tracked how uncertain each brand’s detection rate was, measured as a standard error. A rough 95% range is twice that number on either side of the rate. At 7 same-day runs the standard error fell below 0.10, an interval of about ±16 percentage points. With daily data it fell below 0.05 at 24 days.7

In those campaigns, seven runs in a day still left a brand’s rate uncertain by about 16 points either way, so a rate built on a handful of runs of one question is unlikely to settle a five-point change. Pooling many questions narrows that band, by an amount the preprint did not measure. And however often a question runs, it only ever measures that question:

No number of daily repeats measures a topic the question set never asks about.

In the category we tested, a brand’s figure on an unchanged set of 1,000 questions moved a median 1.4 percentage points from one week to the next, across 279 brand and product-line pairs.1 That number mixes real changes with measurement variation, and we have no matched smaller-set series beside it.

What it does show is scale. A five-point move on the full set would be several times the one-week change we saw. A 20-question reading misses the full figure by 5.3 points on its own, so the same five points on 20 questions sit inside ordinary sampling error.

Practitioners who track this weekly draw the same line between more runs and more questions:

Tips from Reddit

How practitioners split repeat runs from question count

With a yes or no outcome ten runs leaves your share uncertain by something like thirty points either way, so 8 of 10 against 6 of 10 isn’t a change you can act on.

r/AISEOforBeginners, u/StatedMethod, September 2026

prompt count matters more than run count. 20 prompts run 3 times tells you far more than 5 prompts run 10 times, because you’re sampling how buyers actually ask instead of sampling the model’s dice

r/AISEOforBeginners, u/ChapterAfter8368, September 2026

Reruns smooth out model randomness. But variants, which are different ways someone might ask the same question, tell you if your brand shows up regardless of how it’s phrased. Both matter but they do different jobs.

r/aeo, u/Intrepid_Spell_8711, May 2026

How the Choice and Wording of Questions Change Your Visibility

Drawing questions at random from a fixed set does not make that set representative of everything buyers ask. In the same category, across 40 brands averaging 5% or more on 30 September, the median gap between a brand’s best and worst product line was 40 percentage points, so which product lines a set covers can change a brand’s number by tens of points.1

40 points

median gap between a brand’s best and worst product line

Source: Empact Partners

If your questions all come from your strongest product line, the score describes that line, and anyone who reads it as your overall visibility will overrate you.

The same split shows up for a single rival on the Empact Panel of another SaaS company we work with, in a different category. Over 90 days on ChatGPT, RingCentral led one of that company’s product lines and fell to fourth in another, more than 40 points lower. In both cards the company’s name is replaced with “Partner”, our word for the companies we work with:

Competitors Ranking card on the Empact Panel for one product line on ChatGPT over 90 days: RingCentral first at 62.4% visibility, the partner sixth at 12.8%
Empact Panel example: in one of the partner’s product lines, RingCentral leads ChatGPT’s answers with 62.4% visibility and the partner sits sixth.Empact Panel
The same card for the partner’s second product line on ChatGPT over 90 days: RingCentral fourth at 19.9% visibility, behind Microsoft, 1Stream and the partner
Empact Panel example: in the partner’s second product line, RingCentral drops to fourth at 19.9%, behind Microsoft, 1Stream and the partner.Empact Panel

Wording moves the number too. A preprint tested rewordings on OpenAI and Anthropic models, comparing the brands recommended for two phrasings of one buying question. It scored the overlap with Jaccard similarity:

Two rewordings scored 0.288, and a rewording that added a constraint 0.135, against 0.50 to 0.61 for the same question asked again.8 A reworded question shares roughly half the overlap a repeat does, so changing the wording between two reports changes the measure.

Pollsters met this problem long before AI answers, and their rule fits a question set well. The American Association for Public Opinion Research (AAPOR) asks probability surveys to report their sampling error, because each respondent had a known chance of being picked. A non-probability sample may report a precision figure only alongside how its model was specified and its assumptions validated.9

A question set a team writes is chosen by judgment, so as a whole it is a non-probability sample of all buyer questions. Drawing random subsets from it, as we did here, tells you how precise a smaller set is compared with the full one. It tells you nothing about the questions the full set never asked, and no amount of rerunning changes that:

Four steps: chosen buyer topics become a fixed question set, the answers are run again and again, and a report follows. A topic left out of the set never reaches the report.
More runs sharpen the number for the topics the set asks about and do nothing for the rest.

So before a question set’s number reaches a board, check five things about the set itself:

  • It covers every part of the category a buyer asks about, with enough questions in each topic
  • Its questions are unbranded, in the words a buyer types
  • Each engine and each market is read separately
  • The wording is frozen between reports
  • Its size and repeats were tested against a larger fixed set

If you are writing that set yourself, our playbook starts you at a hundred buyer questions, with a scoring sheet for the answers and a test for whether a later change in the number is real:

How to Tell Whether Your Visibility Number Can Support a Decision

Those checks are for a set you build. When someone else shows you a visibility number, ask for six things to go with it:

Compare a number only with one taken on the same engine, market, window and question set. The number of questions and runs decides how far either reading could be off, so a smaller set needs a wider margin. If a number moves less than it does in an ordinary week with nothing changed, treat the move as noise. If it moves more, it is worth looking into. The figures in this post have limits of their own:

Most of what this post found can be checked on the questions a team already tracks, so that is where we would begin with yours. We would look at which parts of your category they cover, whether any of them name you, and whether their size holds up against a larger set.

You would see which of your numbers rest on enough questions to compare from week to week, and which product lines the engines barely name you in. Getting you named in those lines is the rest of our work on AI search visibility, and it covers:

  • An audit and a roadmap for the work
  • Your own pages, made easy for ChatGPT, Google AI Mode and Claude to read and quote
  • Mentions on the listicles and other pages those answers cite
  • Threads on Reddit where people who use your product recommend it
  • Standing reports on visibility, citations and Reddit, each recommendation with its first step

Frequently Asked Questions

How many questions do we need to measure AI visibility?

No study sets a universal minimum. In our Panel data, random readings landed within five points of the 1,000-question figure 48% of the time at 20 questions, 88% at 100 and 98% at 200. Decide what the number has to support, then test smaller question sets against a larger fixed one in your own category before you settle on a size. Our own playbook starts at 100 buyer questions, which is also about where most of the precision in our tests had arrived.

Is a 20-question check worth running at all?

Yes, as a first reading of where the engines name you on those questions. In our Panel data, where one brand led by two points or more, 82% of 20-question draws picked that leader.

What such a check cannot settle is a close race, because its typical miss of 5.3 points is wider than a gap of a point or two. A question where you were not named is a reason to test further, not proof that you are absent from the topic. Use it to choose where a larger set of questions should look first, and which topics need more coverage.

How many times should we run each question?

Runs help less than they seem to. In the Swiss-German preprint, seven same-day runs still left a brand’s rate uncertain by about 16 points either way, and the standard error only fell below 0.05 after 24 days of daily data. Our view is to add questions before adding runs, then repeat the larger set on a fixed schedule so the readings can be compared. Spreading the runs across days also catches engine changes that a single session would miss.

Can we trust a five-point increase in our AI visibility?

Check it against the spread you see when nothing changed. Across our 1,000-question sets, a brand’s figure moved a median 1.4 points a week, so five points there deserves a look. On 20 questions, where a reading typically misses the full figure by 5.3 points, the same increase can be pure chance. Before anyone reports it as progress, rerun the larger set and check that the engine, market and wording all match.

Does asking the same questions every day fix a narrow question set?

No. Daily runs steady the estimate for the questions in the set and do nothing for the questions left out of it. A narrow set run every day gives a precise reading of a narrow slice of your market, which is why coverage comes first. Add questions for the topics your sales team hears before you add more runs of the ones you already ask.

Should our tracking questions include our brand name?

Keep them unbranded. Questions in the buyer’s own words, naming no vendor, measure whether the engines recommend you to someone who has not decided yet. Questions with your name in them test something else, whether the engine can describe you once asked, so track the two separately. A rise in branded readings tells you little about whether new buyers are being sent your way.

How much does rewording a question change the brands AI recommends?

A lot. In the paraphrase preprint, two rewordings of one buying question shared brands at a Jaccard similarity of 0.288, and 0.135 when the rewording added a constraint, against 0.50 to 0.61 for the same question asked twice. Freeze the wording between reports, or the comparison measures your edits.

If a question has to change, start a new series and keep the old one running for a few weeks beside it. Running both on the same weeks lets you compare the two series, though some of the difference will still be ordinary week-to-week variation.

Why does our visibility differ so much between product lines?

Because each part of a category draws its own set of rivals and sources. In our Panel data the median gap between a brand’s best and worst product line was 40 points. Measure each part of the category separately, and never let your strongest line stand in for the whole company. A report that averages every line together can hide a line where rivals dominate.

Is a 1,000-question panel the true picture of our market?

No. It is a reference set that people chose, large enough to test smaller sets against and to track change on the same questions. It shares the blind spots of whoever wrote it, so its coverage of buyer topics deserves the same scrutiny as its size. Review the topics it covers each quarter against what your sales and support teams hear.

What should an AI visibility report disclose beside the number?

Enough to tell whether two numbers are comparable. A different engine, market, window or question set means they measure different things. Drawn from one fixed set with only the number of questions or runs changed, they measure the same thing at different precision. AAPOR asks a non-probability sample to say how it was built before quoting precision, and a hand-written question set is one, so its selection method belongs beside the number.

Sources (9)
  1. Empact Partners, "The Empact Panel, a B2B software category we measure for a partner" (2026)Empact Partners' own data↩1↩2↩3↩4↩5↩6↩7↩8↩9↩10Unbranded buyer questions, 1,000 per product line, ChatGPT and Google AI Mode, 23 and 30 September 2026; the partner’s brand excluded; random subsets against the full line. Re-read 12 November 2026.
  2. SparkToro, "NEW Research: AIs are highly inconsistent when recommending brands or products; marketers should take care when tracking AI visibility" (2026)↩1↩2600 volunteers, 12 prompts, 2,961 runs. Read 12 November 2026.
  3. Search Engine Land, "What repeated ChatGPT runs reveal about brand visibility" (2026)↩1↩2↩3Mike Sonders; 12 B2B software prompts, 100 runs each. Read 12 November 2026.
  4. Detailed.com, "September 2026: I Compared Prompt Tracking Across 8 AI Visibility Tools" (2026)↩1↩21,300+ prompts daily, four weeks. Read 12 November 2026.
  5. SE Ranking, "AI Mode research: Volatility, source patterns, and differences from AIO and organic results" (2025)↩110,000 keywords, June 2025. Read 12 November 2026.
  6. OtterlyAI, "AI Search Visibility: How Stable Are Brand Mentions and Citations?" (2026)↩1↩2↩3One brand, a 320-prompt, 30-day dataset. Read 12 November 2026.
  7. Schulte, Bleeker and Kaufmann, "Don’t Measure Once: Measuring Visibility in AI Search (GEO)" (2026)↩1arXiv preprint 2604.07585. Read 12 November 2026.
  8. Jack, Lehman, Maloney and Xu, "Paraphrase Brittleness in Production Retrieval-Augmented Commercial Recommendation" (2026)↩1↩2arXiv preprint 2605.27440. Read 12 November 2026.
  9. American Association for Public Opinion Research, "Disclosure Standards" (2021)↩1The AAPOR Code, revised April 2021. Read 12 November 2026.

Ready
To Connect?

Let's Partner