How to Write a Buyer Question Set for AI Visibility Tracking
Your AI visibility number is only as honest as its questions. Draft them in Claude from your own pages, then cut the ones that flatter you.
On this page
Empact Newsletter
One email a month on what is working in AI search and SaaS go-to-market.
Your AI visibility tracker says ChatGPT names you in close to half of your buyers’ questions, and the board slide is half written. Before it goes, open the questions behind the number. Whether sales picked them or a model wrote them from your homepage, they lean toward what you already sell well, and the number ends up reporting how well the engines know your pitch.
Empact Partners grows how often ChatGPT, Google AI Mode and Claude recommend SaaS products, and we track that work every week on each partner’s own buyer questions. Every number we report rests on those questions.
We ran the job on 17 November 2026 on the public pages of Umami, an open-source analytics product with no ties to us, and the topics alone moved its share of answers from 10% to 83%. You finish with a CSV (comma-separated values) file of about forty brand-free buyer questions, tagged by topic, persona and market. Ask for the support or sales questions first, since someone else has to send them:
This page builds the question set. Running it and counting the answers is what the baseline sheet does next.
Drafting Sixty Questions From Your Own Material
Step 1: Collect three kinds of your own material
Each kind of material gives the draft something different. Use-case pages say which jobs the product does, support or sales questions carry the words people use when they are stuck, and reviews say what buyers compared you with. Paste them as text into three files, with images and links stripped. For Umami that came to this:
Scroll horizontally to see all columns when the table is wider than this page.
| Material | What we pasted | What it gave the draft |
|---|---|---|
| Use-case pages | Umami’s homepage, its product analytics page and its platform page | The topics, and almost every question Step 4 cuts |
| Support questions | Its documentation FAQ, its cloud FAQ and the 50 newest questions in its public GitHub forum | The privacy and data-region questions. The forum gave nearly nothing |
| Review language | Its six G2 reviews | The ad-blocker worry, “too complicated”, and the name of the category leader |
The forum gave the draft almost nothing, and the reason matters. All fifty questions were about setting the product up, and a setup question is not a buying question, so Claude rightly left them alone. If your product is technical, take the questions sales hears on first calls instead of the support queue.
Step 2: Paste the prompt into Claude
Start a fresh conversation in Claude with the prompt below. Edit the “My category” and “My buyers” lines for your own product, and put each file from Step 1 where its bracketed line sits.
I want to measure how often AI assistants such as ChatGPT, Google AI Mode and Claude recommend products in my category when nobody names a brand. Write the buyer questions for that measurement from the material below.
My category: open-source, privacy-friendly web and product analytics, offered as a cloud service or self-hosted
My buyers: founders, marketers and developers at small and mid-sized software companies
The material has three parts: our use-case pages, the questions our users ask support, and what reviewers write about products like ours. Use it to find the jobs buyers are trying to get done and the words they use for them.
Write 60 questions a buyer would type into an AI assistant while choosing a product.
- Group them into 8 to 12 topics. A topic is one buying need. Write 4 to 6 questions per topic that ask for the same thing in different words, from different people.
- Every question is one full sentence of the kind a person types into a chat, with the context a real buyer adds: their role, their company, what they need, the kind of tool they use today.
- A good answer to every question names products to choose from. Skip questions about how to set up or use a product.
- Never name a company, product or brand, mine included. Describe a tool by its type, never by its name.
- Never compare two options. Never use the words alternative, alternatives, vs, versus or compared to.
Return CSV with this header: topic,persona,market,question
- topic: a short name for the buying need
- persona: who is asking, in a few words
- market: the country or region the asker names, or global if they name none
Put every field in double quotes. Return the CSV and nothing else.
--- OUR USE-CASE PAGES ---
[Paste your use-case pages here]
--- QUESTIONS OUR USERS ASK SUPPORT ---
[Paste your support or sales questions here]
--- WHAT REVIEWERS WRITE ---
[Paste your review pages here]Claude answered in 47 seconds with a CSV of sixty questions in ten topics. Every rule written into the prompt held. What the rules could not catch shows in the last two numbers:
No brand and no comparison are the rules most published guides stop at, and Claude kept both on every row. The bias left to cut by hand is the one the prompt never named, and Steps 4 and 5 deal with it.
Claude kept every rule we wrote down and still wrote a third of the set from the vendor’s own pages.
Cutting the Questions That Measure the Wrong Thing
Step 3: Run four checks on every row
Paste the CSV into a sheet, add a column called Verdict, and mark every row keep, rewrite or cut against four checks. Claude passed all four on all sixty rows, and you still run them, because questions added later need the same checks:
A row stays only if
- It names no company, product or brand, yours included.
- It compares nothing and asks for no alternative to anything.
- It reads in full sentences, the way a buyer types to an engine, with their context in it.
- A good answer to it names products to choose from.
The second check matters most when a tracker is already full of comparisons. To see what that shape does, we wrote sixteen questions asking for alternatives to Google Analytics, two for each topic, and ran them beside the kept set on ChatGPT and Google AI Mode, three runs each.1
The Brand a Question Names Shows Up in Nearly Every Answer
Share of answers naming each brand, 432 ChatGPT and Google AI Mode answers, 17 Nov 2026
View the data
| Label | Umami | Google Analytics |
|---|---|---|
| Open buyer questions | 43% | 61% |
| Echo questions | 26% | 69% |
| Alternatives to Google Analytics | 47% | 98% |
Google Analytics appeared in 94 of 96 answers to those questions, against 61% of the answers to the open ones. A question that names a brand hands the engine its first name, which is what our query set composition test found for comparisons too.
If the brand a question names is yours, expect the same as Google Analytics got here: a mention in nearly every answer your tracker collects. Umami, the challenger such a question invites, rose only from 43% to 47%. Two rows went for a reason no check covers: the asker was building a site for someone else, so the question measured a reseller rather than a buyer.
Step 4: Strike the questions your own pages wrote
Then read every row for the fault Claude added without being asked. We call it an echo question: a buying question written in your own page’s words, about your own feature. To test a row, copy its most specific phrase and search your site for it.
Scroll horizontally to see all columns when the table is wider than this page.
| Drafted question (shortened) | What Umami’s own page says | Answers naming Umami |
|---|---|---|
| “Which analytics platforms offer an official MCP server so I can ask Claude or ChatGPT questions…” | “Umami’s official MCP server connects Claude, ChatGPT…” | 0 of 6 |
| “I’m paying for a traffic tool, a heatmap tool and a product analytics tool, so is there a single platform…” | “Most teams stitch together a traffic tool, a heatmap tool, and a product analytics tool…” | 3 of 6 |
| “I want to attribute payments and refunds… without building my own data pipeline” | “…without building your own payment data pipeline.” | 1 of 6 |
| “…open-source analytics we can audit, self-host, and also fall back to a managed cloud version” | “Self-host the open-source platform on your own infrastructure, or use Umami Cloud.” | 5 of 6 |
| “…web analytics platforms that never sell visitor data to advertisers” | “…no data sold to advertisers.” | 5 of 6 |
Twenty of the sixty drafts were echoes, and their counts followed Umami’s marketing calendar. The three that repeated its long-standing pitch, privacy and open source with a cloud plan, named it in 14 of 18 answers. The four that asked only about the assistant integration it announced on 10 September 2026, ten weeks before our run, named it in none of 24.
Either way, those rows measured how the engines respond to Umami’s own words, which is a different thing from what a buyer with no brand in mind gets told. What separates the two kinds of question is where the words came from:

Read the two sides as a rule for the set: a question in a buyer’s words leaves every product in the category in play, and one in your words narrows it to whoever has that feature.
Strike the echoes, or rewrite the ones that hide a real need. One of ours asked for session replay “with form inputs masked by default”, six words from Umami’s own page. The need underneath is real, so it became “Which tools record user sessions without capturing what people type into forms?”
Track a launch you care about in a separate set, on its own line, where it can tell you whether the engines have caught up. The reason to be strict about it fits in two sentences:
An echo question asks the engine whether it has read your homepage. Buyers do not shop in your feature names, so a set built from them reports on your release notes and says nothing about whether a stranger with your problem hears your name.
Gabby Saayman on LinkedInOpens in a new tabTelling the two kinds apart is where Generative Engine Optimization needs judgment rather than rules, so it is the cut we would make first on your set. A question about a real buying need can still be phrased in your homepage’s words, and the fix is often a rewrite rather than a strike. Once the set is cut, we put it to the engines every week and work on the pages and threads its answers cite.
Step 5: Give every topic four or more wordings
Which topics go in decides the headline before a single question runs. In the kept set, the cookieless and self-hosting topics named Umami in 83% of their answers, and the funnel and attribution topics in 10%.1 A set built only from the first two would report Umami in most answers. One built from the last two would report it in about one answer in ten. The kept US questions together landed at 43%.
So choose topics from the needs buyers bring to sales calls and reviews, never from the places you know you win. Give each buying need one topic and each topic a similar number of wordings, and report every topic’s share beside the total, so the board sees which needs carry the number. Inside each topic, one wording is not enough either:
How Far a Single Wording Moved Each Topic’s Share
Open questions, least and most favorable wording, 6 answers each, 17 Nov 2026
View the data
| Label | Range |
|---|---|
| Cookieless analytics | 67% to 100% |
| Self-hosted analytics | 50% to 100% |
| Free and affordable plans | 17% to 100% |
| Leaving a complex tool | 17% to 100% |
| Data residency | 0% to 50% |
| Replays and heatmaps | 0% to 50% |
| Funnels and retention | 0% to 33% |
| Attribution and revenue | 0% to 50% |
In the topic about leaving a complex tool, one wording named Umami in all six answers and another in one. So each topic carries at least four wordings, written for different people asking for the same thing. Ours ended with five to seven.
Watch the endings too. Twenty-five of Claude’s sixty drafts ended on a “…, so which…” or “…, so what…” clause. That is harmless when the rest of each sentence differs. Where two wordings differ only in that clause, rewrite one of them.
Practitioners who track this weekly reach the same place:
Tips from Reddit
How practitioners keep a question set honest
I keep a fixed list of prompts grouped under topics, with 4-5 phrasings per topic so one quirky wording doesn't skew the result. The list stays fixed so the trend actually means something.
We split ours by intent first and then froze the important ones. Adding new prompts is fine but changing the original set every month makes the trend data kind of meaningless.
Wordings and runs solve different problems. Even with the wording held still, 39 of the 144 question-and-engine pairs we ran named Umami in some runs and not in others. One run per question is never a sample size you can report upward.
Tagging, Exporting and Freezing the Set
Step 6: Fill the persona and market columns yourself
Claude filled both columns, and neither came back in a form a tracker can report on. The persona column held sixty different descriptions for sixty rows. The market column scattered fifteen countries across twenty-two rows, most of them with one question each, and one question tells you nothing about a country.
Scroll horizontally to see all columns when the table is wider than this page.
| Column | Claude’s draft | After the cut |
|---|---|---|
| persona | 60 descriptions, one per row | 6 roles: founder, marketing lead, developer, CTO, product manager, compliance lead |
| market | Global, plus 15 countries on 22 rows | US for the main run, and Germany, the UK, France, Austria or Ireland only where the question needs the country |
The market column says where a row is asked from, and a market you report on gets its whole set. A question that names a country because the law there is the reason for asking, a German consent banner or an EU data region, belongs to that country and waits there until you run that market in full.
Step 7: Export the set as a CSV
Our cut left 43 questions in eight topics. Twenty echoes and two questions from people buying for someone else came out, four were rewritten, and five were written by hand where a topic came out short. Thirty-six run in the US, and seven wait for their own markets. Export the sheet as CSV with these four columns. Eight of our rows, one per topic, look like this:
"topic","persona","market","question"
"Cookieless analytics","developer","US","I want to track visitors on my SaaS landing page without cookies or fingerprinting, so which lightweight analytics tools should I consider?"
"Self-hosted analytics","CTO","US","Our security policy says visitor data can't leave our infrastructure, so which open-source analytics platforms can we run ourselves with Docker?"
"Funnels and retention","founder","US","What is the easiest product analytics tool for a non-technical founder to understand signups, activation and returning users?"
"Attribution and revenue","marketing lead","US","Which analytics tools automatically track UTM parameters and show which channels and campaigns drive the most signups?"
"Replays and heatmaps","developer","US","Which open-source analytics projects include session replay that I can host myself?"
"Leaving a complex tool","marketing lead","US","Our current free analytics tool is too complicated and tracks users with cookies, so what simpler privacy-focused platforms should we consider?"
"Data residency","compliance lead","DE","Which managed analytics services let me choose EU data storage and provide a data processing agreement for GDPR purposes?"
"Free and affordable plans","developer","US","What analytics tools have a free plan for personal projects and low-traffic sites, with paid plans that stay affordable as I grow?"Check your tracker’s import format before loading it. In a sheet, each row is one question to run. To set your cut against a hundred questions written for you from your website and put to ChatGPT and Google AI Mode, try the free AI Visibility Report, and read its questions with the same echo test. Partners get the full version of that agent.
Step 8: Freeze the set before the first count
Once the tracker has run the set, stop editing it. New questions join as new rows, dated from their first run, and the original wording stays as it was. The trend only means something while the questions under it stay the same.
A frozen set is what lets you read a weekly line like this one. The card below comes from the Empact Panel, our own build, which keeps every answer, citation and competitor for a partner’s buyer questions. It follows one partner’s share of voice beside three rivals’ across 90 days of weekly collections. Share of voice is each brand’s slice of every brand mention the answers make, and earlier mentions weigh more:

From the second week on, the partner’s line sits in a narrow band below every rival’s. On a set that stays still, read a one-week dip against the weeks around it before you call it a change. The free playbook below takes the set from there: a baseline in a week, one scoring sheet, and how to read the noise between weeks.
The echo questions you struck still have a use. Keep the ones about your newest launch as that separate launch set, and run it beside the main set with the same number of runs. If the launch questions come back without your name while the open ones carry it, look at your launch pages and at the pages those answers cite before anything else, and the launch set will show you whether the work reaches the answers.
Frequently Asked Questions
Can Claude write the whole set without a person cutting it?
Ours passed every rule in the prompt, and a third of it still repeated the vendor’s own pages. In our run Claude could not tell a buyer’s question from marketing copy, because it learned the category from the pages we pasted. Keep Claude for the draft, and give the cut to someone who has heard buyers describe the problem. They will know which rows nobody outside the company would ask.
What if we have no support inbox to draw questions from?
Use what sales hears instead: first-call notes, the free-text field on your demo form and the questions prospects send before they buy. Those sit closer to buying than support tickets, since a ticket comes from someone who already chose you. Keep the review pages either way, because reviewers use the words strangers use.
How do I spot an echo question without reading every page we own?
Search Google for the question’s most specific phrase with site: and your domain. If your homepage, a launch post or your pricing page says it nearly word for word, treat the question as yours until you have read it again. A faster tell: would a buyer who has never heard of your product know the feature exists? If not, the question describes your roadmap.
Should we track questions about a feature we just launched?
Yes, in a separate set of launch questions, reported on its own line. In our run, questions about an assistant integration Umami had announced ten weeks earlier never brought up its name unless they also used its privacy or open-source words. That is worth knowing, and it belongs nowhere near the headline number your board reads.
How many topics should the set have?
As many as your buyers have distinct reasons to shop, and no more. Claude proposed ten for one analytics product. After the cut, eight held up with five to seven wordings each: one was dropped as the roadmap, and two merged because they were the same need. A topic you cannot write four honest wordings for is usually a feature dressed as a need, and it belongs in the launch set.
Does every persona need its own questions?
Only if you will read that persona’s number on its own. A founder and a compliance lead ask about the same product for different reasons, and the answers they get differ. A persona with two rows in a set of forty tells you nothing about that persona. Either give it a real share of the questions or fold it into the persona closest to it.
What goes in the market column when we sell everywhere?
The country you will run the set from first, usually where most of your pipeline sits. Global is not a place an engine answers from, since every run happens from some location. Add a second market only when you will run the whole set there, and keep questions that name a country, a law or a regulator in that country’s market. Seven of our forty-three rows sit outside the US for that reason.
Will ChatGPT and Google AI Mode score the same set differently?
Yes, and you should report them apart. On our kept questions ChatGPT named Umami in 49% of its answers and Google AI Mode in 37%, over the same three runs. The alternatives questions split the same way, 62% against 31%.
The set does not need to change per engine. The reading does, because a gap between engines shows where the work has to go, and a blended number hides it. Claude was not in this run, since the data service we used does not reach it.
Can we start from a template set from a tracking tool?
As raw material, yes. Run it through the same four checks and the echo test you use on your own pages. Template sets are written for a category rather than for your buyers, and they often carry the leader’s name or a vendor’s feature language. The rows that survive still need your topics and your wordings.
Should the questions name our product category?
Yes, in the words buyers use for it, which are not always your homepage’s. Umami’s pages say web and product analytics, while its G2 reviewers wrote website analytics, privacy and simple. A buyer’s words leave the question open to the whole category. Your own term risks narrowing it toward the company that coined it.
Can we keep the questions Claude wrote word for word?
Most of them. Of our sixty, thirty-four kept Claude’s wording exactly, four were rewritten and twenty-two were cut. Resist polishing the keepers into neater English. A slightly clumsy sentence with a team size and a tool type in it is closer to what a buyer types, and the tidy version drifts back toward your own copy.
When does it make sense to build the set with Empact?
When the number will steer a budget, or the set has to cover several products, personas or markets and nobody on your team can give it an afternoon per market. Every partnership we run begins with an audit, and the question set is where we would want yours to begin.
Sources (1)
- Empact Partners, "question-set run" (2026)Empact Partners' own data↩1↩2↩3↩4↩5sixty questions drafted by Claude (Sonnet 5.5) from Umami’s public use-case pages, support questions and G2 reviews, cut by hand to 43. 72 questions in three shapes put to ChatGPT with web search and Google AI Mode, three runs each from a US location, 432 answers, 17 November 2026.
Keep reading
GEO How to Log What ChatGPT and Claude Search Before They Answer
Log the searches ChatGPT and Claude run through their APIs for your buyer questions, keep the ones that recur, and decide what each needs.
GEO Which Pillar Of AI Engine Visibility To Fund First
Two questions about your category, how crowded it is and who buys, set whether your site, mentions or Reddit gets the money first.
GEO How to Get Your SaaS Company Named in AI Search
Find what keeps your SaaS company out of AI answers: blocked pages, missing listings or content that misses the buyer's question.


