How to Test Your Website the Way a Buyer’s AI Agent Would

Buyers now hand the first round of a software search to an AI agent: find the price, compare the plans, book the demo. The agent does it on your website, and nothing in your analytics shows where it gave up. This is how to find out: one browser agent, three buyer tasks, a log the agent fills in itself, and a fix list with an owner on every row.

The usual check is somebody on the team clicking through the site, and it fails because they know where everything is. Empact Partners, a consultancy that runs go-to-market work inside B2B software companies, uses this test in its Product-Led Growth work: an agent that knows nothing about the company, a fresh browser per task, and a record of every click.

The test starts where an AI answer sends the buyer, which is rarely the homepage. We ran it on our own site first, on 28 October 2026, and the agent finished all three tasks. To finish the second it had to read our homepage’s raw HTML, because our main menu shows an agent a button and none of the pages behind it. Before you start, you need four things:

This tests whether an agent can finish three sales tasks on your public pages. It does not test your support desk, a logged-in product, or whether AI crawlers can read your HTML, which are separate tests with separate fixes.

Start on the Page an AI Answer Sends Your Buyer To

A buyer’s agent does not arrive at your homepage. It arrives on whatever page the answer cited, reads that page, and has to find the price, the plans and the booking from there. So the first job is knowing which pages those are.

Step 1: List the pages AI answers cite for your buyer questions

Type the questions your sales team hears on first calls into ChatGPT and Google AI Mode. Open the sources under each answer and write every URL on your own domain into a sheet, one row each, with the question that produced it beside it. The page that turns up most is where your test starts.

We do this with our own software. The Empact Panel is our build, it has collected more than half a million AI answers so far across nineteen panels, and it does three things a sheet cannot:

Those three are how we know which pages matter and what they look like to a machine. For our own site, from the questions buyers ask in our category, it recorded this:

So the agent starts on that case study. It links to my booking page and, as far as an agent can see, to nothing that says what we sell. If none of your pages show up at all, the agent will reach you by name and land on your homepage: start there, and treat the absence as a visibility problem before a usability one.

Give One Agent Three Buyer Tasks, Each in a Fresh Session

The agent is Claude Code driving a real Chrome through Microsoft’s Playwright MCP server, a small program that hands an AI model a browser to click through. We run our agent work in Claude Code, and the record it leaves is a file you can read afterwards, which matters more than anything the agent says about itself.

Step 2: Save the browser config and start it once

Make an empty folder for the test. In it, save this file as playwright.json:

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": [
        "@playwright/mcp@0.0.83",
        "--headless",
        "--isolated",
        "--save-session",
        "--output-dir",
        "playwright-output"
      ]
    }
  }
}

The file starts the browser headless, so no window opens, and isolated, so every session gets a new browser with no cookies and no history. It also writes a log of every action into a folder called playwright-output.

Then, in a terminal in that folder, start the server once by hand so its package is downloaded and cached before any agent needs it:

npx @playwright/mcp@0.0.83 --version

The command should print Version 0.0.83. Keep the version pinned, as in the file. With @latest, npx asks the package registry on every start, which took between 39 and 51 seconds on our machine. Pinned and cached, the server started in under three seconds.

You will know if the browser was not ready in time, because the agent says so. Three of our six task runs began before the server was up, and each time the agent reported that it had no browser, refused to guess, and filled in PAGES: none visited. An empty PAGES line means the test did not run. It says nothing about your site.

Run the task again. If it keeps happening with the version pinned, the session is starting without waiting for its servers. Ours was, because our own environment exported MCP_CONNECTION_NONBLOCKING, which a plain terminal does not carry, and the runs went through once it was removed.

Step 3: Write the three tasks

Each task is one prompt, and the three differ in one paragraph. This is task A as we ran it. For your own site, change the URL to the page your sheet from step 1 puts first, and change “Reddit marketing” to what your buyer was asking about.

You are a buyer's research agent, and the browser is your only tool. An AI assistant cited this page when I asked it for help with Reddit marketing:
https://www.empact.partners/empact-zone/reddit-marketing-case-study-insurance-broker-leads

Your task: find out what it costs to work with this company and how they set their price. If the site does not say, tell me that, and tell me what the site wants a buyer to do to get a price.

Rules:
- Start at that page. Stay on this company's own website and the pages it links to. Do not use a search engine.
- Do not submit any form, book any time, or type any personal details.
- If a cookie banner or popup appears, close it.

When you have finished, or cannot get any further, end your answer with this report, filled in:

RESULT: done / partly / stuck
ANSWER: what you found, in one or two sentences
PAGES: every URL you visited, in order
STOPPED AT: the page and the element where you stopped, and why
WOULD HAVE HELPED: what on the site would have let you finish faster, or at all

Tasks B and C replace the paragraph that begins “Your task”, and nothing else:

Your task: I run a product-led SaaS company. Signups arrive but stall, and I want my product recommended when buyers ask ChatGPT about our category. Find what this company offers, tell me which of its offerings fit my situation, and how they differ from each other.

Your task: start a request to talk to this company about working together. Go as far as the step where I would type my own name or email, then stop. Do not submit anything. Tell me each step it took to get there, and what the final step asks me for.

Every line in the prompt is there for something you will see in the log:

Write down what done means for each task before you run it. Ours, for the price, was the agent saying we publish no figure, saying what the site tells a buyer about how a price is set, and naming the route to one. Without that line written first, the run grades itself.

What Reddit says

“Once it gets slightly confused it just keeps digging itself deeper. Limiting scope seems like the only thing that actually works in practice.” — r/AI_Agents, Mar 2026

“In other words - Browser agents are great for testing if your site/service has good UX :)” — r/AI_Agents, Mar 2026

Step 4: Run each task in its own session

Save the three prompts as task-a.txt, task-b.txt and task-c.txt in the test folder. Run task A from that folder:

claude -p --model claude-sonnet-5-5 --effort medium \
  --setting-sources project \
  --strict-mcp-config --mcp-config playwright.json \
  --tools "" --allowedTools mcp__playwright \
  < task-a.txt

Each flag closes a door the test must not use. --tools "" removes every built-in tool, so the agent cannot read your pages any way but through the browser. --strict-mcp-config and --mcp-config load the Playwright server and nothing else. --setting-sources project keeps your own Claude settings and instruction files out, so the agent arrives knowing nothing about you.

When task A’s answer ends in the filled report, run the same command with task-b.txt, then task-c.txt. Each run is a new session with no memory of the last, and ours took 61, 91 and 112 seconds.

We ran them through our own wrapper for headless Claude, which adds a session id, a turn cap, a permission mode and a machine-readable transcript. The flags above are every one that decides what the agent can see and do.

Read the Trace, Not the Agent’s Verdict

The report at the end of each answer is the agent’s account of itself. It is useful, and it is not the result.

Step 5: Copy each report into the log

Put the report lines into your sheet, one row per task, and add the column the agent does not fill: your own verdict, against what you wrote down as done. These are our three rows.

All three runs ended on the same page, my booking page, and none of them typed a character into it.

Step 6: Check each claim against the session log

Open playwright-output, then the folder named session- and a number, then session.md. Every action the agent took is there in order, under a heading that starts “Tool call”. Check that every page in PAGES was opened, that the log ends where STOPPED AT says, and how the agent got from the entry page to the answer.

Ours showed two things the reports did not say. The agent marked A and B “partly” where our outcome says done. It had done the task, and was marking our site down for what it lacked: a word on how we price, and a page comparing our workstreams. And B’s log shows how it reached our workstream pages at all.

On the case study it noted “Workstreams is a menu button.” On Partners it clicked the button and found nothing: “The dropdown didn’t expose links. I’ll try the homepage.” There a script listing every link in the raw HTML found the pages that say what we sell. Four pages and a script stood between the cited page and the first workstream page.

An agent’s report grades your site and itself in the same breath, so the verdict that counts is the one you wrote down before it ran.

Sort Every Stall Before Anyone Fixes It

A stall is one of three things: a fault in the site, something working as intended, or noise from the test itself. Sorting them first is most of the skill, because a list that treats every flag as a fault sends somebody to undo a decision.

Step 7: Give each stall a cause and an owner

Add three columns to the log for the cause, the fix and who makes it, one row per stall the session logs showed. Our five site faults:

The menu is the one to fix first. To a person, our Workstreams menu opens on hover and lists eight pages. The links sit in the HTML, hidden until the menu opens, and the agent never saw them open: on every page it tried, it found a button with nothing under it.

Every page on our site reaches the workstreams only through that menu, so an agent arriving on an article cannot see what we sell unless it reads the raw HTML. The fix is one template change: in Webflow, the dropdown set to open on click rather than on hover, and the same eight pages listed again in the footer.

Vlad Shvets

Founder @ Empact Partners

Our menu looked fine to everyone at Empact Partners, because every one of us knows where the workstream pages are. The agent knew nothing, read every word of a case study an engine had cited, and could not see the eight pages that say what we sell. The stall nobody on the team can see is the one to fix first, and the only way to see it is to send something that knows nothing.

The rest of what the agent flagged stays as it is, and the log should say why:

This sorting is the part our consultants do on a partnership. In Product-Led Growth work the test sits in the audit a partnership opens with: the entry pages come from the partner’s own panel, and the stalls are sorted against the trace.

The list then goes into the roadmap the partnership’s senior consultant owns. The copy rows go to Brand Marketing, the workstream that writes what a site says, and the template rows go to the partner’s own web team with the re-run attached.

Hand Each Fix to Its Owner, With the Re-Run as the Check

A fix is finished when the same task, run again from the same page, gets further.

Step 8: Write the re-run beside every fix

For each fault, write the check that proves it: the same prompt, from the same entry page, in a fresh session, read against PAGES and STOPPED AT. For our menu, it is a workstream page in PAGES within two steps of the case study, with no script in the log. For the price row, it is task A’s ANSWER naming how a partnership is priced.

If your agent can reach your CMS through a connector, let it change only what is reversible and low in consequence, such as a page description or an image’s alt text. Navigation, templates and what your pages say about price stay with a person. Every fault our run found was one of those three, so no agent changed anything.

What One Run Cannot Tell You

Our run was one agent on one day. An agent that reads screenshots, such as Claude in Chrome or ChatGPT’s agent, sees the page the way a person does and may open a hover menu ours never saw, or miss text ours read straight from the source. If you use one of those, paste the same three prompts into it and keep the same log. The columns do not change.

After the fixes, watch one number: how many pages stand between the page an engine cites and the first page that says what you sell. On our site it was four pages and a script.

If buyers keep landing on pages you never built as a front door, book a call with me. We’ll find out together whether fixing that is a job for us.