Measuring AI Search Visibility With Share of Answer
AI search visibility comes down to one number: share of answer. How often does your brand show up across a fixed panel of prompts you run repeatedly against AI assistants, scored for whether you're mentioned, where you land, and how you're framed? Because these models are non-deterministic, you run each prompt N times and report a mention rate, never a yes-or-no. Treat the number as a defensible directional metric. Not attribution ground truth.
Why "share of answer" is suddenly a board-level metric
For most of the last decade, the marketing question was "did they click?" That question is quietly dying.
Google's AI Overviews crossed over 2 billion monthly users across more than 200 countries by mid-2025, according to comments Sundar Pichai made on the company's earnings call and reported by Digiday. ChatGPT reached 900 million weekly active users as of OpenAI's late-February 2026 announcement, per TechCrunch. Those two surfaces alone now sit between an enormous number of buyers and their first mention of any product category. People ask the machine, and the machine answers.
The problem is what the answering does to clicks. Pew Research Center analyzed the actual browsing behavior of 900 U.S. adults during March 2025 and found that when a Google AI summary was present, users clicked a traditional search result only 8% of the time, versus 15% without one. Campaign US reported the same collapse. Worse, users clicked a link inside the AI summary just 1% of the time. So you can be cited, correctly, favorably, and still get almost no traffic from it.
That's the pincer. Mass adoption on one side, click collapse on the other. The value of appearing in an answer has shifted from "sends me visitors" to "shapes what the buyer believes before they ever reach my funnel." If placement is what matters now, placement is what you have to measure. Traffic was a convenient proxy. It's stopped being one.
What share of answer actually is
Here's the definition I'd defend in a room.
Share of answer (SoA) is the proportion of qualifying brand mentions across a fixed prompt panel that belong to your brand, measured over repeated runs and reported as a rate with variance.
Two formulas do the work. The first is the aggregate share across your panel. The second is the per-prompt reliability layer that keeps you honest.
| Metric | Formula |
|---|---|
| Share of Answer | (your brand mentions) / (total qualifying brand mentions) across the full panel |
| Mention rate (per prompt) | (runs that mentioned you) / (N total runs) for a single prompt |
"Qualifying" means brand mentions inside the relevant competitive set. If a prompt about analytics tools surfaces your brand plus four competitors, those five are the denominator. A passing reference to Excel doesn't count unless Excel is genuinely a competitor for that intent.
Score three dimensions on every run: mention (were you named), position (first-named or buried at the bottom), and sentiment (framed positively, neutrally, or negatively). Mention without position is a vanity flag. You can be technically present in every answer and last in every one, which is not a win you want to put on a slide.
Why you run each prompt N times, not once
This is the part that separates a real methodology from a screenshot someone pasted into Slack.
Large language models produce different outputs for identical inputs, and setting temperature to zero doesn't fix it. An October 2025 arXiv paper puts the reason plainly: "even with temperature set to 0 (the most deterministic setting) determinism is not guaranteed due to hardware factors like GPU concurrency." The randomness isn't a knob you forgot to turn. It's baked into how these things run at scale.
Earlier peer-reviewed work quantified the effect. A July 2024 arXiv study found that "even at a temperature setting of zero, ChatGPT provided non-deterministic answers to simple prompts related to software curriculum nearly 10% of the time with an error rate of over 5%." If a simple factual prompt flips one time in ten, your brand-mention question, which is far mushier, will flip more.
So a single query is a coin you flipped once. You wouldn't report a conversion rate from one session. Don't report AI visibility from one prompt.
Assumptions box #1 — the run design
- N = 10 runs per prompt per assistant, per measurement window. Enough to see a rate, cheap enough to schedule.
- Temperature left at each assistant's default. We're measuring what real users get, not a lab setting. Forcing temp-0 wouldn't even guarantee determinism (see above), so don't pretend it buys you cleanliness.
- We report mention rate ± the run-to-run spread, never a binary "yes, we appear in ChatGPT."
- Disagree? If your prompts are high-variance, push N to 20. The method holds; the number is yours.
Designing the prompt panel
The panel is the instrument. If it drifts, your trend is noise wearing a suit. I build a 30-prompt panel split across four intent categories, then freeze it.
| # | Category | Example prompt | Buyer intent | Expected competitor set |
|---|---|---|---|---|
| 1–8 | Category-defining | "best product analytics tools for B2B SaaS" | Shortlisting | You + top 4 category leaders |
| 9–16 | Problem-led | "how do I track feature adoption in my mobile app" | Problem-aware, pre-brand | Tools + generic methods |
| 17–24 | Comparison | "X vs Y for session replay" | Late-stage, comparing | Named head-to-head rivals |
| 25–30 | Branded-recall | "what is [your brand] and who is it for" | Already knows you | You (checking accuracy) |
Category-defining prompts tell you if you exist in the model's mental shortlist at all. Problem-led prompts are where you catch demand before a buyer knows any brand names, which is the most valuable and most-ignored slice. Comparison prompts show how the model narrates you against a specific rival. Branded-recall prompts are your accuracy check: if the assistant describes your product wrong, that's a content problem you can actually go fix.
Keep the split roughly balanced, write the prompts the way a real buyer would type them, and then stop touching them.
A CFO I worked with once killed a channel-performance slide mid-meeting because the definition of "qualified lead" had changed between Q2 and Q3. Not the number. The definition. Her line was, "You've measured two different things and drawn a line between them." She was right, and it's the exact trap a prompt panel sets. Swap prompts between windows and your quarter-over-quarter SoA line is comparing two different instruments. Freeze the panel before you present the first trend, version it, and log the date you locked it. When you eventually revise the panel — and you will, as the category shifts — you start a new baseline and say so out loud.
The scoring rubric
Scoring has to be mechanical enough that two analysts get the same answer. Here's the rubric.
| Dimension | Scale | How to code it | Weight |
|---|---|---|---|
| Mention | 0 / 1 per run | Named or clearly referenced = 1 | 0.5 |
| Position | 1 (first) → 5 (last/absent) | Order of first appearance in the answer | 0.3 |
| Sentiment | −1 / 0 / +1 | Negative / neutral / positive framing | 0.2 |
Now a worked example. One problem-led prompt, 10 runs against a single assistant. Fictional brand "Acme" for the walkthrough.
| Run | Mention | Position | Sentiment |
|---|---|---|---|
| 1 | 1 | 2 | +1 |
| 2 | 1 | 1 | 0 |
| 3 | 0 | – | – |
| 4 | 1 | 3 | +1 |
| 5 | 0 | – | – |
| 6 | 1 | 2 | 0 |
| 7 | 1 | 1 | +1 |
| 8 | 0 | – | – |
| 9 | 1 | 4 | −1 |
| 10 | 1 | 2 | +1 |
Roll it up. Mention rate = 6/10 = 0.60. Average position across the six mentions = (2+1+3+2+1+4)/6 = 2.17, so when Acme shows up it lands near the top third. Sentiment across mentions: four positive, one neutral, one negative, netting +0.50.
Composite for this prompt, normalizing position to a 0–1 scale where first place = 1 (I use (6 − position)/5):
- Mention component: 0.60 × 0.5 = 0.30
- Position component:
(6 − 2.17)/5= 0.77 × 0.3 = 0.23 - Sentiment component:
(0.50 + 1)/2= 0.75 × 0.2 = 0.15 - Prompt composite ≈ 0.68
The single negative run in slot 9 is the one I'd actually chase. Six-in-ten presence is fine. A model narrating you negatively 10% of the time, on a problem-led prompt, is a content gap you can trace and close. The composite is the headline. The raw runs are where the work lives. If you already keep a metric dictionary for your team, this belongs next to the rest of them in the growth metrics glossary so nobody reinvents the scoring next quarter.
Scheduling runs across assistants
One assistant is a partial picture. With AI Overviews past 2 billion monthly users and ChatGPT past 900 million weekly, a buyer's first impression could form on either, so cover at least both, and add others your audience actually uses.
Run cadence: weekly or biweekly, same day of week, same rough time window. Log every run with a fixed schema — prompt ID, assistant, model version if exposed, date, region, run number, and the three scores. Hold model, date, and region as constant as the platforms let you, because you can't separate a real shift in your visibility from a silent model update if you've let three variables move at once. When something jumps, the log tells you whether you changed or the model did. Boring infrastructure. It's the difference between a metric and a rumor.
Closing the loop: from mention to activation
A prompt panel measures presence. It can't tell you what a mentioned user did. That gap is honest and worth naming.
The fix is to tag AI-assistant referral traffic in your product analytics — flag sessions arriving from ChatGPT, AI Overviews, and similar — then compare their activation and retention against your SEO traffic. If AI-referred users activate at a higher rate, placement is doing quiet compounding work your click reports never showed. This referral-vs-SEO activation loop is the kind of question a product-analytics tool like Kixo is pointed at, not something a SoA panel will ever answer on its own. Two different instruments, two different jobs.
What prompt-panel SoA cannot claim
Time for the slide that keeps you employed after the follow-up question.
A panel measures a moving target. Ahrefs found in its May 2026 analysis that only about 38% of AI Overview citations came from top-10 organic pages, down from roughly 76% in its July 2025 study — a shift they attribute partly to query fan-out and model changes. The rules connecting your rankings to your citations are being rewritten mid-game. Your SoA can move because the model changed its sourcing, not because your visibility changed.
Even the platform owner offers an incomplete meter. Google's first-party AI visibility reporting, launched June 3, 2026, tracks impressions in generative AI features but, as Search Engine Land reported, "does not include click data" and was geographically limited at rollout. When Google itself can't hand you clicks, no third-party panel should promise them.
And a panel structurally can't see personalization, session memory, or regional model variation. Your prompts run cold, without a logged-in user's history behind them. Real buyers carry context you can't replicate. Treat vendor "AI visibility leaderboards" with suspicion, too. They're survivorship-biased toward brands already winning, because those are the ones easy to detect and fun to chart.
Assumptions box #2 — what we're pretending is stable
- The model's sourcing behavior is roughly constant between windows. (Ahrefs suggests it isn't — hence the version log.)
- Our cold, un-personalized runs approximate a typical user. They approximate an average, not any real person.
- The competitive set in each prompt is stable enough to compare. Re-audit it every quarter.
- Disagree with any of these and the whole trend line is up for debate. Good. That's the point of writing them down.
If you're building this alongside spend efficiency, the same discipline of named assumptions carries straight into MER versus ROAS decisions — different metric, identical rule about stating what you're pretending is fixed.
Putting it on the board deck
Present SoA as a trend with a confidence band, not a single hero number. Show the mention rate over the last several windows, the run-to-run spread as error bars, and a one-line note on any model change you logged. When someone asks "is this real or did the model shift," you point at the version log. When someone asks "what does one point of share buy us," you point at the activation comparison, not the panel.
That's the whole posture. A directional metric, assumptions written on the slide, stress-tested before the meeting. The CFO who killed my lead-definition slide would have let this one stand, because when she asked the follow-up, the answer was already on the page.
FAQ
How do I track my brand in ChatGPT? Build a fixed prompt panel, run each prompt around 10 times per window, and score every run for mention, position, and sentiment. Report a mention rate, not a one-off screenshot, because the model gives non-deterministic answers even at temperature zero.
How many prompts do I need? Thirty is a workable panel: eight or so category-defining, eight problem-led, eight comparison, and six branded-recall. Fewer and your share estimate gets jumpy. More and the run cost climbs without much added signal.
Does share of answer replace SEO reporting? No. It measures placement inside AI answers, which is now largely decoupled from clicks — Pew found link click-through roughly halved when an AI summary appeared. Run it beside your traffic and activation reporting, not instead of it.