Measuring visibility in AI answers
A repeatable way to answer “does ChatGPT recommend us?” engine by engine, with repeated runs and the sources each engine read. In the HVAC audit it showed why the fixes belonged on the Business Profile and directories, not the blog.
Why a method matters here
AI answers change from run to run, differ by engine, and are easy to over-read. Published research on generative search finds that cited sources barely overlap across engines and shift between sessions. A single screenshot proves very little. The useful question is narrower: across a fixed set of real customer questions, which engines name the business, where, and what did they read to decide?
The method
- Write the questions customers ask. Service plus place, in plain words: “My furnace stopped working. Who can I call in [town]?” The same set is kept for every re-run.
- Ask each engine separately. ChatGPT, Perplexity, Google AI Mode and Claude are counted on their own lines. Totals across engines hide the pattern that matters.
- Repeat. Three fresh runs per question where the engine allows automation, so a result is a rate (16 of 24), not an anecdote.
- Use clean sessions. Logged out, clean browser. A signed-in answer is recorded but flagged as personalized and kept out of the counts.
- Record citations, not just mentions. Whether the business is named, its position in the list, and every source the engine cited. A sentence such as “X did not come up” contains the name but is not a recommendation; counting it inflates results.
- Date everything and re-run. The same questions at month 3 are the baseline to beat.
What it found in a real audit
From the anonymized HVAC audit (37 answers, 27 September 2026):
| Engine | Named | What it read |
|---|---|---|
| Perplexity | 4 of 4 | The company’s own website |
| Claude | 16 of 24 | Yelp, Angi, the company’s website |
| ChatGPT | 4 of 5, never first | Birdeye, Angi, HomeAdvisor, BBB, a dealer page |
| Google AI Mode | 1 of 4 | Maps listings and Google review text |
The engines that read the website already favored the business. The engines that read listings and reviews did not. So the fixes were a Business Profile category test, accurate hours, and corrections on the directories ChatGPT quotes (its map showed a lower star rating than Google’s). Writing more articles would not have moved the weak engines.
The same method, automated
For the studio’s own market a nightly Python job asks a fixed set of eight questions (for example “best SEO company in Bend Oregon” and “how to check whether ChatGPT recommends my business in Bend Oregon”) through a headless AI client with web search, and stores one row per engine and question: the answer, whether the domain is cited, and every source URL. Engines without an automatable route are listed as skipped, not estimated.
Quality control that made it into the method
- A batch loop once fed the rest of the question list into one answer, so a question about one town listed businesses from another. The run was discarded, re-run clean and the report corrected before it was finalized. Each question now runs in isolation.
- An early tally counted any mention of the brand. Some “mentions” were answers saying the brand did not appear. The tracker now reports citations separately.
Deliverable
A question-by-engine table, an engine summary with the sources each one read, the fixes those sources imply, and the dated baseline for the re-run. Sold on its own as the AEO visibility check, or inside the full audit.
Want this done
for your business?
Every engagement starts with a short scoping conversation and a written quote.
Last reviewed: