Updated: •Published: •11 min read•6 sources cited
The short answer
AI visibility tracking counts how often AI answers name your brand for a set of prompts. In our test of 36 Google AI Mode searches, a single repeated prompt can look rock solid or change every time, and rewording it changed the list. The reliable signal is how often a brand appears across many varied prompts, not its position.
What we tested (Oct 13, 2026) “Best SEO audit tools for small business” “Best Shopify SEO apps” “Best website builder for small business”
Same prompt, asked again and again 8 runs, 8 different lists 8 runs, 1 identical answer 5 runs, 1 identical answer
Same intent, reworded 5 ways 5 different lists 5 different lists 3 different lists
Named in every single answer 4 tools 1 app 4 builders
Linked sites that were also in Google’s top 10 24% to 42% 76% to 100% 8% to 24%

Rand Fishkin and SparkToro showed in January that AI tools almost never give the same brand list twice when hundreds of different people ask. I wanted to know something narrower, because it’s how most tracking tools actually work: what happens when one account asks Google AI Mode the same buyer questions over and over, and then rewords them?

Below: how we ran it, the three things that surprised me, what it means before you pay for an AI visibility tool, and a free way to run the same test on your own category.

Key takeaways
  • Two of our three prompts returned the exact same AI Mode answer on every repeat. The third changed every time. A tracker that only re-runs one prompt can report stability or chaos for reasons that have nothing to do with your brand.
  • Rewording the question changed the brand list in 13 of 15 rewordings, but a core set of brands appeared every time.
  • How closely AI Mode’s links matched the classic top 10 depended heavily on the category: from 8% to 100% in our runs.
  • Track mention frequency across many reworded prompts. Ignore “AI rank position”.

What is AI visibility tracking?

AI visibility tracking is the practice of running a fixed set of prompts through AI tools such as Google AI Mode, AI Overviews, ChatGPT or Perplexity, and recording which brands and websites the answers mention or link to. Most tools turn that into a share-of-voice percentage, and some into a “rank” for each brand.

The idea is sound. The problem is that AI answers aren’t fixed lists. Google says in its AI features documentation that AI Mode and AI Overviews may use a “query fan-out” technique, “issuing multiple related searches across subtopics and data sources” to build a response. So the same intent can be answered from different searches each time.

If you’re new to the measurement side, Google’s own numbers for AI features live in Search Console. I explained how to read them in my guide to the Search Console AI performance report.

Why we ran this test

Three people whose work I follow shaped this test, and they don’t fully agree with each other, which is why it was worth running.

  • Rand Fishkin (SparkToro) published research with Gumshoe.ai in which 600 volunteers ran 12 prompts through ChatGPT, Claude and Google’s AI a combined 2,961 times. They found a less than 1 in 100 chance of getting the same brand list in any two responses. His conclusion: visibility percentage across many prompts is a reasonable metric, but a “ranking position in AI” isn’t. Search Engine Journal’s write-up covers the method in short.
  • Chris Long (Nectiv) has been studying the searches behind AI answers. He spoke at Ahrefs Evolve this week on an analysis of more than 17,000 ChatGPT fan-out queries. If the answer depends on hidden sub-searches, wording should matter a lot.
  • Cyrus Shepard often frames search debates around evidence. This week he was surveying SEOs because, in his words, he wants “to look at what the data actually says.” That’s the spirit here: a small test you can repeat, not a big claim.

SparkToro’s study used hundreds of different people. Most tracking tools use one setup that asks the same prompt on a schedule. So my question was: what does that single-setup view look like?

How we tested Google AI Mode

We ran 36 Google AI Mode searches from one signed-in Google account in Chrome, with US English settings, on October 13, 2026, within about an hour. Each search was a fresh page load, not a follow-up in the same conversation.

Part What we did Searches
Exact repeats The same prompt, word for word, run again and again 21 (8 + 8 + 5)
Rewordings Five other ways a buyer might ask the same thing, for each prompt 15
Classic results The top 10 web results for each original prompt, for comparison 3 (not counted in the 36)

The three prompts are things our clients’ customers actually ask: “best seo audit tools for small business”, “best shopify seo apps” and “best website builder for small business”. Rewordings included “which shopify app should i use for seo” and “recommend a website builder for my small business”.

From each answer we recorded the brands named (in bold text or in the comparison table), the order they first appeared and every external site the answer linked to. We matched brands by name, by hand checked the list, and dropped generic phrases.

Finding 1: the same prompt can give a frozen answer, or a new one every time

Repeating a prompt in Google AI Mode didn’t behave the same way across our three prompts. Two returned the identical answer on every repeat: same brands, same order, same length. The third returned a different list every time.

Google AI Mode answer for best shopify seo apps showing a comparison table that starts with Tiny SEO and source cards from the Shopify App Store, EcommerceGold and Charle Agency
The “best shopify seo apps” answer we got on all 8 repeats and a ninth check, October 13, 2026. Screenshot from Google AI Mode.
  • “Best Shopify SEO apps”: 8 out of 8 runs returned the same five apps in the same order: Tiny SEO, SEO Lab, SearchPie, SEOAnt and Booster AI SEO.
  • “Best website builder for small business”: 5 out of 5 runs returned Wix, Shopify, Squarespace, Hostinger and WordPress, in that order.
  • “Best SEO audit tools for small business”: 8 runs, 8 different lists, between 8 and 11 tools each, in a different order every time. On the first run the page said it was “Thinking a little longer” before answering.

Google doesn’t explain when AI Mode reuses an answer and when it writes a new one, so I won’t guess at the cause. What matters is the effect on tracking.

Why this matters for tracking
If a tool re-runs one prompt daily and gets a stored answer, your brand looks perfectly stable. If it happens to hit a prompt that regenerates, your brand looks like it’s swinging wildly. Neither tells you how real buyers, who all word things differently, see you.

Finding 2: rewording changes the list, but a core set survives

Rewording a prompt changed which brands Google AI Mode named in 13 of our 15 rewordings. Only two rewordings, both for website builders, returned the same five builders as the original. Yet in every category, a small core set of brands showed up no matter how we asked.

AI visibility tracking test: how often Google AI Mode named each tool. SEO audit tools, 13 asks: SEOptimer, SE Ranking, Screaming Frog and Google Search Console 13 of 13, Semrush 11, Ahrefs 9, PageSpeed Insights 8, Bing Webmaster Tools 7, WooRank 6. Shopify SEO apps, 6 asks: Tiny SEO 6, SearchPie 5, Booster AI SEO, Plug in SEO and Yoast 3, Avada and Smart SEO 2. Website builders, 6 asks: Squarespace, Shopify, Wix and Hostinger 6 of 6, WordPress 3, GoDaddy 2
Share of asks that named each tool (original prompt plus rewordings; identical repeats counted once). Source: HR92 test, 36 Google AI Mode searches, October 13, 2026.
Prompt family Named every time Sometimes Different brands seen
SEO audit tools (13 asks) SEOptimer, SE Ranking, Screaming Frog, Google Search Console Semrush, Ahrefs, PageSpeed Insights, Bing Webmaster Tools, WooRank and others 14
Shopify SEO apps (6 asks) Tiny SEO SearchPie, Booster AI SEO, Plug in SEO, Yoast SEO for Shopify, Avada, Smart SEO and others 11
Website builders (6 asks) Squarespace, Shopify, Wix, Hostinger WordPress, GoDaddy, one other 7

This lines up with what SparkToro found at a much bigger scale: individual answers vary a lot, but how often a brand appears across many answers is far more stable. The Shopify category was the loosest. Two apps from the “frozen” original answer, SEO Lab and SEOAnt, didn’t appear in any of the five rewordings.

My take
If you sell in a tight category (website builders), being in the core set is a brand problem: you need to be one of the names everyone already writes about. If you sell in a loose one (Shopify apps), there’s room, and the rewordings show which questions you could win. That’s a content and digital PR job, not a tracking job.

Finding 3: AI Mode’s sources only sometimes match the top 10

The websites Google AI Mode linked to overlapped with the classic top 10 results to very different degrees depending on the category. That fits Google’s own description: AI features can show “a wider and more diverse set of helpful links” than a classic web search.

Prompt family Linked sites also in the top 10 (original prompt) Same, for rewordings
Shopify SEO apps 100% (6 of 6, every repeat) 76% (32 of 42 links)
SEO audit tools 42% (27 of 65 links across 8 runs) 24% (13 of 55)
Website builders 8% (2 of 24, every repeat) 24% (12 of 49)

For rewordings we compared against the top 10 of the original prompt, so part of the drop is simply that a reworded search has its own results. Still, the lesson holds: ranking in the top 10 helps in some categories much more than others, and AI answers also lean on sites that don’t rank on page one at all.

That’s why strong classic SEO still matters for AI visibility, but it isn’t the whole job. Which brings me to what I’d actually measure.

What to track instead of “AI rank”

The metric to track is how often your brand appears across a large, varied set of prompts, measured over time, next to your normal search data. Here’s how I’d weigh what most AI visibility tools report:

Metric a tool shows you Trust it? Why
Your “position” in an AI answer No Order changed on every regenerated answer in our test
Day-to-day change on one prompt No Could be a stored answer or a new one; you can’t tell which
Mention rate across 50+ reworded prompts, monthly Yes, as a trend The core-set pattern held in every category we tested
Which sites AI answers cite in your category Yes Tells you where to earn coverage and links
AI impressions in Search Console Yes Google’s own data, by page (impressions only)

If you want help building a prompt set and turning it into content and PR priorities, that’s what our generative engine optimization (GEO) work covers.

How to run this test on your own category (free, about an hour)

You can run a small version of this AI visibility test yourself with a browser and a spreadsheet. This is the routine we used:

  1. Write 3 buyer prompts in your customers’ words (“best [category] for [type of business]”).
  2. Write 5 rewordings of each. Ask a colleague to write two of them; people phrase things differently than you think.
  3. Run each original prompt 5 times in Google AI Mode, each as a fresh page, not a follow-up.
  4. Run each rewording once.
  5. Log the brands named and the sites linked in a sheet: one row per answer, one column per brand.
  6. Check whether repeats were identical. If they were, count them once.
  7. Calculate mention rate: answers that named you divided by answers in that prompt family.
  8. Note which cited sites cover your category but not your brand. That’s your outreach list.
AI visibility test checklist
  • 3 prompts written in customer language
  • 5 rewordings per prompt
  • Fresh page for every search
  • Identical repeats counted once
  • Brands and linked sites logged per answer
  • Mention rate per prompt family
  • Core set vs “sometimes” brands marked
  • Cited sites without your brand listed for outreach
  • Same test repeated next month

And start with the basics on your own site: AI features draw on Google’s index, so pages that aren’t indexed or are slow and thin won’t help you. Our free SEO audit checks that, including an AI citability score.

Questions to ask before you buy an AI visibility tool

Before paying for an AI visibility tracking tool, ask the vendor these questions. A good vendor will have clear answers.

  • How many times do you run each prompt, and do you report the spread or just one result?
  • Do you detect identical, reused answers and count them once?
  • How many rewordings per topic do you track, and can I add my own?
  • Do you report “position”? If so, how do you handle answers that reorder every run?
  • Which accounts, locations and devices do your runs use?
  • Do you show the sites each answer cites, not only brand mentions?
Limits of our test
This was one account, one day, three categories and 36 searches in Google AI Mode only. It shows patterns worth checking, not universal rules. Results can differ by account, location, device and over time, and ChatGPT, Perplexity and AI Overviews behave differently.

FAQ

Is AI visibility tracking worth paying for?

It can be, if the tool runs many varied prompts many times and reports mention rates as trends. It isn’t worth paying for a single “AI rank” or daily swings on one prompt. Our test showed one prompt can look frozen while another changes every run.

Does Google AI Mode give the same answer every time?

Sometimes. In our test, two of three prompts returned an identical answer on every repeat, while the third gave a different list each time. Rewording the question changed the brands in 13 of 15 cases.

How do I check if my brand shows up in Google AI Mode?

Search your buyers’ real questions in AI Mode, each as a fresh page, plus several rewordings, and note if your brand is named. For Google’s own data, check the Generative AI report in Search Console, which shows AI impressions by page.

Do AI Mode answers use the same sites as the top 10?

Partly. In our test the overlap ranged from 8% to 100% depending on the category. Google says AI features can show a wider set of links than classic search, so ranking well helps but isn’t the only route.

What is a good AI visibility metric?

Mention rate across a large, varied set of prompts, tracked monthly, plus the list of sites AI answers cite in your category. Both point to actions: content to create and sites to earn coverage on.

Want to know where your brand stands in AI search?
Start with our free 54-check SEO audit and AI citability score. If you want a proper prompt set, mention tracking and a plan to grow it, talk to our GEO team.

Related: GEO services, digital PR and SEO services.

Abdullah Hussain, founder of HR92
About the author
Founder, HR92 · PMP · Google AI Professional Certificate
Abdullah leads HR92’s work in SEO, AI search, Google Ads and paid social, web development and ecommerce growth from Miami, Florida, with an office in Doha, Qatar. He helps businesses in the US and the Gulf get found online and turn search traffic into revenue, and writes about what is working in search, AI and paid media based on hands-on client work.

Last updated October 13, 2026. Test run by HR92 in Google AI Mode on October 13, 2026; sources checked the same day.