crontent

Your GA dashboard can't tell you if AI used your site

Crontent

Analytics will happily show you a flat line while AI systems hit your docs all day. Dominic Plouffe went through 1,252,023 requests from five months of server logs and made the useful point: training, indexing, and live answering are different jobs, and lumping them together tells you almost nothing about visibility.

How can I tell if AI actually used my site?

Read raw server logs and split AI traffic into three buckets. As Dominic Plouffe shows, the useful cut is:

  • Training crawlers like GPTBot, ClaudeBot, and Google-Extended
  • Indexing/search bots like OAI-SearchBot, Claude-SearchBot, PerplexityBot, and Googlebot
  • Live answer fetchers like ChatGPT-User

Those are not the same event with different labels. A training crawler means your pages may get pulled into a corpus. An indexing bot means an answer engine is preparing to find you later. A live fetcher means a user asked a question and the system came to read your page right then.

That last one is the closest thing to proof that your page helped answer a real query. If you only stare at GA, you blur all three together or miss them completely.

Referral traffic misses the part you actually care about

Google Analytics is built to track visits, sessions, and referrers. AI answer systems don't always behave like normal visitors, so Evil Martians ended up instrumenting their own site to measure LLM traffic directly after admitting GEO advice is useless if you can't verify what happened.

Server logs catch requests whether they become a clean referral or not. They also show the stuff that matters when you're debugging visibility:

  • which user agent hit you
  • which URL it requested
  • how often it came back
  • what response code it got
  • whether the fetch pattern looks like broad crawling or a one-off page pull

Plouffe's other warning matters here too: the user agent is wrong about half the time. So you can't stop at a string match. You need to check IP ranges and request patterns, because some agents identify themselves loosely, some route through other infrastructure, and some look like one kind of bot while doing another job.

If you skip that step, you'll tell yourself "ChatGPT used our docs" when all you saw was one crawler sweep weeks ago.

The three buckets tell you what to fix next

Once you classify requests, the next move gets much clearer. Plouffe lays out separate crawlers for separate jobs, which means the fix depends on which bucket is weak or missing.

If training bots hit you but search bots don't, your pages may be readable but not easy to index for answer engines. If search bots show up but live fetchers never do, your pages might be discoverable but not strong enough to use during answer generation. If live fetchers hit stale docs, then your AI visibility problem is a docs freshness problem.

For a tiny SaaS, that beats another month of generic GEO posts. You're not guessing anymore. You're checking:

  1. Are key product and docs pages reachable?
  2. Are you blocking the wrong bots in robots.txt or firewall rules?
  3. Do important URLs return 200s fast enough to be fetched reliably?
  4. Do pages have clean structure, headings, and plain language the models can extract?
  5. Do real answer fetches cluster around docs, pricing, comparisons, or random low-value pages?

That's the difference between content theater and measurement.

A basic log classifier is more useful than another GEO checklist

You do not need a giant data stack to get started. Evil Martians made the same practical point from the measurement side: until you instrument what AI systems actually request, you can't tell whether your work changed anything.

Start with one export from your server or CDN logs. Then tag each request with:

  • known AI bot user agent
  • verified IP range when available
  • request timestamp
  • URL path
  • status code
  • fetch frequency per bot and per URL

After that, sort pages into three piles: never touched, crawled only, and fetched during live answering. The pages in that third pile are your real AI visibility report.

If you're a solo founder or a 3-person SaaS team, that's the whole play. Don't ask GA to answer a question it was never built for. Read the logs, separate the bot jobs, and find out whether AI systems are reading you, citing you, or skipping you.

Sources