"AI traffic" is three unrelated events wearing one label, and adding them together produces a number that cannot be acted on. This page explains how we split them apart, and — more usefully — what each one is evidence of.

The four kinds

Every request we see gets exactly one kind. The classification is mechanical: it reads the user agent and the referrer's hostname, and nothing else.

  • Crawler. A known AI fetcher requested the page. Nobody was reading it at that moment. This is a machine building or refreshing something — an index, a training corpus, a cache — and it is evidence about your accessibility, not your popularity. The useful questions here are whether they are getting a 200, which pages they are taking, and whether that changed.
  • Referral. A person clicked a link to your site from an AI product's interface. There was a human, they arrived, and they arrived because something in an answer pointed at you. This is the closest thing to a citation you can observe from your own server, and it is the number most worth watching.
  • Search. A person arrived from a search engine. We keep this separate from AI referral rather than folding it in, even though the line between a search engine and an answer engine is blurring — because collapsing them would make the AI number grow every time someone redesigned a results page, which is not a fact about you.
  • Agent. A request that looks like software acting on a person's behalf right now, rather than a crawler working through a queue. The distinction matters because an agent's fetch usually has a person waiting at the other end of it, so a slow or blocked response costs something immediately.

An important consequence: a crawl is not a visit. A site whose AI numbers are climbing purely on crawler volume is being read by machines and visited by nobody, and the two trends want completely different responses. If your dashboard shows one combined figure, it cannot tell you which you have.

Why some AI arrivals are invisible

Referral detection depends on the referrer header, and it is not universal. Some AI interfaces send one, some strip it, some route through an intermediate redirect, and behaviour changes without notice. Some answers never produce a click at all — the person got what they needed and moved on.

So the honest framing for the referral number is: this is a floor, not a total. It undercounts by an amount nobody can quantify, including us. It is still worth having, because a floor that moves is informative even when the absolute value is wrong. It is not worth reporting to a board as "AI-driven traffic" without that caveat attached.

The nine counting categories

Alongside the kinds, we count every request we serve into one of nine categories. This is a different lane with a different purpose: the kinds answer "who is reading me and how", the categories answer "what did my site actually serve today".

They divide up like this:

  • Human page views, split by where the person came from — four of the nine categories are page views: direct or unattributed, from a search engine, from an AI assistant, and from another website.
  • Bot traffic, split into AI crawlers and everything else.
  • Static assets — stylesheets, scripts, images, fonts.
  • A catch-all for everything remaining: redirects, errors, and the robots.txt and sitemap fetches.
  • A bucket for requests we could not classify, kept visible rather than distributed.

Two properties are worth knowing because they change how you read the chart:

  • Every request lands in exactly one category, so the categories sum to your total request count with no residue. There is no "other, uncounted" branch. That is what makes the total trustworthy.
  • "Unknown" is a visible row, never a silent fold. When we cannot classify a request — most often because it came from an edge running an older build — it is counted as unknown and drawn as its own band. Folding those into "direct human traffic" would invent a human-traffic figure out of a population nobody classified, which is precisely the kind of quiet fabrication this taxonomy exists to prevent. A large unknown band is a real signal: it means part of your traffic is not being measured, and you should know that rather than see it distributed into the numbers you trust.

What we deliberately do not collect

Classification reads the referrer's hostname and stops. Never the path, never the query string. That is not a policy statement bolted onto the product — it is how the classifier is written, and it is why the AI-versus- search decision can be a guarantee rather than a hope: the tests are anchored against a parsed hostname, so gemini.google.com is AI and www.google.com cannot be.

The practical cost of that choice is that we cannot tell you which question someone asked before they landed on you. We think that is the right trade, and we would rather name the limitation than let you assume we have data we chose not to keep.

Reading it in practice

A short version, for the dashboard:

  • Crawler volume up, referrals flat. You are being indexed more. That is a precondition for citation, not citation. Check that the crawlers are getting real content and not a JavaScript shell.
  • Crawler volume drops to nothing. Look at access rules before you look at anything else. A blocked fetcher is the most common cause and the most easily fixed.
  • Referrals up on specific pages. Those pages are answering something well. That is your best available signal about what to write next, and it is far more reliable than any aggregate score.
  • A growing unknown band. A measurement problem, not a traffic problem. Fix the measurement before drawing conclusions from anything beside it.

None of these numbers tell you whether a model quoted you in an answer nobody clicked. That measurement does not exist from where we sit, and we would rather say so than sell you a proxy for it.