Site Tracking

Site Tracking reports every page request your server handles to Ranqo, and Ranqo works out who made it: an AI crawler collecting training data, an AI search index, an assistant fetching your page because someone asked it a question, a search engine, or a person. The results appear in the Site Tracking view of the Traffic page.

Why it runs on your server#

Analytics tools such as GA4 count a visit by running JavaScript in the visitor's browser. AI crawlers fetch your HTML and leave without running any script, so those tools never see them. Site Tracking runs where every request arrives, on your server or edge, so it sees the crawlers, the files they read first, and visitors whose browsers block analytics tags.

There is no script on your pages and no cookie. For each page request, your server sends one short request to Ranqo with the page URL, the user agent, the referrer and the visitor's IP address, then moves on without waiting for the answer. See Privacy for exactly what is sent and stored.

What gets counted#

The install code for Next.js, Express, Cloudflare Workers and Python applies the same rules:

• Page requests only. It reports GET requests and skips everything else.
• Real page views only. Link prefetches and framework data requests are skipped. On a Next.js site, counting them would make traffic read several times higher than it is.
• The files AI crawlers read first. robots.txt, llms.txt, llms-full.txt, sitemap.xml and its parts, ai.txt and anything under /.well-known/ are reported, even though they look like static files.
• No assets or API calls. Images, fonts, scripts, stylesheets, other .txt, .xml and .json files, and paths under /api/ are skipped.

Ranqo also drops, after receiving them, requests for paths that only vulnerability scanners ask for (such as /.env, /.git/, /wp-admin, /wp-login and xmlrpc.php) and requests whose user agent is a web address. Those are probes, not visitors.

How visits are classified#

Bots#

Ranqo matches each request's user agent against its catalog of known crawlers and sorts the visit by why the bot came. The labels are the ones the Traffic page uses:

• Training: crawlers collecting pages to train models, such as GPTBot, ClaudeBot, CCBot and Meta-ExternalAgent.
• Indexing: AI search indexes that answers cite from, such as OAI-SearchBot, Claude-SearchBot and PerplexityBot.
• Agentic: fetches made on demand because a person asked an assistant something, such as ChatGPT-User, Claude-User and Perplexity-User.
• Search Engine: traditional search crawlers, such as Googlebot, bingbot and Applebot.
• Social Preview: link-preview generators such as Slackbot and LinkedInBot, and automated browsers such as Lighthouse and Playwright, so that test runs and speed checks are not counted as people.

A request from anything else that is not a web browser counts as Other bots: scripts, SEO tools' crawlers, uptime checks, crawlers the catalog does not list, and requests with no user agent at all. So does a browser user agent that claims a version no up-to-date browser runs (see Human visits).

See the crawler catalog for what the main AI crawlers are for.

Verified bots#

A user agent is only a claim, and anyone can send GPTBot. For bots from OpenAI, Anthropic, Perplexity, Google, Microsoft, and Common Crawl, whose operators publish the IP ranges they crawl from in a list Ranqo reads, Ranqo checks the visitor's IP address against those ranges:

• Verified: the address is in the operator's published ranges.
• Unverified: Ranqo had the ranges and the address was not in them, so the visit probably is not the crawler it claims to be.
• Not checkable: Ranqo has no published ranges for that bot's operator (or could not load them), the bot is a tool anyone can run (such as Lighthouse or Playwright), or the install sent no IP address Ranqo could read. The Agents detected table shows these as n/a, and they are left out of the verified rate rather than counted as suspect.

In the Live Feed, a verified visit shows a green shield and an unverified one an amber shield. A visit Ranqo could not check shows neither.

Verification depends on your install sending the real visitor IP, taken from somewhere the visitor cannot write. An install that sends the first entry of X-Forwarded-For lets anyone pass for a crawler by putting its address there, and one that passes on a proxy's own address makes genuine crawlers come out unverified. Each stack's guide says how its install reads the address.

Your install also reports Ranqo's own requests. Site Access, which runs on its own after your tracking runs, and the free AI Crawler Inspector fetch your pages from Ranqo's servers under crawlers' user agents. Most of those requests appear as visits from the crawler they name, unverified where Ranqo has the operator's ranges; the rest count as Other bots, and so does the browser request each check compares against, because its user agent claims an outdated Chrome. See Ranqo's requests in Site Tracking.

Human visits#

A request is a person only when its user agent is a web browser's: Chrome, Safari, Firefox, Edge, or the browser built into an app such as LinkedIn.

It also has to be a version an up-to-date browser could send. Chrome and Edge update themselves, while scrapers rotate through old browser strings, so a visit counts as Other bots when it claims a Chrome or Edge version Google had replaced more than 180 days before the visit, a full Chrome build number such as Chrome/150.0.7871.46 (Chrome has sent only the major version, as in Chrome/150.0.0.0, since version 101), or iOS 11 or 13, neither of which was the last version any iPhone could run. Each visit is judged by the versions out on the day it was made. The exceptions are the last Chrome for a system Google stopped updating, which those machines keep: 116 (macOS 10.13 and 10.14), 128 (macOS 10.15), 138 (macOS 11), 150 (macOS 12), 119 (Android 7), and 138 (Android 8 and 9). Windows 7 and 8.1 get no exception, because Chrome reports every Windows machine as Windows 10, so their last version, 109, looks the same as the scrapers that send it. Chromebooks are never counted this way, and a visit whose referrer is an AI assistant keeps its person label whatever version it claims.

When an AI assistant sent the visit, it counts as a visit from that assistant: the list is ChatGPT, Gemini, Perplexity, Claude, Copilot, Grok, DeepSeek, and Meta AI. Ranqo reads that from the referrer, and when there is none, from the campaign tag on the link (utm_source, which assistants such as ChatGPT add to the links they show) or from the in-app browser of an assistant's own app, such as the Gemini app on iPhone. Other human visits are sorted by referrer into Search, Social, Email, Direct (no referrer) and Other.

A referrer always wins over a campaign tag. Search engines index links that carry an assistant's tag, so a click on one from a results page is a search visit. A visit with no referrer, no campaign tag and no assistant app's browser still counts as Direct, because nothing in the request says where it came from.

A user agent is only a claim. A bot that sends a current browser's user agent still counts as a person, so automated traffic that imitates an up-to-date browser can appear in human visits.

Install it#

1. Open the install page

   On the Traffic page, press Set up Site Tracking. The button reads Continue install if a key already exists, and Add Site Tracking when Google Analytics is already delivering data. The Install Site Tracking page opens.

2. Generate a key

   Press Generate site key. The key starts with rk_live_ and belongs to this brand. Only the account owner can generate one. See Site keys.

3. Choose how to install

   Under Install method, choose AI prompt for a ready-made prompt to paste into a coding agent or chat assistant, Code snippet for the code itself, or Developer brief for a message to forward to whoever maintains your site. All three carry your key.

4. Deploy, then verify

   Deploy the change, press I've installed it — verify now, and visit a page on your site. See Verify your install.

Guides for each stack:

Limits#

Two plan limits apply:

• Keys: active keys are counted across all your brands, leaving out revoked keys and keys on deleted brands. The install page gives each brand one key, so through the dashboard the key limit is also how many brands can run Site Tracking at once.
• Visits are counted per brand per calendar month (UTC), bots and people together. The Site Tracking view shows how much of the allowance the brand has used, as N / M visits this month. The allowance is not enforced: visits above it are still accepted and recorded.
• History: the Traffic page shows as far back as your plan's history window. See Plans and limits.

Business is a custom plan with limits set together with you. Book a demo to shape one.

Related#