Site Tracking
Site Tracking for Express
Send AI crawler visits and AI referrals from an Express app to Ranqo with one middleware function, with notes for Fastify, Hono and Koa.
On an Express app, Site Tracking is one middleware function. It reports each page request to Ranqo after your response has gone out, so your visitors never wait on it.
Copy your site key
On the Traffic page, press Set up Site Tracking, then Generate site key on the Install Site Tracking page. Copy the key. It starts with
rk_live_.Add the key to your environment
.envRANQO_SITE_KEY=rk_live_YOUR_SITE_KEYSet the same variable in your hosting provider's environment for every environment you want tracked. Keep it out of your repository.
Add the middleware file
Save this as
ranqo-intake.ts(or.js, without the type imports). It uses the globalfetch, available from Node.js 18.ranqo-intake.ts// ranqo-intake.ts — wire with: app.use(ranqoIntake); import type { Request, Response, NextFunction } from 'express'; const SKIP_EXT = /\.(png|jpg|jpeg|svg|gif|webp|ico|css|js|mjs|map|woff2?|ttf|otf|eot|txt|xml|json)$/i; const TRACK_AI = /^\/(robots\.txt|llms(-full)?\.txt|sitemap(-.*)?\.xml|ai\.txt|\.well-known\/.*)$/; // OPTIONAL: skip self-tracking on subdomains you don't want counted as // "marketing site visitors" (e.g. your authenticated dashboard). const SKIP_HOSTS = new Set<string>([ // 'app.yoursite.com', // 'admin.yoursite.com', ]); // The header your CDN writes with the visitor's country: 'cf-ipcountry' // behind Cloudflare, 'cloudfront-viewer-country' behind CloudFront. On Vercel // this is ignored: Vercel writes x-vercel-ip-country itself. Left empty, no // country is sent. Name only a header your CDN sets on every request: a visitor // can send any header themselves. const COUNTRY_HEADER = ''; export function ranqoIntake(req: Request, res: Response, next: NextFunction) { // Forward immediately — fire-and-forget happens after the handler returns. next(); res.on('finish', () => { // Only track GET. POST/PUT/DELETE/HEAD/OPTIONS never represent a page view. if (req.method !== 'GET') return; // Skip prefetch / RSC requests (matters when Express is proxying or // hosting a Next.js app — every <Link> hover would otherwise count). const secFetchDest = req.get('sec-fetch-dest'); if (secFetchDest && secFetchDest !== 'document') return; if (req.get('next-router-prefetch')) return; if (req.get('next-router-segment-prefetch')) return; if (req.get('rsc') === '1') return; const secPurpose = req.get('sec-purpose'); if (secPurpose && secPurpose.startsWith('prefetch')) return; if (req.get('purpose') === 'prefetch') return; const path = req.path; if (path.startsWith('/api/')) return; if (SKIP_EXT.test(path) && !TRACK_AI.test(path)) return; if (SKIP_HOSTS.has(req.get('host') || '')) return; const params = new URLSearchParams({ url: `${req.protocol}://${req.get('host')}${req.originalUrl}`, userAgent: req.get('user-agent') || '', ref: req.get('referer') || '', ip: visitorIp(req), country: visitorCountry(req), websiteKey: process.env.RANQO_SITE_KEY || '', }); fetch(`https://app.ranqo.ai/api/v1/intake/pageview?${params}`).catch(() => {}); }); } // req.ip is the connection's own address or, once app.set('trust proxy', ...) // names your proxies, the X-Forwarded-For entry they added, read from the // right: 'loopback' for nginx or Caddy on this machine, 'loopback, uniquelocal' // to add a load balancer on your private network. `true` trusts every entry, // including the ones a visitor writes, so it is read here as no proxy at all. function visitorIp(req: Request): string { if (req.app.get('trust proxy') === true) return req.socket.remoteAddress || ''; return req.ip || ''; } function visitorCountry(req: Request): string { const header = process.env.VERCEL ? 'x-vercel-ip-country' : COUNTRY_HEADER; return (header && req.get(header)) || ''; }Register it before your routes
Import it and call
app.use(ranqoIntake)after yourtrust proxysetting and before your routes and anyexpress.staticmiddleware. Express runs middleware in order, and a route or static handler that sends a response does not pass the request on, so middleware registered after it never sees that request.app.tsimport express from "express"; import { ranqoIntake } from "./ranqo-intake"; const app = express(); app.set("trust proxy", "loopback"); // nginx or Caddy on this machine; see below app.use(ranqoIntake); app.use(express.static("public")); // ...your routesDeploy and verify
Deploy, then follow Verify your install. The install page in the dashboard watches for your first real visit and confirms it when it arrives.
What the middleware does
- Adds no latency. It calls
next()straight away and sends the report to Ranqo when the response has finished, without waiting for Ranqo's answer. A failed report is ignored. - Counts page views only. It skips anything but
GET, paths under/api/, and the prefetch and framework data requests a Next.js front end makes when it is served through Express. - Keeps the files AI crawlers read first.
robots.txt,llms.txt,sitemap.xmland/.well-known/*are reported even though other files with those extensions are skipped as static assets. - Sends the visitor's country on Vercel, from Vercel's
x-vercel-ip-countryheader. Anywhere else, setCOUNTRY_HEADERto your CDN's country header, such ascf-ipcountrybehind Cloudflare; until then, visits are recorded without a country. - Sends an address the visitor cannot choose. It reports
req.ip: the connection's own address or, when the connection comes from a proxy yourtrust proxysetting names, the address that proxy added toX-Forwarded-For. See The visitor's address.
The visitor's address
Ranqo checks each crawler visit against the IP ranges the crawler's operator publishes, so the report has to carry an address the visitor cannot choose. The first entry of X-Forwarded-For is not one: anyone can send the header with Googlebot's address in it, and a proxy adds its own entry to the right of whatever arrived.
The middleware reports req.ip, which follows Express's trust proxy setting:
- Nothing set: the connection's own address. That is the visitor when Express takes requests straight from the internet, and your proxy when one sits in front, which makes genuine crawlers look unverified.
- Your proxies named:
"loopback"for nginx or Caddy on the same machine,"loopback, uniquelocal"to add a load balancer or another container on your private network. Express then readsX-Forwarded-Forfrom the right, past those addresses, to the first one that is not yours. Each proxy must add the address it received to the header; in nginx,proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;. true: never. It trusts every entry, soreq.ipbecomes the first one, which the visitor writes. When the middleware seestrue, it reports the connection's own address instead.
A CDN that connects to your server from public addresses, such as Cloudflare, needs its published ranges added to the setting. Express also accepts a number of proxies, but a number trusts whoever connects, so a visitor who reaches your server around its proxies can still choose the address.
Fastify, Hono and Koa
The same logic works in any Node framework: after the response is sent, apply the same checks, build the same query string and call the intake endpoint without awaiting it.
- Fastify: put the body of the
res.on('finish', ...)handler in anonResponsehook, reading headers fromrequest.headers. - Hono and Koa: in
app.use(async (c, next) => { ... })(Koa:(ctx, next)), callawait next()first, then run the checks and fire the request. - The visitor's address: take it from the framework's own proxy setting, never from the first
X-Forwarded-Forentry. In Fastify, settrustProxyto your proxies' addresses (such as"127.0.0.1"), nottrue, and readrequest.ip. In Koa,app.proxy = truemakesctx.ipthe first entry unlessapp.maxIpsCountis also set to your number of proxies, and Koa does not check the proxies' addresses, so keep the app reachable only through them. In Hono, read the connection's address with thegetConnInfohelper for your runtime.
On a serverless or edge runtime, a request still in flight when the handler returns can be cut off. Use the platform's way of keeping work alive after the response, as the Cloudflare Worker install does with waitUntil.