Grok fetches web pages to answer questions, to run DeepSearch research reports that launched with Grok 3 in Feb 2025, and to summarize links people paste. Users trigger it on grok.com, in the X app and through the xAI API. Unlike OpenAI, Anthropic, Google, Perplexity and Meta, xAI has published no crawler page for these fetches, so there is no official token, IP list or robots.txt statement to check against.
Names like GrokBot, xAI-SearchBot, Grok-DeepSearch and GrokAgent circulate in third-party bot lists, but xAI has never confirmed them. The one documented xAI product that acts on websites is Grok Bot, an always-on agent that launched on Aug 11, 2026 and works inside web apps for a person, and it is easy to confuse with the unconfirmed GrokBot crawler name.
How Grok's bots show up on your site
Grok's search fetches carry no declared identity, so to your server they look like ordinary browser or HTTP-client requests from networks xAI does not publish. Observed fetches have used Go-http-client or outdated Chrome and iPhone user agents from datacenter proxies, and a single Grok prompt can fan out into a burst of requests across many residential and mobile IPs with rotating browser user agents. Grok Bot is different. Each member gets a cloud VM with a browser, and its traffic leaves through shared static egress IPs that xAI gives customers on request, with an option to route through the user's own desktop. xAI's docs say the Bot should hand CAPTCHAs and blocks back to the person rather than work around them.
| Check | What Grok's bots do |
|---|---|
| User agent | No confirmed token for search fetches |
| Web Bot Auth signature | Grok Bot signs, search fetches don't |
| IP ranges | None public, Grok Bot ranges on request |
| robots.txt | No policy published by xAI |
Why Grok's fetchers are hard to detect
With no token, no IP list and no robots.txt policy, there is nothing to match on, which means a Disallow rule for a Grok name is a preference at best. A single request from Grok's fetchers can look like any visitor, and the burst pattern across residential IPs looks a lot like proxy-based scraping, so detection has to come from the browser and the pattern of the session rather than from the request headers. Cloudflare Radar publishes no separate traffic share for Grok's fetchers, which makes their volume hard to size from outside.
Grok Bot is the easier half, because a signed request can be checked against xAI's key, and its cloud browser carries a virtual machine device profile that a real person's laptop doesn't. Cloudflare Radar's bot directory has an entry for Grok Bot, which lets Cloudflare customers allow or block it by name.
How Verisoul detects Grok Bot
Verisoul names Grok Bot by verifying its signature, which is the strongest kind of agent identification because it can't be borrowed. We back that with the session itself, where a cloud-hosted browser with a virtual machine device profile and automated clicks, typing and page flow confirm it is the agent and not the person. Grok's unnamed search fetchers don't get an agent name, while Verisoul still flags them as automated through bot score.
"agent": {
"id": "grok_bot",
"name": "Grok Bot",
"provider": "xAI",
"type": "ai_assistant",
"detection_method": "signed",
"detection_signals": [
"cryptographic",
"behavioral"
]
}idisgrok_bot, the stable value to write rules against for xAI's agenttypeisai_assistant, because Grok Bot acts on sites for a real persondetection_methodissigned, meaning the request's signature was verified cryptographicallydetection_signalslistscryptographic(the signature),device(a cloud VM browser) andbehavioral(automated page flow)
See the Grok Bot entry in Agent Atlas and the AI agent detection docs for the full response format.
- 01Person assigns Grok Bot a task
- 02Grok Bot's cloud browser opens your site
- 03It sends a signed request
- 04Your page sees a cloud VM browser
How to detect Grok's bots step by step
- Turn on Verisoul and read the
agentobject in the API response - Write a rule for
grok_boton sign-up, payouts and account changes - Rate limit or block unnamed sessions with a high bot score on content you protect
- Ask xAI for Grok Bot's egress ranges if you need an allowlist
Should you block Grok's bots?
Grok Bot works for a real person, so many teams let it browse and send it to review at sign-up, payouts and account changes. Grok's unnamed search fetchers are a scraping question for content sites, so rate limit or block them by bot score where it matters.