How to detect Anthropic's bots

You detect Anthropic's bots by matching the user agent token each one declares (ClaudeBot, Claude-SearchBot or Claude-User) and checking that the source IP appears in the list Anthropic publishes at claude.com/crawling/bots.json, since a Claude token from any other IP is spoofed. All three identify themselves and follow robots.txt, and ClaudeBot was the largest single AI crawler on Cloudflare in July 2026 at 16.5% of AI-bot traffic, though none of them signs its requests yet. Verisoul verifies each bot against Anthropic's network and returns it as agent id claudebot, claude_searchbot or claude_user.

Detection method

Self-declaredThe agent names itself in its user agent, and Verisoul confirms it with network, device or behavioral data. A user agent alone is never enough.

Signals that identify it

NetworkSource IP confirmed as the operator's own network

Anthropic's crawler help page lists three bots. ClaudeBot collects public pages that may be used for training, Claude-SearchBot indexes content to improve Claude's search results, and Claude-User fetches a page when a Claude user asks a question that needs it. Anthropic updated the page on Feb 20, 2026 to split out the three bots, and the older anthropic-ai and Claude-Web tokens are now deprecated.

ClaudeBot has no published launch date, though it was already crawling by July 2024, when iFixit's CEO reported nearly a million hits in a day. Anthropic said it respects robots.txt, and the crawling stopped after iFixit added a Crawl-delay, which is still the first lever most site owners reach for.

It is also one of the busiest crawlers on the web. In Cloudflare Radar data for July 2026, ClaudeBot made up 16.5% of AI-bot traffic, down from about 20% in June, with Claude-SearchBot at 3.5%. That share has swung a lot, since Cloudflare's 2025 crawler review had ClaudeBot falling from 11.7% to 5.4% of crawler traffic between May 2024 and May 2025.

How Anthropic's bots show up on your site

All three are server-side HTTP fetchers that run from the prefixes Anthropic publishes in bots.json, which held 14 prefixes on Nov 1, 2026. Each request declares its token, such as ClaudeBot/1.0 or Claude-User/1.0, while the full Claude-SearchBot string is not documented. Anthropic says the bots honor robots.txt and the non-standard Crawl-delay directive, set per subdomain.

Check	What Anthropic's bots do
User agent	Declared tokens ClaudeBot, Claude-SearchBot, Claude-User
Web Bot Auth signature	None, Anthropic documents only an IP file
IP ranges	Published at claude.com/crawling/bots.json
robots.txt	Followed by all three, including Crawl-delay

Why Anthropic's bots are easy to detect

Every bot declares a named token and one IP file covers all three, so spoofed tokens are easy to reject. Since there is no signature yet, verification rests on that IP list alone, and Anthropic was not among the first Cloudflare signed agents. Anthropic prefers robots.txt over IP blocks, because blocking its IPs can stop the bots from reading your rules in the first place.

These checks stop at Claude's agentic browsing. Claude in Chrome and the Claude desktop app's browser run inside a real browser without these tokens or IPs, which is why Verisoul names them separately. Across sites that use Verisoul, from Oct 29 to Nov 1, 2026 (Nov 1 partial), the Claude desktop app browser made up 3% of the personal-agent sessions we named.

How Verisoul detects Anthropic's bots

The bots are self-declared, so Verisoul reads the token in the user agent and confirms the source IP belongs to Anthropic's own network before naming it. A Claude token from any other network is a spoofer and is not returned as Anthropic.

API response

"agent": {
  "id": "claudebot",
  "name": "ClaudeBot",
  "provider": "Anthropic",
  "type": "ai_crawler",
  "detection_method": "self_declared",
  "detection_signals": [
    "network",
    "behavioral"
  ]
}

id is the bot, here claudebot, with claude_searchbot and claude_user for the other two
type is ai_crawler, which separates read-only crawlers from agents that act on a page
detection_method is self_declared, because the bot named itself
detection_signals is network, meaning the source IP was confirmed as Anthropic's own

Each bot has its own Agent Atlas entry, and the AI agent detection docs list every id the API returns.

01Claude user asks about your page
02Claude-User fetches it from Anthropic's IPs
03Request carries the Claude-User token
04Your server matches token and IP

How Anthropic's bots reaches your site

How to detect Anthropic's bots step by step

Turn on Verisoul and read the agent object in the API response
Allow or block claudebot, claude_searchbot and claude_user one by one to match your content policy
Set robots.txt per bot on each subdomain, and use Crawl-delay if ClaudeBot is heavy
Treat any Claude token that fails the IP check as a spoofer

Should you block Anthropic's bots?

These bots only read pages, so allowing them is a content decision rather than a fraud one. Many teams allow Claude-SearchBot and Claude-User for visibility in Claude, decide on ClaudeBot by training policy, and block any Claude token that fails the IP check.

Questions about Anthropic's bots

How do I block ClaudeBot?

Add a User-agent: ClaudeBot rule with Disallow in robots.txt on each subdomain. Anthropic says ClaudeBot follows it and prefers this to IP blocking, which can stop the bot from reading your rules.

Does ClaudeBot respect robots.txt?

Anthropic says all three of its bots honor robots.txt, including the Crawl-delay directive.

What is the difference between ClaudeBot and Claude-User?

ClaudeBot crawls public pages that may be used for training, while Claude-User fetches one page when a person asks Claude a question that needs it. You can allow one and block the other in robots.txt.

Why is ClaudeBot hitting my site so much?

ClaudeBot crawls broadly for training and was 16.5% of AI-bot traffic on Cloudflare in July 2026. In 2024 iFixit reported nearly a million hits in a day, and the crawling stopped after it added a Crawl-delay, which Anthropic honors.

What IP addresses does ClaudeBot use?

Anthropic lists them at claude.com/crawling/bots.json, which covers ClaudeBot, Claude-SearchBot and Claude-User. Treat any Claude user agent from an IP outside that file as spoofed.

Anthropic's bots