CCBot
Run by Common Crawl
How to detect CCBot
CCBot names itself in its user agent, but anyone can copy a user agent, so it is detected by confirming the user agent against other signals: source IP confirmed as the operator's own network; and how the session clicks, types, moves and navigates matches an automated agent.
- Detection method
- Self-declaredThe agent names itself in its user agent, and Verisoul confirms it with network, device or behavioral data. A user agent alone is never enough.
- Signals that identify it
- NetworkSource IP confirmed as the operator's own network
- DeviceNames itself as CCBot in its user agent, which we check against the rest of the session
- BehavioralHow the session clicks, types, moves and navigates matches an automated agent
Every signal Verisoul checks for CCBot
CCBot names itself in its user agent. Anyone can copy that, so we only label a session CCBot when the other signals check out too.
- Not used
Cryptographic
Not needed to identify it
- Used
Network
Source IP confirmed as the operator's own network
- Used
Device
Names itself as CCBot in its user agent, which we check against the rest of the session
- Used
Behavioral
How the session clicks, types, moves and navigates matches an automated agent
How often you'll see it
Uncommon
Under 1% of global crawler traffic in September 2026, per Cloudflare Radar
Questions about CCBot
How do you detect CCBot?
CCBot names itself in its user agent, but anyone can copy a user agent, so it is detected by confirming the user agent against other signals: source IP confirmed as the operator's own network; and how the session clicks, types, moves and navigates matches an automated agent.
What is CCBot?
CCBot is Common Crawl's crawler for its open web dataset, which is widely used to train AI models.
Does CCBot identify itself?
Partly. CCBot names itself in its user agent, but user agents are easy to copy, so the name alone is not proof.
Can a bot pretend to be CCBot?
A bot can copy CCBot's user agent or markers, but the network, device and behavior signals will not match, so it is not labeled CCBot.
How common is CCBot?
Uncommon. Under 1% of global crawler traffic in September 2026, per Cloudflare Radar.
What does the Verisoul API return for CCBot?
Every session Verisoul identifies as CCBot carries an agent block with id "ccbot", name, provider, type "ai_crawler", detection_method "self_declared" and the detection_signals used (network, behavioral). Build rules on the id, type and detection method.