Meta's AI bots

Meta documents its bots on one web crawlers page. Meta-ExternalAgent crawls to train foundation AI models and index content for Meta products, Meta-WebIndexer indexes pages for Meta AI search citations, Meta-ExternalFetcher fetches single links when a person asks Meta AI, and facebookexternalhit builds link previews, which it has done for years. Meta-ExternalAds checks ad landing pages.

Meta-ExternalAgent was quietly documented in July 2024, and Fortune reported it on Aug 20, 2024. It grew quickly from there, from about 0.9% to about 7.5% of crawler traffic between July 2024 and July 2025, and in Cloudflare Radar data for July 2026 it made up about 12.7% of AI-bot traffic, second only to Googlebot.

Meta's agent traffic is a different story. Across sites that use Verisoul, from Oct 29 to Nov 1, 2026 (Nov 1 partial), Meta's personal agent Muse made up 68% of the personal-agent sessions we named and produced roughly 20 times as many sessions as Meta-ExternalAds, even though Muse uses none of these tokens.

How Meta's bots show up on your site

All of them are HTTP crawlers and fetchers that run from Meta's own network, AS32934, each with a short declared user agent such as meta-externalagent/1.1 or facebookexternalhit/1.1. Meta does not publish a JSON IP list and instead points site owners to a RADb whois lookup for AS32934, warning that the IPs change often.

Why Meta's bots are easy to detect

Every bot declares a Meta token and all of the traffic comes from Meta's own network, so spoofing is easy to spot, which matters because scrapers often fake facebookexternalhit to get past sites that allowlist link previews. The friction is that there is no signature or IP file, so verification means a whois lookup you have to keep fresh.

Robots.txt only goes so far. Meta-ExternalAgent and Meta-WebIndexer respect it, with changes taking up to 24 hours, while Meta-ExternalFetcher may bypass it for user requests and facebookexternalhit may bypass it for security and integrity checks. Meta's in-app browsers carry real people and are not these bots at all, and Muse reports as desktop Chrome on Linux and exits through large CDN networks rather than consumer ISPs or AS32934, so these checks don't reach either.

How Verisoul detects Meta's bots

The bots are self-declared, so Verisoul reads the token and confirms the source IP belongs to Meta's own network before naming it. A Meta token from anywhere else is a spoofer. Muse is a separate case that we name by inference from its hosting network, device and behavior, and it has its own guide.

"agent": {
  "id": "meta_externalagent",
  "name": "Meta-ExternalAgent",
  "provider": "Meta",
  "type": "ai_crawler",
  "detection_method": "self_declared",
  "detection_signals": [
    "network",
    "behavioral"
  ]
}

• id is the bot, here meta_externalagent, with meta_externalfetcher, meta_webindexer, facebookexternalhit and meta_externalads for the others
• type is ai_crawler, which separates read-only crawlers from agents that act on a page
• detection_method is self_declared, because the bot named itself
• detection_signals is network, meaning the source IP was confirmed as Meta's own

Each bot has its own Agent Atlas entry, and the AI agent detection docs list every Meta id.

How to detect Meta's bots step by step

1. Turn on Verisoul and read the agent object in the API response
2. Keep facebookexternalhit allowed so link previews work, and decide on meta_externalagent by training policy
3. Block training with User-agent: meta-externalagent in robots.txt, and use a rule on meta_externalfetcher if you need it stopped
4. Treat any Meta token from outside Meta's network as spoofed, and handle Muse with its own rule

Should you block Meta's bots?

These bots read pages and build previews, so allowing them is a content decision rather than a fraud one. Most teams keep facebookexternalhit allowed, decide on Meta-ExternalAgent by training policy, and block any Meta token that doesn't come from Meta's network.