Meta · User-triggered fetch
Meta-ExternalFetcher
- Operator
- Meta
- Purpose
- User-triggered fetch
- Verification
- None
- Follows robots.txt
- Operator says it may ignore robots.txt
01Identify and verify
How it shows up in your logs
User-agent contains
meta-externalfetcher
Matching is case-insensitive. Anyone can send this string, so a match alone says only what the request claims to be.
Is it really Meta-ExternalFetcher?
The operator publishes no ranges or keys, so any request using this name is unverified.
Meta documents no crawler IP ranges.
02Publishers
How major publishers treat it
33%of 319 publishers on the leaderboard block
Meta-ExternalFetcher outright (104). 150 more close some sections. 99 name it explicitly in their robots.txt.03robots.txt
A rule you can paste
The group names meta-externalfetcher exactly. A group for a different token from the same company does not cover it.
Meta says this fetcher may bypass robots.txt because a person asked for the page. A rule here records your preference; enforcing it takes a block at your CDN or firewall.
# Meta-ExternalFetcher (Meta) User-agent: meta-externalfetcher Disallow: /
04Same operator
Meta's other agents
- Meta-ExternalAgentModel training
- FacebookBotModel training