Apple · Model training
Applebot-Extended
A robots.txt token, not a separate crawler. Apple fetches with Applebot; this token controls whether that content may train Apple's models.
- Operator
- Apple
- Purpose
- Model training
- Verification
- Published IP ranges
- Follows robots.txt
- Operator says it follows robots.txt
01Identify and verify
How it shows up in your logs
User-agent contains
Applebot-Extended
Matching is case-insensitive. Anyone can send this string, so a match alone says only what the request claims to be.
Is it really Applebot-Extended?
Plumb checks the request IP against the operator's published ranges.
Range checked: apple:applebot
02Publishers
How major publishers treat it
51%of 319 publishers on the leaderboard block
Applebot-Extended outright (163). 105 more close some sections. 159 name it explicitly in their robots.txt.03robots.txt
A rule you can paste
The group names Applebot-Extended exactly. A group for a different token from the same company does not cover it.
# Applebot-Extended (Apple) User-agent: Applebot-Extended Disallow: /
04Same operator
Apple's other agents
- ApplebotSearch engine