Amazon · Model training
Amazonbot
- Operator
- Amazon
- Purpose
- Model training
- Verification
- Reverse DNS only
- Follows robots.txt
- Operator says it follows robots.txt
01Identify and verify
How it shows up in your logs
User-agent contains
Amazonbot
Matching is case-insensitive. Anyone can send this string, so a match alone says only what the request claims to be.
Is it really Amazonbot?
The operator documents reverse DNS only. Plumb does not run reverse DNS, so these requests stay unverified.
Amazon documents Amazonbot by reverse DNS only (*.crawl.amazonbot.amazon).
02Publishers
How major publishers treat it
47%of 319 publishers on the leaderboard block
Amazonbot outright (149). 116 more close some sections. 154 name it explicitly in their robots.txt.03robots.txt
A rule you can paste
The group names Amazonbot exactly. A group for a different token from the same company does not cover it.
# Amazonbot (Amazon) User-agent: Amazonbot Disallow: /
04Same operator
Amazon's other agents
- Amzn-SearchBotAI search indexing