OpenAI · Model training
GPTBot
- Operator
- OpenAI
- Purpose
- Model training
- Verification
- Published IP ranges
- Follows robots.txt
- Operator says it follows robots.txt
01Identify and verify
How it shows up in your logs
User-agent contains
GPTBot
Matching is case-insensitive. Anyone can send this string, so a match alone says only what the request claims to be.
Is it really GPTBot?
Plumb checks the request IP against the operator's published ranges.
Range checked: openai:gptbot
Signing keys: openai.com
02Publishers
How major publishers treat it
53%of 319 publishers on the leaderboard block
GPTBot outright (169). 105 more close some sections. 180 name it explicitly in their robots.txt.03robots.txt
A rule you can paste
The group names GPTBot exactly. A group for a different token from the same company does not cover it.
# GPTBot (OpenAI) User-agent: GPTBot Disallow: /
04Same operator
OpenAI's other agents
- OAI-SearchBotAI search indexing
- ChatGPT-UserUser-triggered fetch