← AI agent directory

Anthropic · User-triggered fetch

Claude-User

Operator
Anthropic
Purpose
User-triggered fetch
Verification
Web Bot Auth signature
Follows robots.txt
Operator says it follows robots.txt

Operator documentation: https://support.anthropic.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler

01Identify and verify

How it shows up in your logs

User-agent contains

  • Claude-User

Matching is case-insensitive. Anyone can send this string, so a match alone says only what the request claims to be.

Is it really Claude-User?

Plumb checks the Web Bot Auth signature against the operator's published key directory.

Signing keys: www.anthropic.com

Anthropic publishes no IP ranges; ClaudeBot / Claude-User / Claude-SearchBot verify only by signature.

02Publishers

How major publishers treat it

28%of 319 publishers on the leaderboard block Claude-User outright (90). 158 more close some sections. 96 name it explicitly in their robots.txt.

From the scan of 21 November 2026 · see every publisher

03robots.txt

A rule you can paste

The group names Claude-User exactly. A group for a different token from the same company does not cover it.

# Claude-User (Anthropic)
User-agent: Claude-User
Disallow: /

04Same operator

Anthropic's other agents

Claude-User (Anthropic): what it is and how to block it · Plumb