← AI agent directory

Google · User-triggered fetch

Google-Agent

Operator
Google
Purpose
User-triggered fetch
Verification
Published IP ranges
Follows robots.txt
Operator says it may ignore robots.txt

Operator documentation: https://developers.google.com/search/docs/crawling-indexing/google-user-triggered-fetchers

01Identify and verify

How it shows up in your logs

User-agent contains

  • Google-Agent

Matching is case-insensitive. Anyone can send this string, so a match alone says only what the request claims to be.

Is it really Google-Agent?

Plumb checks the request IP against the operator's published ranges.

Ranges checked: google:user-fetchers, google:googlebot, google:special, google:user-fetchers-google

02Publishers

How major publishers treat it

The leaderboard does not grade Google-Agent yet, so there is no publisher figure for it.

03robots.txt

A rule you can paste

The group names Google-Agent exactly. A group for a different token from the same company does not cover it.

Google says user-triggered fetchers generally ignore robots.txt, since a person asked for the page. A rule here records your preference; enforcing it takes a block at your CDN or firewall.

# Google-Agent (Google)
User-agent: Google-Agent
Disallow: /

04Same operator

Google's other agents

Google-Agent (Google): what it is and how to block it · Plumb