Google-Extended
Not a crawler - a control token. Google crawls with ordinary Googlebot and reads Google-Extended in robots.txt to decide whether that content may train Gemini models or ground Gemini answers. Disallowing it does not change your Google Search crawling or ranking.
What it feeds
Google-Extended is a permission token, not a crawler. Google describes it as a standalone product token publishers use to manage whether content Google crawls may train future Gemini models.[1] The fetch itself is done by ordinary Googlebot; this token only governs what Google may then do with what it already has. The generic GoogleOther crawler is documented on the same page and is a different thing: it fetches pages, this token does not.[2]
How to identify it
It never appears in server logs. Google states that it has no separate HTTP user agent and that crawling is done with existing Google user agent strings.[3] A log line claiming to be Google-Extended is therefore fabricated by definition.
Blocking and its consequences
Disallowing it is the cleanest AI training opt-out on the web, because it is decoupled from crawling: Search still crawls the site and still ranks it, and the pages stay fully eligible for ordinary results. The cost falls on Gemini grounding and training rather than on visibility in Search.
Verification
Operator documentation: https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers
Robots-only token - never appears in logs
A user agent is a claim, not a proof: any client can send any name. Where an operator publishes IP ranges, a reverse lookup is the only way to tell a real visit from an impersonation.
Seen by Baseline
Requests from this bot across sites tracked by Baseline, last 30 days. Aggregated over - sites, never reported per site.
Last updated 2026-09-05. Written and maintained by Baseline Labs.
