What does Google-Extended actually control?
A control token with no user agent of its own, covering a narrower product set than most people assume.
What does Google-Extended actually control?
Google-Extended is a robots.txt control token, not a crawler. Google states that it has no separate HTTP request user agent string and that crawling is performed with existing Google user agents. Its documented scope is whether crawled content may be used for training future Gemini models that power Gemini Apps and the Vertex AI API for Gemini, and for grounding in Gemini Apps and Grounding with Google Search on Vertex AI. Google also states directly that Google-Extended does not impact a site's inclusion in Google Search and is not used as a ranking signal there.
It does not crawl anything
This is the part that surprises people, so it goes first. Google writes: "Google-Extended doesn't have a separate HTTP request user agent string. Crawling is done with existing Google user agent strings; the robots.txt user-agent token is used in a control capacity."
You will therefore never see Google-Extended in an access log. Adding a Disallow for it changes how already-crawled data may be used downstream, not whether a fetch happens.
Apple built the same mechanism with Applebot-Extended, which likewise performs no crawling and only governs the use of data that Applebot already collected. Two vendors independently landed on the pattern of a use-control token rather than a second crawler.
The exact scope Google enumerates
Quoted in full, because paraphrasing this sentence is how the confusion starts: "Google-Extended is a standalone product token that web publishers can use to manage whether content Google crawls from their sites may be used for training future generations of Gemini models that power Gemini Apps and Vertex AI API for Gemini and for grounding (providing content from the Google Search index to the model at prompt time to improve factuality and relevancy) in Gemini Apps and Grounding with Google Search on Vertex AI."
Note that the token covers two things, not one. Training is the obvious one. Grounding is the second, and Google defines grounding inline as supplying content from the Search index to the model at prompt time.
Note also what is named: Gemini Apps, the Vertex AI API for Gemini, and Grounding with Google Search on Vertex AI. That is the complete published list.
| Token | Is it a crawler? | Documented effect |
|---|---|---|
| Googlebot | Yes | Affects Google Search including Discover and all Search features, plus Images, Video and News |
| Google-Extended | No, control token only | Training and grounding for Gemini Apps and Vertex AI API for Gemini; no effect on Search inclusion or ranking |
| Google-CloudVertexBot | Yes | Site-owner-requested crawls for building Vertex AI Agents; no effect on Search or other products |
| GoogleOther | Yes | Generic fetching by various product teams; preferences do not affect any specific product |
| Google-Safety | Yes, special case | Documented as ignoring robots.txt rules |
What it explicitly does not touch
Google prints the negative plainly here, which is unusual and worth using: "Google-Extended does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search."
So disallowing this token is not an SEO decision. It is a decision about whether Google may use your content in the Gemini product family.
AI Overviews and AI Mode: read the join, do not take our word for it
The most common question about this token is whether it removes a site from AI Overviews or AI Mode. Google never answers that in one sentence, and anyone who tells you it does is compressing two documents into a claim Google did not make.
Here is the first half. The Google-Extended entry enumerates Gemini Apps, Vertex AI API for Gemini, and Grounding with Google Search on Vertex AI. AI Overviews and AI Mode do not appear in that enumeration.
Here is the second half, from Google's AI features guidance: "AI is built into Search and integral to how Search functions, which is why robots.txt directives for Googlebot is the control for site owners to manage access to how their sites are crawled for Search. To limit the information shown from your pages in Search, use nosnippet, data-nosnippet, max-snippet, or noindex controls. To limit AI training and grounding in some of Google's other systems, read more about Google-Extended."
Putting those together gets you to: Googlebot and the snippet controls govern Search AI features, and Google-Extended governs a different product set. We label that an inference across two pages, because it is one. Google's own phrase is "in some of Google's other systems", and Google does not enumerate which systems are in that set and which are not.
One related fact is stated outright and is easy to act on: "To be eligible to be shown as a supporting link in AI Overviews or AI Mode, a page must be indexed".
A documentation move that breaks older citations
Google relocated its crawler documentation out of the /search/docs/crawling-indexing/ path into /crawling/docs/crawlers-fetchers/. Old URLs still resolve by redirect.
If you are checking a third-party article about this token, look at where its links land. A citation to the old path is a citation to a redirect, which is a decent signal of when the piece was written and whether anyone has revisited it.
The common crawlers page carried a last-updated stamp of 2026-07-14 UTC when we read it, and the AI features page 2025-12-10 UTC. Those two dates are eight months apart, which is worth holding in mind when you join claims across them.
What is actually established, and how
Sorted by how strong the evidence is, not by how convenient it is.
| Claim | Basis |
|---|---|
| Google-Extended has no separate HTTP request user agent string, and crawling is done with existing Google user agents. | Documented by the platform |
| Google-Extended governs whether crawled content may train future Gemini models powering Gemini Apps and the Vertex AI API for Gemini. | Documented by the platform |
| Google-Extended also governs grounding, defined by Google as providing content from the Search index to the model at prompt time, in Gemini Apps and Grounding with Google Search on Vertex AI. | Documented by the platform |
| Google states that Google-Extended does not impact inclusion in Google Search and is not a ranking signal there. | Documented by the platform |
| A page must be indexed to be eligible as a supporting link in AI Overviews or AI Mode. | Documented by the platform |
| Google-Extended does not control AI Overviews or AI Mode. This is assembled from two separate Google pages and is our inference, not a sentence Google publishes. | Inference, not documentation |
| Which systems Google means by the phrase about limiting training and grounding in some of its other systems. | Not publicly documented |
What nobody can currently tell you
Stated because the alternative is implying a certainty that does not exist.
We could find no single primary-source sentence stating that Google-Extended does not control AI Overviews. The conclusion requires joining the token scope on one page to the Search AI guidance on another, and we present both quotes so a reader can check the join rather than trust it.
The hedge "in some of Google's other systems" is not expanded anywhere we could find. Google neither lists the covered systems nor lists the excluded ones.
Because Google-Extended leaves no trace in server logs, a site owner has no way to observe whether the directive was read or honoured. There is nothing to measure from outside.
Google does not state whether content crawled before a Google-Extended Disallow was added remains usable for the purposes the token covers.
The Google IP ranges JSON referenced by the crawlers page was identified from the page source but its body was not fetched or validated during this research.
What people get wrong about this
-
Blocking Google-Extended keeps you out of AI Overviews.
What is actually the case
AI Overviews and AI Mode are not in the enumerated scope of that token, and Google names Googlebot plus snippet controls as the levers for AI features inside Search.
-
Google-Extended will show up in your logs so you can see how often it crawls.
What is actually the case
It sends no requests. Google states it has no user agent string of its own and functions purely as a control token.
-
Disallowing Google-Extended risks your rankings.
What is actually the case
Google states outright that this token does not affect inclusion in Google Search and is not used as a ranking signal.
-
It only affects training.
What is actually the case
The published scope covers grounding as well, which Google defines as feeding Search index content to the model at prompt time in the named Gemini products.
Questions we get asked constantly
Where do I put a Google-Extended rule?
In robots.txt, addressed to the Google-Extended user-agent token like any other directive. It is read as a control instruction rather than matched against an incoming request header.
Does Google-Extended cover Google-CloudVertexBot?
They are separate entries. Google documents Google-CloudVertexBot as handling crawls requested by site owners for building Vertex AI Agents, with no effect on Search or other products.
If I block Google-Extended, will Gemini still mention my business?
Google does not answer that. The token addresses training and grounding for the named products, and no Google page describes the downstream effect on what an assistant says about a business.
Is there an equivalent token at other companies?
Apple publishes Applebot-Extended on the same principle, a non-crawling token that only decides how already-crawled data may be used.
Where this came from
Every factual claim above traces to one of these. Each entry says what it supports and the date it was read, because platform documentation changes without notice.
-
Google common crawlers — Google
Google-Extended has no user agent string, its scope covers Gemini training and grounding, and it does not affect Search inclusion or ranking. Page last updated 2026-07-14 UTC.
Primary source · read 2026-09-03
developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers
-
Google Search AI features and your website — Google
Googlebot directives are named as the control for Search crawling, snippet controls limit what is shown, and a page must be indexed to appear as a supporting link in AI Overviews or AI Mode. Page last updated 2025-12-10 UTC.
Primary source · read 2026-09-03
-
Google crawlers and fetchers: Google-CloudVertexBot — Google
Google-CloudVertexBot handles site-owner-requested crawls for Vertex AI Agents and has no effect on Google Search or other products.
Primary source · read 2026-09-03
developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers
-
Google special-case crawlers — Google
The Google-Safety user agent is documented as ignoring robots.txt rules.
Primary source · read 2026-09-03
developers.google.com/crawling/docs/crawlers-fetchers/google-special-case-crawlers
-
About Applebot — Apple
Applebot-Extended is the parallel non-crawling control token, used only to determine how data crawled by Applebot may be used.
Primary source · read 2026-09-03