Googlebot
What Googlebot means
Googlebot is Google's main search crawler. Google says crawling preferences addressed to it "affect Google Search (including Discover and all Google Search features), as well as other products such as Google Images, Google Video, Google News, and Discover."
More precisely
It crawls with a smartphone and a desktop user-agent string, both ending in `(compatible; Googlebot/2.1; +http://www.google.com/bot.html)`, where the `Chrome/W.X.Y.Z` fragment is a version placeholder. Requests can be verified by reverse DNS under googlebot.com or against the ranges in Google's common-crawlers.json. Google also documents that a page "must be indexed" to be eligible as a supporting link in AI Overviews or AI Mode.
Why it matters
Because AI is built into Search, Googlebot rules are the ones that reach AI Overviews and AI Mode, not Google-Extended. One side effect is worth knowing: Apple documents that if robots instructions mention Googlebot but not Applebot, Applebot follows the Googlebot rules.
Example
Snippet length in AI features is limited with nosnippet, data-nosnippet or max-snippet rather than by adjusting the Googlebot crawl rules.
Where this came from
Definitions here follow the specification or documentation that defines the term, not a paraphrase of another glossary.
-
Google's common crawlers — Google
What Googlebot crawling preferences affect, the published user-agent strings and version placeholder, and the reverse DNS and IP range verification methods. Page last updated 2026-07-14 UTC.
Primary source · read 2026-09-03
developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers
-
AI features and your website — Google
Googlebot directives are the control for how sites are crawled for Search, and a page must be indexed to be eligible as a supporting link in AI Overviews or AI Mode.
Primary source · read 2026-09-03
-
About Applebot — Apple
Applebot follows Googlebot instructions when robots instructions mention Googlebot but not Applebot.
Primary source · read 2026-09-03