GPTBot
What GPTBot means
GPTBot is OpenAI's training crawler. OpenAI documents it as the robots.txt token that governs whether a site's content may be used to train OpenAI's generative AI foundation models. It does not govern whether the site appears in ChatGPT search, which is a separate token.
More precisely
The robots.txt product token is `GPTBot`. OpenAI publishes an example user-agent string of `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot`, and notes that a robots.txt marker may be added to the string when the bot fetches robots.txt itself. Source IP ranges are published at openai.com/gptbot.json. OpenAI states that disallowing GPTBot "indicates a site's content should not be used in training generative AI foundation models."
Why it matters
OpenAI describes allowing OAI-SearchBot while disallowing GPTBot as a supported configuration, so a training opt-out does not have to cost ChatGPT search presence. Sites that block every OpenAI token usually intended only the first of those two.
Example
A training opt-out that keeps ChatGPT search visibility is two blocks: `User-agent: GPTBot` with `Disallow: /`, and `User-agent: OAI-SearchBot` with `Allow: /`.
Where this came from
Definitions here follow the specification or documentation that defines the term, not a paraphrase of another glossary.
-
Overview of OpenAI Crawlers — OpenAI
GPTBot is the training crawler, its user-agent string and published IP endpoint, and OpenAI's statement that GPTBot and OAI-SearchBot settings are independent of each other.
Primary source · read 2026-09-03