OAI-SearchBot
What OAI-SearchBot means
OAI-SearchBot is the OpenAI crawler that decides whether a website can be shown in ChatGPT's search answers. OpenAI states that sites opted out of it "will not be shown in ChatGPT search answers, though can still appear as navigational links."
More precisely
The robots.txt product token is `OAI-SearchBot`. The published user-agent string is `Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot`, with a variant carrying `robots.txt;` when fetching that file. IP ranges are at openai.com/searchbot.json. OpenAI warns that a robots.txt change can take roughly 24 hours to take effect in search.
Why it matters
This is the OpenAI token that controls visibility rather than training. OpenAI also states that ChatGPT-User, the agent used when a person asks ChatGPT to look at a page, "is not used to determine whether content may appear in Search."
Example
A publisher who wants ChatGPT to be able to surface its pages allows `OAI-SearchBot` in robots.txt, regardless of what it does with the training token.
Where this came from
Definitions here follow the specification or documentation that defines the term, not a paraphrase of another glossary.
-
Overview of OpenAI Crawlers — OpenAI
OAI-SearchBot is the search crawler, the consequence of opting out, its user-agent strings, the roughly 24 hour propagation note, and the statement that ChatGPT-User does not determine search inclusion.
Primary source · read 2026-09-03