What is the difference between GPTBot and OAI-SearchBot?
Same company, two crawlers, two robots.txt tokens, and two separate IP range endpoints.
What is the difference between GPTBot and OAI-SearchBot?
GPTBot and OAI-SearchBot are two distinct OpenAI crawlers with two distinct robots.txt tokens. OpenAI writes that GPTBot "is used to crawl content that may be used in training our generative AI foundation models". It writes that OAI-SearchBot "is used to surface websites in search results in ChatGPT's search features". They send different user-agent strings, they publish separate IP range files at openai.com/gptbot.json and openai.com/searchbot.json, and a robots.txt rule naming one token has no effect on the other. OpenAI also runs ChatGPT-User for pages fetched because a person asked something.
What each one is for
GPTBot exists to gather training material. OpenAI states: "GPTBot is used to make our generative AI foundation models more useful and safe."
OAI-SearchBot exists to populate search. It is the agent behind whether a site can show up when someone uses ChatGPT to look something up, and OpenAI names it as the token for managing search opt-outs and automatic crawl.
The practical consequence: these two answer different questions about your site. One is about whether your content contributes to a model. The other is about whether your content can be found and cited at answer time.
How to tell them apart in a log file
The published user-agent strings differ in shape as well as in token. GPTBot sends a short string. OAI-SearchBot sends a full Chrome-style string with the token appended.
OpenAI documents an extra wrinkle that is genuinely useful when reading logs: when fetching robots.txt, it may add a robots.txt marker to the user-agent string, so those requests can be told apart from content requests even when the log does not record paths.
Version numbers in these strings change. Match on the token substring, never on the full string.
| Token | Published user-agent string | IP range endpoint |
|---|---|---|
| GPTBot | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot | https://openai.com/gptbot.json |
| OAI-SearchBot | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot | https://openai.com/searchbot.json |
| ChatGPT-User | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot | https://openai.com/chatgpt-user.json |
| OAI-AdsBot | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; OAI-AdsBot/1.0; +https://openai.com/adsbot | https://openai.com/adsbot.json |
The third agent people forget
ChatGPT-User is not a crawler in the ordinary sense. OpenAI describes it as visiting a page when a user asks ChatGPT or a Custom GPT a question.
Two properties make it distinct. OpenAI says it is not used for crawling the web in an automatic fashion, and it says: "ChatGPT-User is not used to determine whether content may appear in Search."
So a rule aimed at ChatGPT-User does not manage search inclusion, and OpenAI explicitly redirects site owners to OAI-SearchBot for that.
A fourth agent, OAI-AdsBot, only visits pages submitted as ads on ChatGPT, and OpenAI states that the data it collects is not used to train foundation models.
Where the two tokens are treated as equivalent, and where they are not
Both are documented as obeying robots.txt. Both publish IP ranges you can verify against. Both belong to OpenAI, so a network-level block on OpenAI address space would hit both indiscriminately, which is one reason IP blocking is a poor substitute for robots.txt.
They diverge on scope, on consequence, and on how quickly a change registers. OpenAI attaches its 24-hour lag note specifically to search results.
They also diverge in how third parties classify them. Cloudflare files GPTBot under AI Crawler and OAI-SearchBot under AI Search, which is a fair reflection of the operator documentation.
A note on the URL in the user-agent string
GPTBot advertises openai.com/gptbot in its user agent. That address returns HTTP 403 to an ordinary browser request, so it is an identifier rather than a page you can read.
The documentation lives at developers.openai.com/api/docs/bots. The older platform.openai.com path now redirects there.
This matters for citation hygiene. Anything still pointing at the old path is citing a redirect, and anything pointing at openai.com/gptbot as a source is citing a 403.
What is actually established, and how
Sorted by how strong the evidence is, not by how convenient it is.
| Claim | Basis |
|---|---|
| GPTBot and OAI-SearchBot are separate OpenAI crawlers with separate robots.txt tokens and separately published IP range files. | Documented by the platform |
| OpenAI publishes a distinct robots.txt-fetch variant of the OAI-SearchBot user agent, carrying an extra robots.txt marker. | Documented by the platform |
| ChatGPT-User is not used to determine whether content may appear in Search, and OpenAI points to OAI-SearchBot for that purpose. | Documented by the platform |
| OAI-AdsBot only visits pages submitted as ads, and its data is not used to train foundation models. | Documented by the platform |
| The address openai.com/gptbot returned HTTP 403 to a browser-identified fetch on 2026-09-03, so it functions as an identifier rather than documentation. | We measured this |
| Every OpenAI IP range endpoint listed on this page returned HTTP 200 when checked on 2026-09-03. | We measured this |
| Whether GPTBot and OAI-SearchBot share crawling infrastructure or run as genuinely separate fleets. | Not publicly documented |
What nobody can currently tell you
Stated because the alternative is implying a certainty that does not exist.
Version numbers inside the published user-agent strings will drift. The values in the table above are the ones OpenAI printed on the date of retrieval and should not be treated as stable identifiers.
OpenAI does not describe how the two crawlers are scheduled relative to each other, so there is no documented answer to how often either one returns to a given site.
Because the OpenAI crawler page shows no revision date, we cannot tell whether the OAI-AdsBot entry is new or long-standing. It reads as a recent addition, but that is an impression rather than a finding.
We have not run a controlled comparison of what each agent receives from a live server. Any claim that one is served different content than the other would need measurement, not documentation.
What people get wrong about this
-
GPTBot is the ChatGPT crawler.
What is actually the case
ChatGPT search inclusion runs through OAI-SearchBot. GPTBot collects material for foundation model training and is documented under a separate token.
-
A rule for one OpenAI agent covers the rest.
What is actually the case
robots.txt matches on user-agent token. GPTBot, OAI-SearchBot, ChatGPT-User and OAI-AdsBot each need to be named to be addressed.
-
You can identify the crawler by matching its full user-agent string.
What is actually the case
The published strings include version numbers that change. Matching on the token substring is the only approach that survives an update.
-
openai.com/gptbot is where the crawler is documented.
What is actually the case
That address is part of the user-agent identifier and returns 403 to a browser fetch. The documentation is at developers.openai.com/api/docs/bots.
Questions we get asked constantly
Do both crawlers respect robots.txt?
Yes. OpenAI documents both as robots.txt-controllable tokens, and it publishes a separate user-agent variant used specifically when fetching the robots.txt file itself.
How do I verify a request really came from one of them?
Compare the source IP against the endpoint OpenAI publishes for that specific agent. Each agent has its own JSON file, and the user-agent header alone can be forged by anyone.
Is OAI-SearchBot the same as Bing or Google?
No. It is OpenAI infrastructure with its own token and its own address ranges, unrelated to bingbot or Googlebot.
What does OAI-AdsBot fetch?
Only pages that have been submitted as advertisements on ChatGPT, for safety validation. OpenAI states its collected data does not feed foundation model training.
Where this came from
Every factual claim above traces to one of these. Each entry says what it supports and the date it was read, because platform documentation changes without notice.
-
OpenAI Bots: GPTBot — OpenAI
The GPTBot user-agent string, its stated training purpose, and the gptbot.json IP endpoint.
Primary source · read 2026-09-03
-
OpenAI Bots: OAI-SearchBot — OpenAI
The OAI-SearchBot user-agent string, its robots.txt-fetch variant, its search purpose, and the searchbot.json IP endpoint.
Primary source · read 2026-09-03
-
OpenAI Bots: ChatGPT-User — OpenAI
ChatGPT-User visits pages on user request and is explicitly not the control for search appearance.
Primary source · read 2026-09-03
-
OpenAI Bots: OAI-AdsBot — OpenAI
OAI-AdsBot validates pages submitted as ads and its data is not used for foundation model training.
Primary source · read 2026-09-03
-
AI Crawl Control: bots reference — Cloudflare
Cloudflare files GPTBot as AI Crawler and OAI-SearchBot as AI Search.
Secondary source · read 2026-09-03