Reference

How AI search actually works

Written for people who want the mechanism, not the pitch. Sourced to platform documentation, with the parts nobody has documented labelled as exactly that.

What this is

A reference section about AI search, AI crawlers and structured data. Every factual claim carries a source, the date it was read, and whether it is documented by the platform, observed in a test we ran, or inferred. Where nobody has published an answer, these pages say so instead of guessing. That is the whole editorial standard.

AI crawlers

AI crawlers and access

Does blocking GPTBot remove your website from ChatGPT search? Does blocking GPTBot remove your website from ChatGPT search? What is the difference between an AI search crawler and an AI training crawler? What is the difference between an AI search crawler and an AI training crawler? What is the difference between GPTBot and OAI-SearchBot? What is the difference between GPTBot and OAI-SearchBot? What does Google-Extended actually control? What does Google-Extended actually control? How do you test whether AI crawlers can access your website? How do you test whether AI crawlers can access your website? What does Cloudflare's AI crawler blocking actually do? What does Cloudflare's AI crawler blocking actually do?

llms.txt

llms.txt

What is llms.txt? What is llms.txt? Do AI assistants actually read llms.txt? Do AI assistants actually read llms.txt? How is llms.txt different from robots.txt? How is llms.txt different from robots.txt? Should your website publish an llms.txt file? Should your website publish an llms.txt file?

AI search

AI search optimization

What is AI search optimization? What is AI search optimization? How is generative engine optimization different from SEO? How is generative engine optimization different from SEO? Does schema markup help AI search? Does schema markup help AI search? What is entity authority? What is entity authority?

Local AI search

Local AI search

How does ChatGPT find local businesses? How does ChatGPT find local businesses? Do online reviews affect which businesses AI assistants recommend? Do online reviews affect which businesses AI assistants recommend?

Also here

Definitions, research, and worked examples

GlossaryPlain definitions of the terms this section uses. ResearchStudies we are running, with the methodology published first. Case studiesWork on real sites, including our own. Buying guidesWhat things cost and how to choose. Different question, different section.