Block GPTBot if opting out of training data inclusion, allow OAI-SearchBot for citations.
- Block GPTBot if opting out of training data inclusion, allow OAI-SearchBot for citations1 cited source1 category
Save for later 0
Use × to move an item here. You can restore it when you’re ready.
Your checks and saved items stay in this browser. Nothing is submitted.
Why this matters
Separating GPTBot (training crawler) and OAI-SearchBot (real-time citation crawler) allows selective content usage policies.
Hypothetical standard operating procedure
Adapt this example to your site, access, tools, and change process. It is our suggested procedure, informed by the sources below.
- Review current robots.txt for OpenAI user-agent rules.
- Add User-agent: GPTBot Disallow: / to block training data crawler if desired.
- Confirm User-agent: OAI-SearchBot Allow: / remains to allow citations.
- Check for overriding rules blocking all OpenAI bots.
- Deploy and monitor impact on citations and training data appearance.
Watch for
Blocking both bots removes training data inclusion and real-time ChatGPT citations.
Definition of done
robots.txt explicitly disallows GPTBot but allows OAI-SearchBot with no blanket disallow for all OpenAI bots.
Sources & attribution
Original checklist wording and example SOP by Best SEO Checklist, using source-grounded AI assistance. Editorial contact: Ted Kubaitis, @tedkubaitis on Teams. Check current source guidance before implementation.