VercatusGrade AR-056Answer engines
ChatGPT search and fetchers
OpenAI's OAI-SearchBot is used to surface websites in ChatGPT's search features, while ChatGPT-User visits a page when a user's question calls for it; because those visits are user-initiated, OpenAI says robots.txt rules may not apply to them. Blocking OAI-SearchBot in robots.txt governs appearance in ChatGPT search, not every fetch.
Sources
- 1
AGrade A: official documentation or a standard. A grade describes the provenance of the sources above; it is not a probability that a change will work. The effect basis says how the size of the effect is known: measured by a study, taken from an industry report, only documented without a size, or not quantified.
Put this evidence to work
Related entries
More in Answer engines, or search the whole base.
All Answer engines entries