CrawlLedger
OpenAI · Ad landing-page validation

AI crawler directory / OAI-AdsBot

OAI-AdsBot robots.txt reference.

OpenAI's fetcher for validating the safety and relevance of landing pages that advertisers submit for ChatGPT ads.

20observed domains
0disallowed
0partial policies
0allowed by matched rules

What OAI-AdsBot is documented to do

This summary is tied to a first-party operator page reviewed on Sep 20, 2026. It should be rechecked when the registry version changes.

Operator
OpenAI
Documented purpose
Ad landing-page validation
Separate HTTP fetcher
Documented as a fetcher or crawler
Official documentation
https://developers.openai.com/api/docs/bots
Important operator note
OpenAI says it visits only pages submitted as ads and does not use the collected landing-page data to train generative AI foundation models.

GPTBot vs OAI-SearchBot vs OAI-AdsBot vs ChatGPT-User

OpenAI documents these tokens for different purposes. A rule for one does not automatically control the others.

OAI-AdsBot

Ad landing-page validation. OpenAI's fetcher for validating the safety and relevance of landing pages that advertisers submit for ChatGPT ads.

GPTBot

Model training. OpenAI's automatic crawler for content that may be used to improve and train generative AI foundation models.

OAI-SearchBot

AI search. OpenAI's automatic search crawler, used to surface and link websites in ChatGPT search results.

ChatGPT-User

User-requested fetch. OpenAI's user-initiated fetcher for certain ChatGPT, Custom GPT, and GPT Action requests.

Common full-site robots.txt block pattern

This example states a crawl preference for the named token. It does not authenticate requests, enforce blocking, or automatically control another token from the same operator.

User-agent: OAI-AdsBot
Disallow: /

Check the official operator source before publishing a policy. User-requested fetchers and non-fetcher control tokens can have different behavior from automatic crawlers.

Latest observed pilot policies

These states are calculated from each domain's latest available robots.txt record using the versioned CrawlLedger parser. They are observations of published text, not proof of crawler behavior.

DomainParsed stateMatched groupLast observed
anthropic.comunverifiedNo matching group observed
apnews.comunverifiedNo matching group observed
apple.comunverifiedNo matching group observed
cloudflare.comunverifiedNo matching group observed
commoncrawl.orgunverifiedNo matching group observed
google.comunverifiedNo matching group observed
medium.comunverifiedNo matching group observed
openai.comunverifiedNo matching group observed
oreilly.comunverifiedNo matching group observed
perplexity.aiunverifiedNo matching group observed
quora.comunverifiedNo matching group observed
reddit.comunverifiedNo matching group observed
rslcollective.orgunverifiedNo matching group observed
rslstandard.orgunverifiedNo matching group observed
stackoverflow.comunverifiedNo matching group observed
theguardian.comunverifiedNo matching group observed
usatoday.comunverifiedNo matching group observed
voxmedia.comunverifiedNo matching group observed
yahoo.comunverifiedNo matching group observed
ziffdavis.comunverifiedNo matching group observed

Interpret the state carefully

“Allowed” means a selected group was observed without a blocking rule for the tested policy shape. “Partial” means at least one non-empty disallow path was observed. “No matching group” is silence, not an affirmative grant. None of these states authenticate the requester or establish legal permission.