Is your website blocking ChatGPT?
Type your address and we read your robots.txt, then ask your site for its homepage once as each major AI crawler — and tell you, in plain English, which ones it turned away.
Three kinds of crawler, and only one of them matters.
Blocking a training crawler like GPTBot does not remove your site from ChatGPT’s search results. Blocking OAI-SearchBot can. They are different crawlers with different jobs, and most advice online treats them as one.
Look up any individual crawler — what it does, whether it honours robots.txt, and the lines to block or allow it.
Search crawlers
OAI-SearchBot, Claude-SearchBot, PerplexityBot
These build the index an assistant answers from. If one of these is turned away, your pages are missing from the pool the assistant draws on when someone asks for a business like yours. This is the group that matters most for being recommended.
Answering a user’s question
ChatGPT-User, Claude-User, Perplexity-User
These fetch a page in the moment, because a person just asked something that needs it. If one is turned away, the assistant can’t open your site to check a detail — your hours, your prices, whether you do the thing they asked about.
AI training
GPTBot, ClaudeBot, CCBot
These collect content to train models. Blocking them is a common and entirely legitimate choice, and it is not a visibility problem — we report it as information, never as something to fix.
Where the block probably lives.
A rule in robots.txt is in a file you control. A refusal that robots.txt doesn’t explain is coming from something in front of your site — and that is usually a setting, not a code change.
Cloudflare
Cloudflare can block AI crawlers without the setting ever appearing in your own files, and it can serve a robots.txt that overrides the one on your server. In your Cloudflare dashboard, open the site, and look for the AI crawler controls — they are grouped as Search, Agent and Training. Allow Search (that is the category the assistants answer from), and choose for yourself on Training. Also check whether the managed robots.txt option is on: if it is, the file we read is Cloudflare’s, not yours, and editing your own will change nothing.
WordPress
WordPress usually serves a virtual robots.txt rather than a real file, so there may be nothing to edit on disk. Check your SEO plugin first — Yoast, Rank Math and All in One SEO each expose a robots.txt editor, and a rule added there is what gets served. If a security plugin such as Wordfence is installed, check its blocking rules too: they match on user-agent, and lists that block “bots” broadly often include AI crawlers by name.
Squarespace, Wix or Shopify
These platforms generate robots.txt for you and give you limited or no control over it. Shopify allows editing through a theme file; Squarespace and Wix largely do not. If the block we found is at the server level rather than in robots.txt, it is coming from the platform’s own bot protection and is not something you can switch off from your side — their support is the only route, and it is worth asking them directly whether AI crawlers are allowed.
Your own server
Look for user-agent matching in two places: your robots.txt, and whatever sits in front of it — nginx or Apache config, a WAF, or a hosting provider’s bot protection. A refusal that robots.txt does not explain is coming from one of those. Your access logs are the fastest way to find it: search them for the crawler’s name and see what status it was served.
Three requests, and one thing we can’t know.
- 1
We fetch your robots.txt and read the rules it sets for each AI crawler by name.
- 2
Then we request your homepage once as an ordinary web browser, to see what a normal visitor gets.
- 3
Then we request it again once per crawler, identifying ourselves with that crawler’s name, and compare each answer against the browser one.
Our requests don’t come from OpenAI’s or Anthropic’s servers, so a firewall may treat them differently from the real crawlers. When we say a request was refused, that is exactly what we measured — not proof that the real crawler is refused. To confirm either way, check your Cloudflare dashboard or your server logs for the crawler’s name.
The ones people actually ask.
Something not covered? Email hi@seenvia.com; a real person reads it.
1.Does blocking GPTBot hide me from ChatGPT?
No. GPTBot collects content for training OpenAI’s models. The crawler that builds ChatGPT’s search index is OAI-SearchBot, and it’s a separate agent with a separate rule. You can block GPTBot and still be findable in ChatGPT — plenty of sites are. If you want to be findable, the one to let through is OAI-SearchBot.
2.Is Cloudflare blocking AI crawlers on my site?
It might be, without you having turned anything on. Cloudflare added a managed setting that blocks AI crawlers, and it is applied by default for domains onboarded after September 2025. It can also add a block to a managed robots.txt that overrides the file on your own server. If this check finds a block you don’t recognise, your Cloudflare dashboard is the first place to look — not your website files.
3.Should I block AI training?
That is a judgement call and we won’t make it for you. Some businesses want their content kept out of model training on principle; others don’t mind. What we can say is that it is a separate decision from being findable — blocking training crawlers does not affect whether an assistant can recommend you, so you can decline training and still be visible.
4.Can you tell whether the real GPTBot gets through?
Not with certainty, and we would rather say so. We send a request carrying the crawler’s name from our own servers. The real crawlers request from addresses OpenAI and Anthropic publish, and a firewall can treat the two differently — usually by being stricter with ours. So a refusal here is a strong hint worth investigating, not proof. Your server logs and your CDN dashboard are the definitive answer.
5.Why does my result say “couldn’t confirm”?
Because something stopped us learning the answer, and we won’t guess. The usual causes are a site that didn’t respond in time, a server that refused our ordinary browser request too — in which case nothing can be said about any individual crawler — or a robots.txt we couldn’t read. It is not a pass and it is not a failure. Trying again a few minutes later usually resolves it.
6.Does this affect my Google ranking?
The AI crawlers here are separate from Googlebot, so blocking them does not change your position in Google’s ordinary search results. There is one exception worth knowing about, and this check looks for it: a broad “block all the bots” rule sometimes catches Googlebot or bingbot too. If that has happened to you, we flag it separately — it is a much more urgent problem than any AI crawler, and it is the one thing on this page that can genuinely cost you traffic today.
Being readable is step one. Whether AI actually recommends your business is a separate question.
A site every crawler can read can still go unmentioned when someone asks for the best in your town. The free check asks an AI assistant that question for real and shows you the answer, including who got named instead of you.