Orkkid
Free tool · AI search visibility

Is your site blocking AI crawlers?

We fetch your robots.txt and test it against 12 AI user agents, from GPTBot to PerplexityBot. See exactly which engines can read your site and the rule that blocks the ones that can't.
12 AI crawlers
GPTBot, ClaudeBot, PerplexityBot, Google-Extended and more.
Real robots.txt
Fetched live and parsed to Google's spec, wildcards included.
Exact rule shown
See the group and line that blocks each bot.

How it works

We fetch your robots.txt and parse it the way Google does: user-agent groups, Allow and Disallow with longest-match precedence, and the*and$wildcards. Each of the 12 AI user agents is tested against your homepage path, falling back to the wildcard group when it has no group of its own. "Partial" means the homepage is allowed but some paths are not. Nothing you enter is stored.

Crawler access is step one. Read how ChatGPT recommends businesses or see how it fits into a generative engine optimization plan.

Questions

AI Crawler Checker: common questions

Which AI crawlers does the checker test?

GPTBot, ChatGPT-User and OAI-SearchBot (OpenAI), ClaudeBot and anthropic-ai (Anthropic), Google-Extended (Gemini training), PerplexityBot, CCBot (Common Crawl), Applebot-Extended, Meta-ExternalAgent, Bytespider (TikTok) and DuckAssistBot.

Should I block AI crawlers?

If you want customers to find you through ChatGPT, Perplexity or Gemini, no. Blocking GPTBot or OAI-SearchBot removes you from ChatGPT search results. Many hosts and security plugins add these blocks by default, which is why this check exists.

My robots.txt has no AI rules. Am I blocked?

No. With no matching User-agent group and no wildcard Disallow, every crawler is allowed. The checker shows 'allowed' in that case. Blocks usually come from a 'User-agent: *' group with 'Disallow: /', or from a Cloudflare or WAF setting that never appears in robots.txt.

The checker says allowed but ChatGPT still can't read my page. Why?

robots.txt is only the first gate. Cloudflare Bot Fight Mode, a WAF, IP blocking, or a JavaScript-only page can still stop the crawler. Run the AI Visibility Audit to see whether your HTML is readable without JavaScript.

Related free tools

Browse all free GEO tools

Robots.txt Generator

Build a correct robots.txt in seconds: toggle GPTBot, ClaudeBot, PerplexityBot and other AI crawlers, set rules per bot and add your sitemap.

AI Visibility Audit

Real audit of any website's AI search readiness. Crawler access, llms.txt, schema, sitemap, on-page citability, security. Letter grade in 10 seconds.

llms.txt Generator

Generate a working llms.txt from your sitemap in seconds. It crawls your pages, pulls real titles and descriptions, and outputs a file for your site root.

Be the business AI recommends.

The free GEO audit runs this check and the rest on your site and your competitors, then shows what is stopping ChatGPT and Google from sending you customers.

Get my free GEO auditBook a call with Ned