Free tool
AI crawler access checker
See which AI bots can read a site. Paste its robots.txt and get the answer instantly.
1Open the robots.txt
2Paste it here
No file handy?
Your results show here.
Reading the result
Three possible outcomes
AI can read it
Every bot we check may crawl the homepage.
Some AI is shut out
Some engines can read the site. Others cannot.
Invisible to AI
No bot we check may crawl the homepage.
FAQ
About this checker
Why paste the file instead of entering a URL?
Browsers cannot read another site’s robots.txt directly. Pasting keeps the check instant and private.
Does this tool store what I paste?
No. The check runs in your browser and nothing is sent anywhere.
Which AI bots does it check?
GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, Google-Extended, Applebot-Extended, CCBot, Meta-ExternalAgent, Amazonbot and Bytespider.
Does allowing AI crawlers guarantee citations?
No. It means AI engines can read the page. Nobody controls what they cite.
Get published where AI can read
We never list sites that block every AI crawler.