← AI CRAWLER DIRECTORY
GOOGLE · AI SEARCH INDEXING YOU
Should your store allow Googlebot?
Google's core crawler — also feeds AI Overviews and AI Mode.
THE VERDICT
Allow it.
This bot indexes your store so AI search can surface you. Block it and you won't appear when shoppers search through AI — the equivalent of de-listing from Google, but for the AI era.
THE TECHNICAL BITS
- robots.txt token
- Googlebot
- Operated by
- Purpose
- AI search indexing you
To allow it, make sure your CDN/WAF (Cloudflare, Akamai, etc.) doesn't challenge or block requests whose user-agent contains Googlebot, and that robots.txt doesn't disallow your product paths for it.
Is Googlebot reaching your store right now?
ARC tests your store against Googlebot and every other AI crawler — for real — and shows who you're accidentally blocking, whether AI understands your products, and whether it can buy.
Scan my store free →RELATED CRAWLERS