How to Configure Shopify robots.txt.liquid for ChatGPT, Perplexity & Google AI
Ensure OAI-SearchBot, GPTBot, PerplexityBot, and Google-Extended can index your product catalog without exposing private checkout or cart sessions.
Shopify's default robots.txt.liquid allows standard Googlebot and Bingbot, but frequently treats emerging AI search agents inconsistently. Furthermore, some third-party security apps accidentally block OAI-SearchBot and PerplexityBot. By adding explicit Allow directives for legitimate shopping bots while preserving Shopify's built-in disallow rules for /cart and /checkout, merchants remove crawl barriers that prevent real-time search indexing. While allow directives do not guarantee ranking alone, they establish the essential crawling prerequisite for conversational shopping visibility.
How to Implement the Fix
Create or Open templates/robots.txt.liquid in Your Theme
In Shopify Admin, go to Online Store > Themes > Edit Code. If templates/robots.txt.liquid does not exist, click Add a new template and select robots.txt.
Inject AI Search Allow Rules
Paste this production-tested robots.txt.liquid configuration to whitelist conversational shopping crawlers while safeguarding private cart/checkout paths:
{%- comment -%}
Optimized robots.txt.liquid for Live AI Search & Shopping Agents
Allows: OAI-SearchBot, GPTBot, PerplexityBot, ClaudeBot, Google-Extended
Protects: /checkout, /cart, /admin, customer account portals
{%- endcomment -%}
# Shopify Core System Rules
{% for group in robots.default_groups %}
{{- group.user_agent }}
{%- for rule in group.rules %}
{{- rule }}
{%- endfor %}
{%- if group.sitemap != blank %}
{{- group.sitemap }}
{%- endif %}
{% endfor %}
# Explicit Allow Rules for Conversational AI Shopping Engines
User-agent: OAI-SearchBot
Allow: /products/
Allow: /collections/
Allow: /pages/
Disallow: /cart
Disallow: /checkout
Disallow: /account
User-agent: PerplexityBot
Allow: /products/
Allow: /collections/
Disallow: /cart
Disallow: /checkout
User-agent: GPTBot
Allow: /products/
Allow: /collections/
Disallow: /cart
Disallow: /checkout
User-agent: ClaudeBot
Allow: /products/
Allow: /collections/
Disallow: /cart
Disallow: /checkout
User-agent: Google-Extended
Allow: /products/
Allow: /collections/
# Point AI Agents to Universal Machine Feeds
Sitemap: {{ shop.url }}/sitemap.xml
Sitemap: {{ shop.url }}/llms.txtVerify Crawler Access via Relayeo Extractor
Run your domain through Relayeo's Machine Inspector to confirm HTTP 200 responses and clean crawler headers across all 5 major AI user-agents.
Why Standard Themes Fail in AI Search
Default Blanket Disallows on Query Parameters
Root CauseStandard Shopify robots.txt blocks URLs containing *?* on certain paths, inadvertently preventing AI agents from crawling specific variant URLs like ?variant=481928374.
App Firewall False Positives
Root CauseWAFs and bot protection apps often flag OAI-SearchBot or ClaudeBot as malicious scrapers, returning 403 Forbidden errors when AI search engines attempt real-time quotation.
Confusion Between Training Scrapers and Live Search
Root CauseBlocking all AI bots to prevent model training also blocks real-time search crawlers like OAI-SearchBot, completely erasing the store from ChatGPT Search recommendations.
Run a 1.1-second scan on your live Shopify theme with zero account required.
Automate Crawler Governance with Relayeo
Maintain dynamic AI crawler directives and monitor crawler hit rates across 6 major AI engines automatically.