← Answers Index/Developer Execution
Bot AccessibilityVerified: October 2026

How to verify if OAI-SearchBot and Google-Extended can parse my Shopify variant metadata?

Deterministic Technical Answer

You can verify crawler access by executing headless CLI requests using the specific User-Agent headers of `OAI-SearchBot`, `PerplexityBot`, and `Google-Extended`. You must confirm that your web application firewall (Cloudflare, Fastly, or Shopify bot protection) returns HTTP 200 without presenting a JavaScript challenge, and that the returned HTML stream contains complete Schema.org variant JSON-LD.

Empirical 1,284-Store Benchmark Data

Over 31% of Shopify stores inadvertently block or challenge OAI-SearchBot due to overly aggressive Cloudflare WAF or third-party fraud protection rules, completely de-indexing their catalog from ChatGPT Search.

Technical Root Cause Breakdown

Default security rules often categorize automated headless user agents as generic scrapers, serving an HTTP 403 Forbidden or Cloudflare Turnstile challenge that AI search crawlers cannot solve.

Production Fix Code (bash)Drop-in Snippet
# Terminal verification for OAI-SearchBot
curl -A "OAI-SearchBot" -I https://yourstore.com/products/handle

# Terminal verification for PerplexityBot
curl -A "PerplexityBot" -I https://yourstore.com/products/handle

# Verify variant schema in raw bot payload
curl -s -A "OAI-SearchBot" https://yourstore.com/products/handle | grep -E "schema.org|@type"

Step-by-Step Calibration Sequence:

  1. 1Run the curl commands above against your top product URLs.
  2. 2Confirm that the HTTP status code is 200 OK (not 403, 301, or 503).
  3. 3Check Cloudflare Security Events for blocked AI search user agents.
  4. 4Allowlist OAI-SearchBot and PerplexityBot in WAF settings.
Recommended Diagnostic ToolFREE SCAN

Relayeo Bot Simulator & Checker

Simulate OAI-SearchBot and Google-Extended crawls on your store at relayeo.com/tools/shopify-extractor. Access automated bot health telemetry at app.relayeo.com.

Developer CLI Check:
npx relayeo-check yourstore.com