Robots.txt for Shopify — Edit robots.txt.liquid Safely

Shopify's Default Robots.txt

Every Shopify store serves a robots.txt at your-store.myshopify.com/robots.txt automatically. The defaults block transactional and duplicate-content URLs — typically the admin area, cart and checkout endpoints, account pages, and the sort/filter parameter URLs that collection pages emit (e.g. ?sort_by=..., +-separated tag URLs).

You usually do not need to add anything for a typical store — the defaults are good. Open your own store's /robots.txt in a private window to see the exact list Shopify currently emits for your shop. Customize only when you have specific URLs you want excluded or rules to add.

How to Customize: robots.txt.liquid

Shopify lets you override the auto-generated file with a Liquid template. Steps:

1. Go to Online Store → Themes and click Edit code on your live theme.
2. Under the Templates folder, click Add a new template.
3. Choose template type robots and create robots.txt.liquid.
4. Paste the Liquid code below and save.

Recommended Template (keeps defaults + adds your rules)

{%- # templates/robots.txt.liquid -%}
{%- for group in robots.default_groups -%}
  {{- group.user_agent }}

  {%- for rule in group.rules -%}
    {{ rule }}
  {%- endfor -%}

  {%- if group.user_agent.value == '*' -%}
    {{ 'Disallow: /pages/contact-us-thanks' }}
    {{ 'Disallow: /pages/internal-only' }}
  {%- endif -%}

  {%- if group.sitemap != blank -%}
    {{ group.sitemap }}
  {%- endif -%}
{% endfor %}

User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: CCBot
Disallow: /

The {%- for group in robots.default_groups -%} loop emits Shopify's default rules untouched, then your {%- if group.user_agent.value == '*' -%} block adds extra disallows to the wildcard group. AI crawler rules are appended at the end as their own user-agent groups.

Reference: Shopify — Customize robots.txt.

Things You Can't Do on Shopify

• Upload a static robots.txt file. Shopify always serves the auto-generated version unless overridden by robots.txt.liquid.
• Remove Shopify's defaults entirely. Even with a custom template, you should keep the robots.default_groups loop so cart, checkout, and admin stay blocked. Removing them would leak transactional URLs into the index.
• Use robots.txt to deindex live pages. Shopify's own help docs warn that disallowing a URL in robots.txt does not remove it from Google. To deindex a page, set noindex via your theme or page-level settings.

Sitemap on Shopify

Shopify auto-generates a sitemap at your-store.myshopify.com/sitemap.xml (or your custom domain). The default robots.txt already includes the Sitemap: line, so submit only the URL to Google Search Console; you don't need to add it manually unless you've removed the default loop.

Block AI Crawlers

Append the AI crawler block from the template above. Shopify accepts standard robots.txt syntax outside the Liquid loop. After saving, fetch your-store.myshopify.com/robots.txt in a private window to confirm both Shopify's defaults and your additions appear.

Verify Changes

1. Save robots.txt.liquid and visit https://your-store.myshopify.com/robots.txt.
2. Confirm Shopify's default blocks (cart, checkout, admin) are still present.
3. Confirm your custom Disallow lines and AI crawler blocks appear.
4. In Google Search Console, open Settings → robots.txt and request a re-fetch.