September 3, 2026

GPTBot and your Shopify store: what your robots.txt actually says

Published September 3, 2026 by Adot Technologies Inc, the team behind Arvio: AI Store Operator. The counts below come from reading robots.txt on 58 live Shopify storefronts on September 3, 2026, plus a user-agent test run on our own domain.

Search "GPTBot" and page 1 is OpenAI explaining its own crawler, plus a row of bot-detection vendors explaining why you should buy bot detection. All of it is about GPTBot in general. None of it answers the question a Shopify merchant is actually holding: what is my store telling it right now, and did I choose that?

That question has a checkable answer, and it is public. We read it on 58 live Shopify stores.

Key takeaways

Shopify ships every store a robots.txt you did not write. It is 116 lines, and it names exactly two user agents: * and adsbot-google. GPTBot is not one of them. 53 of the 58 stores we could read were serving a file that matches its shape exactly.
One store in 58 names GPTBot at all, and that one allows it. Zero of the 58 disallow GPTBot at the site root. Whatever the debate about AI crawling looks like online, it has not reached these stores' robots.txt files.
A robots.txt asks; nothing enforces it. We could only test that on a server we are allowed to test — ours, which blocks nobody. A request carrying a GPTBot user agent and one carrying a normal browser user agent came back identical across all 36 URLs in our sitemap: same status, same byte count, same visible character count. That is one server, and it shows a server that does not discriminate — not that no server does.
"Blocking GPTBot" is not "keeping my store out of ChatGPT." OpenAI runs separate agents for training, for search, and for user-triggered fetches, and its documentation is explicit that the robots.txt settings are independent of each other. Disallowing the training crawler does not remove you from ChatGPT's search answers; disallowing the search crawler does.
Whichever way you want it, the fix is one template file in your theme, and you can verify it from a browser in one request. The syntax is the easy part. What takes thought is which of the three agents you actually meant.

Your robots.txt is a decision your store already made for you

Arvio audits what your storefront publishes about your products: catalogue fields, product metadata, page descriptions. It shows you what is there before you change anything — the robots.txt above is a theme file, and that one is yours to edit.

Install Arvio on the Shopify App Store →

First, read your own file

Open https://yourstore.com/robots.txt in a browser tab. That is the whole diagnostic. It is a plain text file, it is public, and it is the file every well-behaved crawler reads before it reads anything else on your store.

You will almost certainly see a long list of Disallow: lines covering /cart, /checkout, /orders, /account, and a pile of /collections/*sort_by* variants. That is Shopify's default, and it is a sensible default — it keeps crawlers out of the pages that are per-visitor or infinite.

What you will not see, unless someone put it there, is any mention of GPTBot.

What Shopify ships you

We fetched robots.txt from all 60 hostnames in a public sample of live Shopify storefronts. 58 answered.

53 of those 58 were serving a file with the platform default's exact shape. We tested the shape, not the bytes — we did not diff the whole file, so "same shape" is not the same claim as "never touched": swapping a Disallow path or adding a Sitemap line changes neither number we counted. What we can say is that on both counts the file was indistinguishable from the default: 109 non-blank lines (116 including blank lines) and exactly two User-agent groups — * and adsbot-google. Every one of the 53 matched on both counts. The same default ships two other agent-facing things nobody installed either — an agents.md instruction file and a live MCP endpoint.

The five that differed, by non-blank line count:

Lines	What it looks like
193	Seven groups: the default's `*` and `adsbot-google`, plus five named ones — Ahrefs (two), MJ12bot, Nutch, Pinterest
165	Default plus eight named groups, including GPTBot. The only file in the sample that names it
159	A different store, the same seven groups as the 193-line file, fewer rules inside
159	A third store, the same seven groups again — but every group header is malformed. See below
20	Far shorter than the default; not the same kind of edit as the other four

Three days earlier, on August 31, we ran a narrower version of the same read against the same 60 hostnames: 59 answered, 54 were on the platform default, 5 were customised. The picture is stable. (That earlier file counted total lines including blanks, which is why its default is 116 and this one's is 109. Same file, two ways of counting — we re-fetched one default store to confirm both numbers describe it.)

The one store that names GPTBot allows it

Of all 58, exactly one has a User-agent: GPTBot group. That group carries no Disallow line at all, which means: crawl anything.

So the count that matters is the boring one. Stores in this sample that block GPTBot from the site root: 0 of 58.

That may not be what you expected. We measured these 58 storefronts and nothing else — we have no reading on what publishers outside this sample are doing, and this survey cannot give you one. Inside it, the file is untouched on this question.

One of the five customised files is broken

One store's robots.txt has every group header collapsed onto one line with the rule that should follow it:

User-agent: *Disallow:

All seven of its groups look like that — the same seven the other two customised stores have — including the * group that carries the cart and checkout rules. Our parser, which requires the header to be its own line, finds no valid User-agent group in the file at all; we did not test it against Google's or OpenAI's parsers, so what any of them does with it is not something this survey measured. What the file does do is return 200 across 159 non-blank lines that look fine at a glance, which is the part worth knowing: nothing about it tells you it is broken.

A robots.txt does not report errors. Nothing turns red. Nobody emails you. If you edit yours, the only way to know it worked is to read it back and check the shape.

robots.txt is a request, not a lock

robots.txt is a file you publish saying what you would like crawlers to do. Well-behaved crawlers read it and comply. It is not enforced by your server, and it does not stop anything from arriving. If you want an actual block, that lives somewhere else — a rule at your CDN or firewall that refuses the request.

We wanted to know whether a server treats a GPTBot request differently from a browser request, and our own domain is the only place we can ask that. Our sitemap has 36 URLs. We requested each one twice: once with OpenAI's published GPTBot user-agent string, once with an ordinary desktop Chrome user-agent string. Then we compared status code, response size in bytes, and the visible character count after stripping tags and scripts.

All 36 came back identical on all three measures. Same 200, same byte count, same visible text. The server does not know or care which one is asking.

We did not send a GPTBot user agent to anyone else's store. Impersonating another company's crawler against servers that may be trying to filter it is a different act from testing your own property, and we did not do it. Every third-party number in this article comes from reading a public robots.txt with our own clearly-identified research agent. So this article can tell you what 58 stores say, and what one server (ours) does. It cannot tell you what those 58 servers do.

The three agents are not one switch

This is where most "how to block AI crawlers" advice quietly gives you the wrong outcome.

OpenAI does not run one crawler. Reading its bot documentation on September 3, 2026, the relevant ones are:

Agent	What OpenAI says it is for
`GPTBot`	Crawling content that may be used in training its generative AI foundation models
`OAI-SearchBot`	Surfacing websites in search results in ChatGPT's search features
`ChatGPT-User`	Visiting a page because a user in ChatGPT asked something that required it

And the documentation is explicit that these are independent controls: "Each setting is independent of the others – for example, a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot to indicate that crawled content should not be used for training OpenAI's generative AI foundation models."

It also states the consequence of the one people rarely mean to pick: "Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links."

So the two mistakes are symmetric, and both are common:

You block GPTBot because you do not want your product copy in a training set. Fine — that is exactly what it does, and it does not take you out of ChatGPT search.
You block everything with openai in the name because you want "AI out of my store." That includes OAI-SearchBot, and it does take you out of ChatGPT search results.

For most stores selling things, those two sentences point in opposite directions, and the decision is genuinely yours to make.

How to change it on Shopify

Shopify lets you edit the file through a theme template. The steps, from the admin:

Online Store → Themes → ⋯ → Edit code on your live theme.
Under Templates, click Add a new template, choose robots.txt from the template-type list, and create it. Shopify creates robots.txt.liquid prefilled with the default rules — it renders the same file you are serving today.
Add your own User-agent groups above or below the default block. Do not delete the default block; those Disallow rules are keeping crawlers out of carts and checkouts.
Save.
Read it back: open https://yourstore.com/robots.txt in a browser and confirm your group is there, on its own line, spelled the way you meant.

A group that disallows the training crawler while leaving search alone looks like this:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

The opposite decision, letting everything in, is simply the absence of any Disallow for those agents. That is what all 58 stores we read have today: 57 because the file never mentions these agents, and one because it names GPTBot and allows it.

Three things that bite people at step 5:

Agent names are exact tokens. GPTBot is the agent name. GPT-Bot, OpenAI and ChatGPT are not that agent, and a group headed by one of them silently matches nothing.
Disallow: with nothing after it means "allow everything." Disallow: / is the block. The empty one is the opposite of what it looks like.
The group header has to be its own line. That malformed store above is what happens when it is not: 200 back, 159 non-blank lines, nothing that looks wrong.

You changed the file. Did the rest of the store follow?

A crawler that is allowed in still only reads what your storefront publishes. Arvio audits that surface: catalogue fields, product metadata, page descriptions. It drafts the changes and you approve them before anything goes live.

Install Arvio on the Shopify App Store →

So should you block it?

We make a Shopify app, and an app that helps you fill in catalogue fields does better in a world where assistants read catalogues. What the measurement supports is narrower:

Right now these agents get whatever you give every other crawler. The default file has no opinion about them, so they fall through to the * group, and 53 of 58 stores look untouched.
If your reason is "I don't want my writing used for training," GPTBot is the agent that maps to that, and blocking it costs you nothing in ChatGPT's search surface.
If your reason is "I want to be findable when someone asks an assistant for what I sell," blocking is the wrong move. Being allowed in is the floor, not the finish — what an assistant actually reads out of a Shopify catalogue is a separate question from whether it is allowed to read it.
If you want a block that holds against traffic that ignores robots.txt, this file will not do it. That is a CDN or firewall rule, and a different piece of work.

What this article does not know

We have no server logs for these stores, so nothing here says whether GPTBot has ever visited any of them, how often, or what it took.
58 stores is a sample, drawn from a public list of stores whose owners posted their own URL asking for feedback. We did not measure their age, their revenue or their catalogue size, so we cannot tell you how that frame skews — only what it is: stores whose owners posted their own URL asking strangers for feedback. It is not the Shopify population, and certainly not the large-merchant population, where a CDN in front of the store makes all of this look different.
robots.txt changes the day someone edits it. Our two reads are three days apart and agree. That is stability over three days, not a law.

How this was measured

The sample. 60 hostnames drawn from a frame of 230 live Shopify storefronts, built on August 26, 2026 from 474 publicly posted store URLs — 244 of those no longer served a public catalogue and were dropped, so the frame is what survived. The draw is random.Random(20260826).sample(frame, 60); 58 of the 60 answered. We have not published the frame file or the per-store readings, because they name 58 real storefronts — but nothing here depends on our copy: robots.txt is public on every Shopify store, and one curl -s https://anystore.com/robots.txt reproduces the unit of measurement we used.

The robots read (September 3, 2026). One GET /robots.txt per host, at one request per second, with a user agent that identifies itself as our research crawler and carries a contact address. 58 of 60 returned a readable file; 2 did not answer. For each, we parsed the file into User-agent groups and recorded: non-blank line count, the agent names present, whether any group names GPTBot, and the Disallow rules inside that group if it exists.

"Platform default" is a shape test, not a diff. A file counted as default if it had 109 non-blank lines and exactly the two agent groups * and adsbot-google. 53 of 58 matched. We separately fetched one of those files whole to confirm the line counts reconcile with the August 31 read (116 lines including blanks).

The user-agent test (September 3, 2026, our domain only). Every <loc> in our own sitemap — 36 URLs — requested twice, once with OpenAI's published GPTBot user-agent string and once with a desktop Chrome user-agent string, one second apart. Recorded: HTTP status, response bytes, and visible characters after removing <script>, <style> and all tags. Three of the 36 needed a re-request on the first pass. Final: 36/36 identical on all three measures.

The OpenAI documentation (platform.openai.com/docs/bots) was fetched the same day (HTTP 200) and the quoted sentences are copied verbatim from that response. That page is public, so every quotation in this article is one you can check yourself.

Exclusion rule. Two hosts that did not return a readable robots.txt are excluded from every denominator in this article. They are counted in "60 asked", never in "58 read."

FAQ

Does Shopify block GPTBot by default?
No. The default robots.txt Shopify serves does not mention GPTBot in any way. It names two user agents — * and adsbot-google — and the rules under * are about carts, checkouts, accounts and sorted collection URLs. In our sample, 53 of 58 were serving a file with that exact shape.

If I block GPTBot, do I disappear from ChatGPT?
Not from its search answers. Per OpenAI's own documentation, GPTBot governs crawling for model training and OAI-SearchBot governs appearing in ChatGPT's search results, and the two settings are independent. If disappearing from ChatGPT search is what you want, OAI-SearchBot is the agent you would disallow — and OpenAI states that opted-out sites "will not be shown in ChatGPT search answers, though can still appear as navigational links."

Will blocking it actually stop anything?
It stops crawlers that read robots.txt and comply, which is what the named, documented ones do. Nothing enforces it. Our own-domain test found the server returning identical responses to a GPTBot user agent and a browser user agent across all 36 sitemap URLs — nothing at the server layer was distinguishing them. If you need enforcement, that is a CDN or firewall rule, not this file.

How do I check whether my edit worked?
Open https://yourstore.com/robots.txt in a browser and read it. Confirm your User-agent: line is on a line by itself, that the agent name is spelled exactly, and that Disallow: / is present if a block is what you meant — an empty Disallow: allows everything. One store in our sample of 58 has a malformed group header and still serves a 200 that looks entirely normal.

Does any of this affect Google?
Not directly. Google's crawlers are separate agents with separate rules, and the Shopify default names only * and adsbot-google, no Google AI agent of any kind. Nothing in this article changes how Googlebot sees your store.

Arvio: AI Store Operator — install it on the Shopify App Store. Once a crawler is allowed in, what it reads is your catalogue fields — and that is the part Arvio works on.