Glotier tells you whether ChatGPT, Gemini and Perplexity name your product when buyers ask what to use in your category. It puts your buyers' real questions to all three as live web queries, records whether you were named and by which of them, which competitors were named instead, and which pages each answer was built from.

How does Glotier check AI visibility?

It runs one live web search per question. ChatGPT and Gemini read the identical results, and Perplexity runs its own retrieval because it does not accept results handed to it. Whether your product is named is decided by a literal presence check on the answer text rather than by asking a model for its opinion, so the number cannot be inflated by a model being agreeable. You can also connect your own ChatGPT or Claude key and have the questions run through your account instead.

Why ask three assistants instead of one?

Because they disagree. In our own study, across the 39 of 40 buying questions where all three answered, all three named the same product only 30% of the time and 47% of products were named by exactly one assistant. Checking a single assistant shows you 58% to 65% of the products named across the three, so a founder who checks ChatGPT, sees their name and relaxes has measured one engine on one phrasing.

Is Glotier free to use?

The check is free with no signup. Paid plans start at $39/month (Solo), $99 (Studio) and $249 (Agency), and cover weekly tracking and the content the agent writes.

Does Glotier make up data about my product?

No. Every result comes from observed data: the live search results, the answers the assistants actually returned, and the sources they cited. It does not invent metrics, ratings or engine claims, and when a measurement has not run it says so rather than printing a zero.

Anyone selling something buyers research before choosing, where the category has enough alternatives that an assistant has to pick a shortlist: software, tools, e-commerce, and services. Founders, marketers and agencies all use it, and the work is the same at any size. It is not useful for local or in-person businesses, where maps and reviews decide the outcome instead.

Study

Who actually blocks AI crawlers?

By Oğuzhan Tüsen @oguzhanglotierPublished 31 July 2026

We read the robots.txt of 259 domains that assistants actually cited in our category, expecting to find sites quietly locking themselves out of AI answers. 235 of them allow every OpenAI agent. The hypothesis was wrong, so here is the disproof.

The claim we set out to test

It is repeated everywhere in this field: companies are removing themselves from AI answers by accident, pasting an over-broad robots.txt or letting a security plugin block crawlers they have never heard of. It is a good story. It gives an audit tool something alarming to report, and the fix is a one-line change, which makes it the easiest recommendation in the category to sell.

We had the material to check it. Our daily probe records every domain an assistant cites when answering a buying question, so we have a list of sites that demonstrably win AI citations. Reading their robots.txt answers the question directly: do the sites that get cited understand crawler access, or are they getting away with it by luck?

Method

On 31 July 2026 we fetched /robots.txt from 259distinct domains that our probes recorded an assistant citing. Each file was parsed with the same code that produces a customer’s audit, so the study cannot disagree with the product. 244 served a parseable file. 15 served none, which means no restrictions rather than missing data, and 0 were unreachable.

Blocked means a full-site Disallow: /for the group matching that agent. A partial path rule is not counted, so this measures whole-site exclusion and nothing narrower. Agents are grouped by job rather than by vendor marketing: OpenAI’s GPTBot trains, OAI-SearchBotfeeds the search answers, and the two are separate permissions in OpenAI’s own documentation.

What we found

Of 259 domains, 235 (91%) allow every OpenAI agent, training and search alike. 18 block the training crawler while leaving the search agent free. 5block OpenAI’s search agent by name. One denies every crawler by default and names no AI agent at all.

Posture	Domains	Share
Open to every OpenAI agent	235	91%
Blocks training, allows search	18	7%
Blocks the search agent by name	5	2%
Denies all crawlers, names no AI agent	1	<1%

The 5 that block OpenAI’s search agent are adage.com, builtin.com, facebook.com, instagram.com, quora.com. Every one is a large publisher or social platform, and for a business whose product is the content itself, refusing an assistant that summarises it is a coherent commercial decision rather than an accident. That is the opposite of the story we set out to check.

The 18 that made the informed choice

The most interesting group is the one blocking GPTBot while leaving OAI-SearchBot free. That is not confusion, it is precision: out of the training data, still in the answers. medium.com, fastcompany.com, glassdoor.com, sitepoint.com, uk.linkedin.com are among them.

It is a real trade rather than free hygiene. A model that has never trained on you can still cite you when it retrieves you today, but it cannot recall you when it is not searching. These sites have kept the surface that works when an assistant looks something up and given up the one that works when it answers from memory.

The parsing detail that changed our own result

reddit.com carries User-agent: * with Disallow: / and no AI-specific group. A correct parser reports every AI agent as blocked, and our first run duly counted it as a site that blocked the AI bots. That was false. It denies every crawler by default and allows named ones back in, which is a decades-old access policy and not a decision about assistants.

Separating the two moved the headline. A site that deliberately disallowed OAI-SearchBot and a site that has never heard of it are different findings, and telling them apart is the whole question. Any count of AI blocking that does not make this distinction is inflated, and most published ones do not say either way.

Perplexity is refused slightly more often, and we got its label wrong

7 domains block PerplexityBot by name: adage.com, builtin.com, facebook.com, instagram.com, nymag.com, quora.com, uk.linkedin.com. That is more than the 5refusing OpenAI’s search agent. The counts are small and the direction is the observation, not the margin.

Our classifier labelled those 7as “blocks search, allows training”, which is wrong: Perplexity publishes no training crawler, so there is no training permission to allow, and blocking its single agent is total exclusion. We found it while preparing this piece and fixed it before publishing. It is in here because a study about other people’s parsing mistakes is worth less from someone hiding their own.

One caveat travels with every Perplexity figure. Cloudflare published evidence of undeclared crawlers reaching pages that disallow them and de-listed Perplexity as a verified bot; Perplexity disputes it. So for this vendor the study records what a site declares and cannot claim to describe what it can read.

What this does not show

This corpus is domains an assistant cited. By construction it cannot contain a site that blocked itself out of citations, because such a site would never have appeared in an answer for us to record. So the honest reading is narrow: among sites that win AI citations, accidental self-removal is rare. It is not evidence about the web in general, and anyone quoting it as “only 2% of sites block AI” is quoting it wrong.

It also measures declared policy, not access. A site can allow an agent in robots.txt and still refuse it at the firewall. OpenAI states two conditions rather than one: allow the agent in robots.txt and allow requests from their published IP ranges. Reading robots.txt can never see the second, which is why we now check response codes by agent in server logs as a separate measurement.

What to do with this

Check whether your robots.txt names the search agents, not just the training ones. That is a two-minute job and it is the only part of the accidental-blocking story that survives contact with the data. If you are open, stop worrying about it: 91% of the sites winning citations in our category are simply open, so access is not what separates them.

Then spend the time you were going to spend on robots.txt on the thing that does separate them. In our own server logs, assistants fetch our pages built from original measurement roughly nine times as often as our product pages. Access gets you eligible. Evidence gets you cited.

Questions

Does blocking GPTBot remove me from ChatGPT's answers?

No. GPTBot is OpenAI's training crawler. The agent that decides whether you appear in ChatGPT's search answers is OAI-SearchBot, and OpenAI documents these as separate. Blocking GPTBot while allowing OAI-SearchBot keeps you in the answers and out of the training data, which is what 18 of the 259 sites in this study have done.

Are sites accidentally blocking themselves out of AI answers?

Among sites that get cited, almost none. We read the robots.txt of 259 domains an assistant actually cited and 235 of them allow every OpenAI agent. Only 5 block the search agent by name, and all 5 are large publishers or social platforms making a deliberate commercial choice. We expected the opposite and the measurement said otherwise.

Does blocking Google-Extended remove me from AI Overviews?

No. Google states that Google-Extended does not affect inclusion in Google Search, and AI Overviews are built on the Search index reached by Googlebot. Google-Extended governs training and grounding for Gemini and Vertex. There is no AI-only opt-out from AI Overviews that keeps you in Search.

What does robots.txt not tell you?

Whether the crawler was actually let through. A site can allow an agent in robots.txt and still refuse it at the firewall, and OpenAI states two conditions rather than one: allow the agent in robots.txt and allow requests from their published IP ranges. Reading robots.txt shows the declared policy. Only server logs show what happened.

Does a site with no robots.txt block anything?

No. No file means no restrictions, which is a real posture rather than missing data. 15 of the 259 domains here served no parseable robots.txt at all, and every one of them is open to every agent by default.

Do this automatically

Get Glotier Solo

Everything on this page is one measurement, taken by hand, on one day. Solo runs it for your product every day and writes the work it points to.

The same buyer questions re-asked every day, so you see the moment an answer changes
Every source page behind each answer, which is where the work actually is
An article written from what it measured that day, ready for you to publish
The subreddits and threads worth answering, with a reply drafted for each

Track 3 products and up to 150 buyer questions across ChatGPT, Gemini and Perplexity. $39/month.

Get Solo, $39/month Compare the plans

Cancel any time. Not for you? Email us within 7 days of a charge and we refund it in full. Refund policy

Not ready to pay? The check is free, with no card and no account. Run it on your own product.