Which sources an AI assistant can actually read about your hotel

The short answer

Three jobs, not one “AI bot”

Every vendor publishes its user agents and what each one is for, and the published descriptions fall into three jobs. Treating them as one thing to allow or block is the most common error in advice on this subject, and it gets the decision backwards: refusing to feed a training set costs a hotel nothing in search, while refusing the answer-engine index removes it from the surface where guests are now asking.

What each source admits, by job

One row per source, measured on that host's representative listing URL. Split means the host treats agents doing the same job differently — the most interesting cell in the table, and the reason the detail below it exists.

Read the pattern rather than the cells: refusal concentrates in the first column. Three of the nine refuse an answer-engine or answer-time agent at all, and in two of those three — Trip.com and Google's travel pages — the page is closed to every crawler alike, for reasons that have nothing to do with AI. Airbnb is the only source that admits one vendor's answer agent to a listing page while refusing another's.

Source by source

The nine sources are the five channels a MisMatchMaker scan reads, the two large surfaces it does not, the corporate channel European operators meet most often, and Google's own travel pages. Each entry gives the deciding directive verbatim, because the difference between “explicitly allowed” and “allowed because nothing mentions it” is the difference between a decision and an accident.

#
# AI Search
#
user-agent: OAI-SearchBot
user-agent: ChatGPT-User
user-agent: PerplexityBot
user-agent: Perplexity-User
user-agent: Claude-User
user-agent: Claude-SearchBot
allow: /
disallow: /p/home
disallow: /*/p/home
disallow: /auth/
disallow: /*/auth/
disallow: /*/sso/

User-Agent: Amazonbot
User-Agent: Applebot-Extended
User-Agent: Bytespider
User-Agent: CCBot
User-Agent: ClaudeBot
User-Agent: Cohere-ai
User-Agent: GPTBot
User-Agent: Google-CloudVertexBot
User-Agent: Img2dataset
User-Agent: PanguBot
User-Agent: PetalBot
User-Agent: SentiBot
User-Agent: YouBot
User-Agent: magpie-crawler
User-Agent: meta-externalagent
Disallow: /

Six findings

Each one is a statement about nine files as published on the measurement date, not about what an assistant answered. That distinction is the whole reason to publish this rather than another prediction.

1. Finding 1

   Refusals land on training, not on answering

   Across the nine sources, the agents that read a page in order to answer a question are admitted almost everywhere: on six of the nine, every answer-engine and answer-time agent may read the representative listing page. The refusals cluster on the training crawlers — Tripadvisor refuses five of them outright, Airbnb three — and of the three sources that do refuse an answer agent, two close the page to every crawler alike for reasons that have nothing to do with AI.

   So what: The common worry, that a hotel's content is being hoovered up for training, is the one the channels have mostly already acted on. The thing actually happening is that assistants read live listing pages to answer questions about your property, which makes those pages' accuracy an operational concern rather than a legal one.

2. Finding 2

   Two channels admit AI agents by name, with more access than they give a search engine

   Expedia and Hotels.com both carry a group commented “# AI Search” that names OAI-SearchBot, ChatGPT-User, PerplexityBot, Perplexity-User, Claude-User and Claude-SearchBot, and opens with “allow: /”. Because a named group replaces the wildcard group entirely, those six agents are not bound by the sixty-odd wildcard exclusions that constrain an ordinary crawler on the same host.

   So what: On the Expedia Group storefronts, the AI answer surfaces are a first-class audience with privileged access. Content that is wrong there is wrong on the surface the channel has gone furthest to feed.

3. Finding 3

   The same hotel can be readable to one assistant and invisible to another

   Airbnb admits OpenAI's search agent to its listing pages and refuses Perplexity's, through two extra lines in an otherwise identical group. Tripadvisor admits Perplexity and Claude to a listing whose location id begins “-g188”, and refuses ChatGPT's two agents on the same page. Neither asymmetry is visible to an operator, and neither follows a principle an operator could anticipate.

   So what: There is no single AI surface to manage. A guest's answer depends on which assistant they used, and the differences are set in files you do not control and cannot negotiate with.

4. Finding 4

   Your Google listing is the one source no external assistant may read

   Google's own robots.txt refuses “/travel/entity” and “/travel/search” to every crawler in the file, with no AI agent named. Google documents that AI Overviews and AI Mode are governed by Googlebot's own access and the ordinary snippet controls, and that no special markup or AI file is needed to appear in them.

   So what: Perfecting the Google hotel panel improves Google's surfaces and does nothing for ChatGPT, Claude or Perplexity. Those read the OTA listings and your own website instead — which is where the effort belongs if an AI answer is what you are trying to fix.

5. Finding 5

   A listing can be closed to everything by a line nobody meant as AI policy

   Trip.com names no AI agent, yet its canonical hotel-detail URL is refused to every crawler by one wildcard line, “Disallow: /hotels/detail/?hotelId=*”. The same page with its two query parameters reversed is allowed, because “?” is a literal character in robots.txt rather than a wildcard.

   So what: Readability is often an accident of URL construction rather than a decision. It also means a channel can vanish from AI answers about your hotel without anyone, including the channel, intending it.

6. Finding 6

   Block lists age faster than the crawlers they name

   HRS refuses thirty-one agents and not one of them is GPTBot, ClaudeBot, OAI-SearchBot, Claude-SearchBot or PerplexityBot. The list names Bytespider, cohere-ai, Diffbot, FacebookBot, YouBot and Timpibot — a careful policy, written against an earlier generation of crawler names.

   So what: Treat every verdict on this page, including the permissive ones, as a reading of a file on one date. That is also the argument for re-running the check rather than citing it a year from now.

The one file you control: your own site's robots.txt

Your website is the only source in the chain you own end to end, and the decision there is per job rather than per vendor. The usual advice — block everything with “AI” in its name — removes your own site from the surfaces where guests ask questions and leaves your OTA listings to answer for you.

Check your own hotel in about ten minutes

Nothing here needs a tool or a login. Step three is the one that decides what an assistant says about you, and it is the step a listing scan automates.

1. 1

   List the sources that describe you, not the ones you manage

   Search your hotel's name and write down every public page about it on the first two pages of results: each OTA listing, the review sites, the aggregators you have never logged into, your own site. An assistant answering a question about you draws from this set, not from your extranet.

2. 2

   Check what each one admits

   Open “https://<that domain>/robots.txt” and search it for GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot and ClaudeBot. A name with “Disallow: /” under it means refused; no mention at all usually means admitted, unless a wildcard rule happens to catch the listing path. “node scripts/check-ai-crawler-access.mjs” does this for the nine sources on this page and prints the deciding line.

3. 3

   Compare what the admitted sources say about you

   Take the facts a guest asks about — breakfast, parking, check-in time, pets, cancellation, fees — and read them off each admitted source side by side. Where two admitted sources disagree, an assistant will eventually repeat one of them, and you do not get to choose which. This is the step a listing scan automates across the five channels it reads.

4. 4

   Fix the source, then re-read the source

   Correct the field in the channel's extranet, then come back to the public page rather than the extranet and confirm the change is visible there. Assistants read the public page, which is also the thing an extranet can show as updated while the page still carries the old value.

5. 5

   Decide your own site's robots.txt deliberately

   Your website is the only source in the chain you control end to end. Decide per role rather than per vendor: refuse training if you prefer, admit the answer-engine indexers, and keep the site indexable with ordinary snippets if you want to appear in Google's AI features. Blanket-blocking every agent with “AI” in its name removes you from the answer surfaces and leaves the OTA listings to speak for you.

node scripts/check-ai-crawler-access.mjs

What this measurement cannot establish

This list is not a disclaimer at the bottom of a confident page. It is the reason the page is worth publishing: most writing about hotels and AI search asserts behaviour nobody measured, and being explicit about the gap between a published directive and an observed answer is the difference.

• A robots.txt records what a host asks crawlers to do. It is not evidence that any assistant read, or did not read, a page. Nothing here was established by observing an assistant's output.
• Two vendors document that their answer-time agents may not apply robots.txt at all — Perplexity-User “generally ignores” it and OpenAI says the rules may not apply to user-initiated fetches — so a refusal is weakest exactly where a guest's question is strongest.
• Commercial licensing is invisible to this method. A channel can refuse every crawler in its file and still supply content to an assistant under an agreement, and that arrangement would not appear in any file we fetched.
• Each verdict is measured on one representative listing URL per host. A host can admit an agent to the page we tested and refuse it on another path, as Airbnb and Trip.com both do within a single file.
• These files change without notice, and two of the nine changed shape in the week they were first read. Treat every verdict as dated, and re-run the script rather than citing this page as current a year from now.
• Tripadvisor's sitemap refused automated retrieval with HTTP 403, so the geographies behind its “-g188” and “-g1875” exclusions are recorded as unenumerated rather than guessed at.
• Tripadvisor's file also names “Gemini-Deep-Research” alongside OpenAI's agents. That name does not appear in either of Google's published crawler lists as accessed on the measurement date, so it is listed here as unidentified rather than described.
• The set is nine sources, not a sample. It covers the channels this product reads plus four surfaces operators ask about; it does not support a claim about OTAs in general.

What this page is not

• This is not a ranking guide. Nothing here establishes how an assistant chooses between two permitted sources, and we have not measured it.
• It is not an audit of what assistants currently say about any hotel. That would need a sampled, repeated test against live assistants, which is a different study and is not what this page is.
• It does not cover rate or availability data in AI answers. Those travel through connectivity and advertising products, not through the content pages measured here.
• It is not legal advice about whether a host may use your content, and it takes no position on training-data disputes. It reports what nine files ask for.
• The scanner does not read Tripadvisor, Airbnb, HRS or Google's travel pages. They are measured here because guests ask assistants about them, and the scope note says which five channels a scan actually compares.

Scanner scope: A MisMatchMaker scan reads a property's public listings on Booking.com, Expedia, Hotels.com, Agoda and Trip.com and compares the factual fields they have in common. That is the subset of this page's nine sources the product can act on, and the four it cannot — Tripadvisor, Airbnb, HRS and Google's travel pages — still appear here because an assistant may read them. A scan tells you whether the sources an assistant is most likely to read agree about breakfast, parking, arrival times, pets, cancellation and fees. It does not tell you what any assistant said, and no result on this site should be read as a measurement of an AI answer.

Frequently asked questions

How do AI assistants decide what to say about my hotel?

For a factual question about a specific property, an assistant generally reads pages at the moment it answers rather than reciting something memorised. Which pages it can read is stated in each source's robots.txt. Measured on 17 October 2026 across nine sources, every agent that fetches a page to answer a question is admitted on six of the nine, while the crawlers that collect training data are the ones getting refused. So in practice an answer about your hotel is assembled from the OTA listings and other public pages an assistant is permitted to read, which means its accuracy is decided by whether those pages agree with each other.

Should I block AI crawlers in my hotel website's robots.txt?

Not as one decision, because the agents do three different jobs. Refusing the training agents is close to free: Google states that Google-Extended does not affect inclusion in Google Search and is not a ranking signal, and Apple states that pages disallowing Applebot-Extended can still appear in search results. Refusing the answer-engine indexers is not free — OpenAI recommends allowing OAI-SearchBot so a site appears in ChatGPT's search results, and Perplexity asks owners to allow PerplexityBot to stay visible. Blocking everything with “AI” in the name removes your own site from the surfaces where guests now ask questions and leaves your OTA listings to answer for you.

Do I need special schema or an AI file to appear in AI Overviews?

No. Google's documentation states that you do not need to create new machine readable files, AI text files, or markup to appear in these features, and that there is no special schema.org structured data you need to add. It describes AI as built into Search, with robots.txt directives for Googlebot as the control site owners have, and the ordinary preview controls — nosnippet, data-nosnippet, max-snippet and noindex — as the way to limit what is shown. To appear as a supporting link a page must be indexed and eligible to be shown with a snippet. A max-snippet or nosnippet rule added to protect content will also limit what Google's AI features can show from it.

Why can't I fix an AI answer by fixing my Google listing?

Because no external assistant can read it. Google's own robots.txt refuses /travel/entity and /travel/search to every crawler in the file, with no AI agent named, so a hotel's Google travel pages are closed to ChatGPT, Claude and Perplexity alike. Google's own products reach that content through Google's internal pipelines rather than by crawling it. A perfect Google panel improves Google's surfaces and changes nothing about what another assistant says, because that assistant is reading your OTA listings and your website instead.

Does every assistant see the same thing about my hotel?

No, and the differences are arbitrary. Airbnb admits OpenAI's search agent to its listing pages while refusing Perplexity's indexer through two extra lines in an otherwise identical group. Tripadvisor refuses ChatGPT's agents on listings whose location id begins -g188 or -g1875 while still admitting Claude and Perplexity to the same pages. Neither difference is visible to an operator or follows a rule you could anticipate, so there is no single AI surface to manage — only the underlying facts, which you can make consistent across every source at once.

Sources

Two kinds. The vendor documentation establishes what each agent is for; the nine robots.txt files are the measurement itself, and are the only evidence behind every verdict on this page. All were opened on 17 October 2026.

What each agent is for

1. Overview of OpenAI crawlers — OpenAI developer documentation, accessed 2026-10-17
2. Does Anthropic crawl data from the web, and how can site owners block the crawler? — Anthropic support, accessed 2026-10-17
3. Google crawlers (user agents) overview — Google Search Central, accessed 2026-10-17
4. About Applebot and Applebot-Extended — Apple Support, accessed 2026-10-17
5. CCBot — Common Crawl, accessed 2026-10-17
6. PerplexityBot and Perplexity-User — Perplexity documentation, accessed 2026-10-17
7. AI features and your website — Google Search Central, accessed 2026-10-17
8. User-triggered fetchers — Google Search Central, accessed 2026-10-17

The nine files measured

1. www.booking.com/robots.txt — Booking.com, accessed 2026-10-17
2. www.expedia.com/robots.txt — Expedia, accessed 2026-10-17
3. www.hotels.com/robots.txt — Hotels.com, accessed 2026-10-17
4. www.agoda.com/robots.txt — Agoda, accessed 2026-10-17
5. www.trip.com/robots.txt — Trip.com, accessed 2026-10-17
6. www.tripadvisor.com/robots.txt — Tripadvisor, accessed 2026-10-17
7. www.airbnb.com/robots.txt — Airbnb, accessed 2026-10-17
8. www.google.com/robots.txt — Google's hotel surface, accessed 2026-10-17
9. www.hrs.com/robots.txt — HRS, accessed 2026-10-17

Related references on this site: how OTA content updates propagate explains why a corrected field can take days to reach the public page an assistant reads, and the listing consistency study measures how often a hotel's own listings disagree in the first place. Why listing consistency matters makes the commercial case, and the methodology sets the standard this page is held to: claims narrowed to what the source supports, gaps left visible, and nothing inferred presented as documented.