How Companies Get Cited in ChatGPT Search: Crawlers, Content, Tracking

A sales director types a question into ChatGPT, something like “who supplies fire pumps in Riyadh” or “best logistics company for sea freight from Jebel Ali”, and gets an answer with a handful of cited sources. Your company is either one of those sources or it is not. Many owners now ask their marketing team, or their agency, to “get us into ChatGPT”. The honest starting point is that OpenAI publishes very little about how it chooses sources, and quite a lot about what a website owner can control.

This guide stays inside that documented ground. It uses OpenAI’s own Publishers and Developers FAQ and its crawlers documentation to explain the settings that decide whether you can appear at all, how to keep pages out properly, and how to measure the visits ChatGPT sends you. Where we go beyond the documentation, we say so, and we suggest how to test it.

What OpenAI actually says about appearing in ChatGPT search

The core statement is short. In its FAQ, OpenAI says: “Any public website can appear in ChatGPT search.” The condition it attaches is technical, not commercial: for your content to be included in summaries and snippets, make sure you are not blocking its search crawler, OAI-SearchBot. The FAQ describes no application form, no paid placement and no list of ranking factors.

When ChatGPT searches the web, OpenAI’s ChatGPT search help article says responses contain inline citations and a Sources button that lists the cited pages and other relevant links. The same article says ChatGPT may share disassociated search queries with third-party search providers such as Bing, and may share general location information based on the user’s IP address with those providers to improve the accuracy of results. (Parts of that help article describe plan availability and model limits that change often; we rely on it only for how search results are sourced and cited.) When OpenAI launched ChatGPT search on 31 October 2024, it said the feature “leverages third-party search providers” alongside content supplied directly by partners.

So there are two practical consequences. First, your site must be reachable by OpenAI’s crawler. Second, it helps to be well indexed in the wider search ecosystem those providers draw on. Everything else is about whether your pages deserve to be cited once they are found.

Three OpenAI bots, three separate decisions

A common mistake in robots.txt files is treating “OpenAI” as one thing. OpenAI’s bots page lists separate user agents with different jobs, and states plainly: “Each setting is independent of the others.” Google’s controls for AI Overviews and AI Mode work differently; see how sites can limit use in Google’s AI features.

This separation is the most useful fact for a business owner. You can decide you do not want your content used for model training and still want to be found and cited in ChatGPT search. OpenAI gives exactly that example: allow OAI-SearchBot, disallow GPTBot. It also notes that if you allow both, it may use the results of a single crawl for both purposes.

Step 1: Make sure nothing is quietly blocking OAI-SearchBot

Blocks rarely come from a deliberate decision. They come from a developer copying a “block all AI bots” snippet, a security plugin, or a hosting or CDN setting nobody reviewed. Check all three layers.

robots.txt

Open yoursite.com/robots.txt and look for any group that names OAI-SearchBot, or a broad rule under “User-agent: *” that disallows the whole site or your key sections. If you want search visibility without training use, the groups you want look like this:

• User-agent: OAI-SearchBot followed by Allow: / (or simply no disallow rules for it)
• User-agent: GPTBot followed by Disallow: /, only if you have decided to opt out of training

After any change, allow time. OpenAI says it takes “~24 hours from a site’s robots.txt update” for its search systems to adjust. Do not judge the result the same afternoon.

CDN, firewall and security plugins

A clean robots.txt does not help if the request is refused before it reaches the page. Many sites sit behind a CDN or web application firewall with bot controls. Cloudflare, for example, announced on 1 July 2025 that it was changing its default to block AI crawlers, and its current documentation treats training, search and agent crawlers as separate categories with their own settings. That documentation also describes updated defaults for new domains from 15 September 2026, under which search crawlers remain allowed while training and agent crawlers are blocked on pages that display ads. Defaults change and older domains may carry earlier settings, so check what your own dashboard actually applies. Whatever provider you use, ask whoever manages it to confirm in writing which AI crawler categories are blocked, challenged or allowed, and on which pages.

Server logs

The definitive check is your access log. Look for requests whose user agent contains “OAI-SearchBot” and whether they received a 200 response or a 403, 429 or challenge page. Because anyone can fake a user-agent string, verify real visits against the IP ranges OpenAI publishes for each bot (the bots page links a JSON file for OAI-SearchBot, GPTBot, ChatGPT-User and OAI-AdsBot). If your firewall allow-lists by IP, use those files rather than guesses.

Step 2: Keep pages out with noindex, not robots.txt

Some pages should not be cited: old price lists, internal tender documents that were accidentally made public, thank-you pages, staging copies. The instinct is to add them to robots.txt. OpenAI’s FAQ explains why that is not enough. If OpenAI obtains the URL of a disallowed page from a third-party search provider or by crawling other pages, and there are signals the page is relevant to a query, it says it may surface “just the link and page title” in ChatGPT Atlas, its browser.

To prevent that, OpenAI says to use the noindex meta tag, and that the crawler must be allowed to crawl the page to read it. This matches how Google describes noindex in its indexing documentation: if robots.txt blocks the page, the crawler “will never see the noindex rule”. The practical rule:

• Use <meta name=”robots” content=”noindex”> in the page head for HTML pages you want kept out.
• For PDFs and other files, Google documents an X-Robots-Tag: noindex HTTP header. OpenAI’s FAQ refers only to the meta tag, so treat the header as standard practice rather than an OpenAI commitment.
• Do not also disallow those URLs in robots.txt, or the tag will not be read.
• Anything genuinely confidential should not be public at all. Put it behind a login.

Step 3: Give the crawler pages worth citing

Here we leave OpenAI’s documentation, because it does not publish how sources are selected or ordered. What follows is observation and sensible practice, and you should test it on your own queries rather than take it on trust.

Be findable in conventional search first. Since OpenAI says it uses third-party providers such as Bing, a site that Bing has not indexed starts at a disadvantage. Some companies have only ever verified their site in Google Search Console. Adding the site to Bing Webmaster Tools and checking that key pages are indexed is a low-cost step. The technical basics (crawlable pages, clean canonicals, a working sitemap) are covered in our technical SEO audit guide.

Answer the question on the page, in text. A cited source usually has to contain the fact the answer needs. If your capabilities, product ranges, branch cities, certifications and approvals sit only inside images, sliders or a downloadable profile, there may be little for a text answer to quote. For Arabic pages, that means writing in the terms buyers actually use, as set out in Arabic SEO for Saudi B2B companies. For example, a contractor whose services page states “MEP contracting for hospitals and residential towers in Riyadh and Jeddah” gives a crawler something concrete; a page that only says “excellence in every project” does not.

One topic per page. A single “Services” page listing twelve offers in two lines each is harder to cite than twelve service pages that each explain scope, standards, who it is for and how to request a quotation.

Cover both languages properly. Buyers in Saudi Arabia, the UAE, Qatar and Egypt ask in Arabic and in English. If your Arabic pages are thin translations, or the Arabic site is a separate subdomain nobody maintains, you are relying on one language to carry both audiences.

Keep facts consistent across the web. Your company name, address, phone and service list should match across your site, your Google Business Profile, LinkedIn and directory listings. We cannot point to an OpenAI statement that this affects selection, but conflicting details give any system a reason to prefer another source.

Step 4: Measure what ChatGPT sends you

OpenAI’s FAQ says ChatGPT “automatically includes the UTM parameter utm_source=chatgpt.com in referral URLs”. That gives you a clean way to see the traffic in Google Analytics 4 or any analytics tool that reads UTM parameters.

1. In GA4, open Reports, Acquisition, Traffic acquisition and set the dimension to Session source or Session source / medium; Google’s campaign URL documentation says UTM values appear in that report. Look for chatgpt.com. Neither OpenAI nor Google documents exactly how these sessions are classified, so treat what you see as a starting point: some visits may arrive without the parameter (for example, when a link is copied) and show as referral or direct traffic instead.
2. Google notes that UTM values are case-sensitive and that missing parameters show as “(not set)”. OpenAI documents only utm_source, so expect the medium to look unusual and do not filter it out by accident.
3. Build an exploration that shows landing pages for chatgpt.com sessions. This tells you which pages are being cited, which is more useful than the total.
4. Connect it to outcomes. Count form submissions, WhatsApp clicks and quotation requests from those sessions, not just visits; setting those up as GA4 key events is covered separately.
5. Check your consent banner and tag setup. If analytics only fires after consent, part of this traffic will be invisible, as it is for every other source.

Expect the numbers to be small at first for most B2B sites. The value is in seeing which pages earn citations and whether those visitors become inquiries.

What no agency can honestly promise

Because OpenAI does not publish ranking factors, no one outside OpenAI can guarantee that your company will be named, cited first, or recommended for a given question. Be cautious of anyone selling “guaranteed ChatGPT placement” or a fixed number of AI mentions. A credible proposal for AI search visibility should instead commit to things that can be checked:

• A written review of robots.txt, CDN and firewall rules for OAI-SearchBot and other AI crawlers, with evidence from server logs.
• A clear decision, agreed with you, on training crawlers such as GPTBot, separate from search.
• Correct noindex handling for pages that should not appear.
• Indexing checks in Google and Bing for the pages that matter.
• Content work on service and product pages in Arabic and English.
• A tracked set of test questions and a GA4 view of chatgpt.com sessions and the inquiries they produce.

This is how we approach AI search visibility within RX Designer’s SEO and search work: the controllable settings first, then content, then measurement, with no promises about what an outside system will choose. When the gap is in the website itself (thin pages, content locked in images, a neglected Arabic version), it usually becomes a website structure and content project rather than a settings fix.

If you are not sure whether your site is visible to ChatGPT at all, start with the first check today: open your robots.txt and your CDN or firewall bot settings, confirm OAI-SearchBot is allowed, and note the date. A day later, search your server logs for OAI-SearchBot visits before you change anything else.