AI Search Readiness Checker

An AI search readiness check shows whether a page can be crawled, read and understood by AI search systems such as ChatGPT search. The free AI Search Readiness Checker reads a page’s source and its site’s robots.txt, then reports on crawler access, indexing directives, page structure and structured data. Our server fetches the public address you enter and the result is not kept.

OpenAI explains in its overview of OpenAI crawlers that OAI-SearchBot is the crawler that finds pages for ChatGPT search, while GPTBot is a separate crawler tied to model training, and a site can allow or block each one on its own in robots.txt.

How to use the AI Search Readiness Checker

1. Type or paste the full address of the page you want to check, starting with https://.
2. Press Check page and wait a few seconds while the tool fetches the page and the site’s robots.txt file.
3. Read the summary first: how many checks passed, how many are worth fixing and how many are blocking issues.
4. Open each section to see the detail behind a mark, starting with Crawler access.
5. Fix the blocking issues, publish the change and run the check again on the same address.

Check the pages that matter most to your business first, such as your home page, your main service pages and your best guides. A single clean page tells you little about the rest of the site.

How to read the results

The results are grouped into four sections. Each row carries a mark: a tick means the signal is in place, an exclamation mark means it is worth improving, a cross means something may stop the page being read or shown, and an i is for information only.

• Crawler access: whether robots.txt lets OAI-SearchBot, ChatGPT-User, GPTBot, PerplexityBot, Claude-SearchBot and ClaudeBot fetch the page, and which rule decided it. It also notes whether an llms.txt file exists.
• Indexing signals: noindex and nosnippet directives in the page or its HTTP headers, and whether the canonical tag points where you expect.
• Content structure: the title, meta description, headings, how much text is in the page source, the opening paragraph and question-style headings.
• Entity and trust signals: structured data, who is named as the author or publisher, and whether a publication or update date can be read by a machine.

Blocking a search crawler and blocking a training crawler are different choices. The tool marks a blocked search crawler as a problem and a blocked training crawler as information only, because many sites choose to opt out of training while staying visible in AI search. If you also want one file that carries the full text of your key pages, use the llms-full.txt generator.

Common problems this finds

• A blanket block: a robots.txt rule such as Disallow: / for all crawlers, often left over from a development site, keeps every crawler out.
• A noindex tag left on: a page marked noindex asks search systems to leave it out, however good the content is.
• Content that only exists after JavaScript runs: if the page source holds almost no text, crawlers that do not run scripts may see an empty page.
• Missing or wrong canonical tags: a canonical pointing at another address tells systems to treat that address as the main version.
• No named author or date: guides without a visible author and update date give systems fewer signals about who stands behind the content and how current it is.
• Broken structured data: JSON-LD with a syntax error is usually ignored. The JSON-LD Validator finds those errors.

Worked example (illustrative)

Imagine a Karachi dental clinic whose new site was built on a staging copy. The staging robots.txt blocked all crawlers and was copied across when the site went live. Run on the clinic’s treatment page, the checker would show OAI-SearchBot as blocked by Disallow: /, a cross in Crawler access and a blocking issue in the summary.

The fix is to replace the blanket rule with one that allows the public pages and blocks only the admin area. After the change is live, running the same address again should show the crawlers as allowed and the cross turn into a tick. The page content did not change at all; only the access rule did.

This is a made-up example to show the process. What you see on your own site depends on how it is set up.

Why it matters for AI search

AI search tools answer questions by finding pages, reading them and sometimes linking to them. A page that a crawler cannot fetch, or that tells systems to leave it out, cannot take part. Beyond access, clear headings, a direct opening answer, named authors and readable dates make a page easier to understand and quote accurately.

These are the same habits that help ordinary search, so the work is rarely wasted. Fixing access and structure is the part you control. Whether a system then chooses to cite your page is its decision.

Limits to know about

The checker reads one page and the site’s robots.txt at one moment. It does not run JavaScript, so it shows what a simple crawler sees in the page source rather than what a browser shows. The question and opening-paragraph checks are rough guides, not rules.

It cannot tell you whether ChatGPT or any other system will cite a page, how often it is crawled or how it ranks. For a wider review of a page’s on-page elements, use the On-Page SEO Checker, and to test one robots.txt rule in detail use the Robots.txt Tester and AI Crawler Checker.

Questions about the AI Search Readiness Checker

If you would like the access, structure and entity signals of your whole site reviewed properly, my technical SEO services cover that work, and you can ask for a free audit through the contact page. You can also browse the rest of the free technical SEO tools.