- Checks a live URLFreeRedirect and Status Code CheckerFollow every redirect hop, see each status code and the final URL, and spot chains and loops.Use the tool →
- Checks a live URLFreeHTTP Header CheckerView response headers including cache, security and X-Robots-Tag directives.Use the tool →
- Runs in your browserFreeRobots.txt GeneratorBuild a robots.txt file with presets for search engines and AI crawlers.Use the tool →
- Checks a live URLFreeRobots.txt Tester and AI Crawler CheckerTest whether Googlebot, Bingbot, GPTBot, ClaudeBot or PerplexityBot may crawl a URL under the robots.txt standard, and check for llms.txt.Use the tool →
- Runs in your browserFreeXML Sitemap GeneratorPaste a list of URLs and download a valid XML sitemap.Use the tool →
- Checks a live URLFreeXML Sitemap ValidatorCheck a sitemap's format and URL count, and sample its URLs for errors and noindex conflicts.Use the tool →
- Runs in your browserFreeHtaccess Redirect GeneratorGenerate 301 redirect rules for Apache and LiteSpeed servers.Use the tool →
- Runs in your browserFreeHreflang Tag Generator and CheckerBuild hreflang tags and validate language and region codes against the ISO lists.Use the tool →
- Checks a live URLFreeBroken Link CheckerCheck every link on a single page and report its status code.Use the tool →
Free technical SEO tools check whether search engines can reach, crawl and index your pages. The set here follows redirects, reads HTTP headers, builds and tests robots.txt, creates and validates sitemaps, builds hreflang tags, finds broken links and tests whether AI crawlers are allowed in.
Crawl files are everywhere on the web: the 2024 Web Almanac found that 83.9% of mobile sites returned a 200 status for robots.txt, and canonical tags on 65% of mobile pages. Having the file is common, so the real question is whether yours says what you intend.
What technical SEO tools check
Technical SEO is the layer under your content. If a page returns an error, redirects through three hops, is blocked in robots.txt or carries a stray noindex directive, its quality is irrelevant because it will not be crawled or indexed properly. The tools here test that layer one question at a time.
They fall into four groups: how a URL responds (redirect and header checkers), what you tell crawlers (robots.txt and sitemap tools), how you signal language and region (hreflang) and how healthy your links are (broken link checker). Some generate files for you, and some read live pages.
Redirects, status codes and headers
Redirect and Status Code Checker
Enter a URL and the tool follows every redirect hop, showing each status code and the final URL, so chains and loops are easy to spot. This is a server-checked tool, because a browser hides intermediate hops from page scripts.
- One hop, 301 to the final page: healthy for a permanent move.
- Two or more hops: a chain. Point the first URL straight at the final destination.
- A loop: the URL redirects back to itself, so nobody can reach it. Fix this first.
- A 302 where you meant a permanent move: change it to a 301.
- A 404 or 5xx at the end: the destination is broken, so the redirect leads nowhere useful.
HTTP Header Checker
This tool shows the response headers a server sends, including cache and security headers and any X-Robots-Tag directives. The one to look for first is X-Robots-Tag. A noindex sent in a header is invisible in the page source, so people lose hours searching the HTML for a directive that is not there. Cache headers show how long browsers and CDNs may keep a copy, which matters when you update a page and still see the old version.
Htaccess Redirect Generator
If your site runs on Apache or LiteSpeed, this browser-based tool generates 301 redirect rules you can paste into your .htaccess file. Keep a backup before editing, because one typo in that file can take a site down. After saving, run the Redirect and Status Code Checker on a few old URLs to confirm each lands on the right page in a single hop.
Robots.txt and AI crawlers
Robots.txt Generator
The generator builds a robots.txt file in your browser, with presets for search engines and AI crawlers. Use it to produce a clean starting file rather than editing from memory. Add your sitemap location, block only areas that have no search value, such as internal search results or cart pages, and leave everything else open.
Robots.txt Tester and AI Crawler Checker
This server-checked tool tests whether Googlebot, Bingbot, GPTBot, ClaudeBot or PerplexityBot may crawl a URL under the robots.txt standard, and it checks for an llms.txt file. Pick the URL you care about, run the test for each bot and read the verdict.
Remember what robots.txt does. Google’s documentation is clear that robots.txt is not a mechanism for keeping a web page out of Google. A blocked URL can still appear in results if other sites link to it. To keep a page out of the index, use noindex or password protection, and make sure the page is not also blocked, because a crawler that cannot fetch the page cannot see the noindex.
Whether to allow AI crawlers is a business decision. Allowing them can make your content eligible to be read and cited by AI tools, and blocking them keeps it out. The tester shows what your current file does, and you choose the policy. Also note that robots.txt is a convention: well-behaved bots follow it, and others may not.
Sitemaps, hreflang and broken links
XML Sitemap Generator and Validator
Paste a list of URLs into the generator and download a valid XML sitemap. It runs in your browser, which suits small sites and quick jobs. Larger sites should generate sitemaps automatically from their CMS.
The validator checks a sitemap’s format and URL count, and samples its URLs for errors and noindex conflicts. A sitemap should list only the URLs you want indexed, each returning a 200 status. If the sample shows redirects, 404s or noindex pages, remove or fix them. A sitemap full of such URLs wastes crawler attention and sends mixed signals. Because the check samples URLs, it is an indicator, not a full crawl of a large site.
Hreflang Tag Generator and Checker
If you publish the same content for different languages or regions, hreflang tells search engines which version suits which audience. This browser-based tool builds the tags and validates language and region codes against the ISO lists. The mistakes it catches most often are invalid codes, such as using a country where a language is expected, and the format of the codes themselves.
Two rules apply beyond what a generator can enforce. Each page should reference itself and all its alternates, and the alternates must point back to each other. Check those reciprocal links on the live pages after you add the tags.
Broken Link Checker
This server-checked tool checks every link on a single page and reports the status code of each. Fix 404s by updating the link to the right page or removing it, and replace links to pages that redirect with the final URL. Internal broken links deserve priority because they waste crawl paths and frustrate visitors. It checks one page at a time, so run it on your homepage, key landing pages and the pages that get the most links.
How these tools work and their honest limits
- Server-checked tools: the Redirect and Status Code Checker, HTTP Header Checker, Robots.txt Tester and AI Crawler Checker, Sitemap Validator and Broken Link Checker send a request from my server to the public URL. They see what the server returns, not content added later by JavaScript.
- Browser-only tools: the Robots.txt Generator, XML Sitemap Generator, Htaccess Redirect Generator and Hreflang Tag Generator run on your device. What you enter is never uploaded.
- Blocking: a firewall or bot protection may block my server. The tool can then only report a refusal, which says little about how Googlebot is treated.
- Standards, not guarantees: the robots.txt tester reports what the robots.txt standard says. It cannot promise how any crawler behaves in practice.
The SEO tools hub explains privacy and rate limits for server checks in more detail.
A workflow for a technical check
- Run the Redirect and Status Code Checker on your homepage and top pages, in both the http and https and www and non-www versions, and fix chains.
- Check headers on any page that is missing from search to rule out an X-Robots-Tag noindex.
- Test robots.txt for Googlebot and Bingbot, then for the AI crawlers you have a policy on.
- Validate your sitemap and clear out URLs that redirect, error or carry noindex.
- Run the Broken Link Checker on your key pages.
- If you serve several languages, build and verify hreflang tags.
- Finish with the page-level checks in the on-page SEO tools.
When you change a redirect rule, a robots.txt line or a sitemap, repeat the relevant check straight afterwards. Most technical disasters come from a small edit nobody verified.
When a professional audit helps
A single-URL tool cannot show you patterns. If thousands of pages are crawled but not indexed, a migration has broken redirects at scale, JavaScript hides content from crawlers, or server logs show Googlebot spending its time on useless URLs, you need a crawl of the whole site and someone to interpret it. The same goes for international setups, where hreflang errors interact with canonicals and redirects.
That is the point where a free tool has done its job by telling you there is a problem, and an audit finds the cause and orders the fixes by impact.
Questions about technical SEO tools
Does blocking a page in robots.txt remove it from Google?
No. Robots.txt controls crawling, not indexing, so a blocked URL can still appear in results if other pages link to it. Use noindex on a crawlable page, or password protection, to keep it out.
Is a 301 or a 302 redirect better?
Use a 301 when a move is permanent and a 302 when it is temporary. The Redirect and Status Code Checker shows which one your server sends, so you can correct the wrong choice.
Should I block AI crawlers like GPTBot?
That depends on whether you want your content available to AI tools. The Robots.txt Tester shows your current position for each bot, and the generator has presets for either choice.
Why does a tool say it cannot fetch my page?
Many sites block automated requests with firewalls or bot protection, and the server may be refused. It can also happen with pages behind a login. Try again later or review the page manually.
If your checks keep turning up problems you cannot trace, my technical SEO services start with a full crawl, and you can ask for a free audit through the contact page.