To add a sitemap to robots.txt, put a line that starts with Sitemap: followed by the full, absolute URL of your sitemap file anywhere in the file. Search engines read it when they fetch robots.txt, so it helps them discover your sitemap even if you have not submitted it elsewhere.
The exact format
Sitemap: https://www.example.com/sitemap.xmlThe URL must be complete, including https and the domain. A relative path such as /sitemap.xml is not valid for this directive.
Where the line goes
The Sitemap line is independent of user-agent groups. You can place it at the top or bottom of the file and it applies regardless of which bot rules sit around it. Most people put it at the end so the file stays easy to read.
Not sure if this is hurting your site?
Get the fixes that matter most, in priority order.
Adding more than one sitemap
List each sitemap on its own line, or point to a single sitemap index file that references the others.
Sitemap: https://www.example.com/sitemap-pages.xml
Sitemap: https://www.example.com/sitemap-posts.xmlDoes this replace Search Console?
No. Submitting the sitemap in Google Search Console still gives you reporting on discovered and indexed URLs. The robots.txt line is a complementary signal that also works for other search engines. For the basics of the file itself, read what an XML sitemap is and how to submit one.
Common mistakes
- Using a relative URL instead of a full one.
- Pointing to a sitemap on a different protocol or host than the site actually uses.
- Listing a sitemap that returns an error or redirects.
- Blocking the sitemap URL itself with a Disallow rule.
Before you publish, build a clean file with my free robots.txt generator, create the sitemap with the XML sitemap generator and confirm it is valid using the sitemap validator.
Frequently asked questions
Can I add a sitemap to robots.txt on a different domain?
The directive supports full URLs, but keep it simple and use sitemaps on your own domain to avoid confusion.
Is the Sitemap line case sensitive?
The directive name is not, but the URL after it is, so copy it exactly.
Next steps
Once the file is live, test it with the robots.txt tester. If you want a technical review of your site, contact me here.
More in this series (robots.txt questions): start with Free Robots.txt Generator, then explore:
- Robots.txt Wildcards: How * and $ Work With Examples
- Indexed, Though Blocked by Robots.txt: Why It Happens and How to Fix It
- WordPress Robots.txt: Where It Is, How to Edit It and What to Include
- Does Google Respect Crawl-Delay in Robots.txt?
Request received
Thank you! Saad will contact you within the next 24 hours.
Keep an eye on your phone and inbox.