robots.txt does not list your sitemap

What it is

Your robots.txt has content but no Sitemap: line. That line gives the full URL of your XML sitemap or sitemap index, and it can go anywhere in the file and is not tied to any user-agent group. It is how any crawler can find your sitemap without you submitting it to each search engine one by one.

Why it matters

Google and Bing both accept a Sitemap line in robots.txt as a way to submit a sitemap . Bing says it then rereads that sitemap, typically at least once a day , for search and for Copilot. Crawlers you never submit to, including AI crawlers, are likely to rely on this line or on guessing a common path, so new pages reach them more slowly without it.

How Asky checks it

Asky reads your robots.txt during its daily site discovery and at the start of each technical audit. The issue is reported when the file has content but no Sitemap: line with a value. A blank file is reported as an empty robots.txt instead, and a missing or failing file under its own issue. The URL in the line is not validated here: Asky fetches it as your sitemap, reporting a 404 as sitemap missing and a 5xx as a server error, while a relative or unreachable URL raises nothing. Each sitemap Asky finds some other way is also reported as not listed in robots.txt.

Reported as an error with high severity.

How to fix it

1. Find your sitemap’s URL. Common ones are /sitemap.xml, /sitemap_index.xml and WordPress’s /wp-sitemap.xml. Open it and check it shows XML.
2. Add a line with its full URL, including https://, such as Sitemap: https://example.com/sitemap.xml. Several sitemaps take one line each.
3. If you have a sitemap index, listing the index is enough: the sitemaps inside it count as listed.
4. Use the URL the sitemap answers on directly, not an old address that redirects to it.
5. Sitemap is a valid field, so the new line raises no syntax error and changes no crawl rules. Do not put it after a #, which turns it into a comment.

Example

https://example.com/robots.txt has rules but never names the sitemap. The last line fixes it:

User-agent: *
Disallow: /checkout/

Sitemap: https://example.com/sitemap_index.xml

← Back to Robots.txt