Sitemap is not listed in robots.txt
What it is
Asky uses a sitemap that no Sitemap: line in your robots.txt points to, directly or through a sitemap index. Asky found it another way: you added it in Asky, or it answered at a standard path such as /sitemap.xml. The finding sits on your robots.txt, since that is the file to change, and names the sitemap URL.
Why it matters
robots.txt is the file crawlers request before anything else, and its Sitemap lines are how crawlers you never submitted to, AI crawlers included, can find your sitemap. Google accepts the line as a submission, and Bing recommends it for automatic discovery. A sitemap submitted only in Search Console reaches Google alone.
How Asky checks it
During its daily site discovery and at the start of each technical audit, Asky compares each sitemap it reads on your domain with the Sitemap: lines in your robots.txt, ignoring http or https, a leading www., letter case and a trailing slash. A sitemap inside a listed index counts as listed; one in an unlisted index is reported too. Nothing is checked on another domain, or when the robots.txt fetch was blocked or got a server error. The finding names the sitemap and clears once it is listed or no longer used. When robots.txt is missing, empty or has no Sitemap line, every sitemap found is reported, on top of that file’s own issue.
Reported as an opportunity with low severity.
How to fix it
- Open the finding to see which sitemap URL it names.
- Add a line with that full URL to your robots.txt, such as
Sitemap: https://example.com/sitemap.xml. If the sitemap belongs to an index, list the index instead: one line then covers every sitemap inside it. - Adding a Sitemap line changes no crawl rules and raises no syntax error, and it can go anywhere in the file.
- If the finding names a leftover you do not use, such as an old platform’s
/sitemap.xml, list your real sitemap instead. Once robots.txt lists a sitemap, Asky stops trying the standard paths, and the old finding clears after the next complete discovery run. If you added the old sitemap in Asky, remove it there too.
Example
Asky found https://example.com/sitemap_index.xml at a standard path, but the robots.txt never mentions it:
User-agent: *
Disallow: /checkout/
# Added
Sitemap: https://example.com/sitemap_index.xml