XML sitemap

An XML sitemap (sitemap.xml) is a file that lists a website’s indexable URLs and when each one last changed. It tells crawlers which pages exist on the site and which ones have been updated recently.

The format is defined by the sitemaps.org protocol, version 0.90, and is supported by all major search engines. A single sitemap file can hold at most 50,000 URLs and must be no larger than 50 MB uncompressed. Larger sites split their URLs across several files and tie them together with a sitemap index file.

What a sitemap looks like

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/en/wiki/mention-rate</loc>
    <lastmod>2026-11-02</lastmod>
  </url>
</urlset>

The two fields that matter in practice are loc, the page address, and lastmod, the date the content last changed. The protocol also defines changefreq and priority, but they are optional hints.

How to submit it

There are two routes, and most sites use both:

• Reference it in robots.txt with a line such as Sitemap: https://example.com/sitemap.xml. Every crawler that reads robots.txt can find it this way, including AI crawlers.
• Submit it in each search engine’s webmaster tool, such as Google Search Console or Bing Webmaster Tools. This only applies to that one search engine, but it gives you reporting on how the URLs were processed.

What to list

List only the canonical URLs you want indexed:

• Match your canonicals. Each URL should be the same version declared by the page’s canonical link: same protocol, same host, same trailing slash convention.
• Leave out noindex pages. Listing a page you have told search engines not to index sends mixed signals.
• Leave out redirects. List the final address, not the one that redirects to it.
• Leave out paths blocked by robots.txt. A crawler that is not allowed to fetch the page cannot use the entry.

Keep lastmod honest. It should reflect real content changes. If every URL on the site shows the date of the last build, the field stops carrying information, and crawlers learn to ignore it.

What it does and does not do

A sitemap is a discovery channel, not an indexing promise. It helps crawlers find new pages that nothing links to yet, which matters most for large sites that update often (see crawl budget).

What it cannot do is make a page worth crawling or citing. Whether a page gets indexed, ranked or used as a citation source by an AI assistant depends on the content itself, not on whether it appears in the sitemap. A thin page listed in the sitemap is still a thin page.

Sitemaps and AI visibility

For AI search, the sitemap plays the same supporting role. Assistants that search the web rely on search indexes and their own crawlers, and both benefit from a clean, accurate list of URLs. Two practical points:

• When you publish new answer pages, glossary entries or service pages, make sure they appear in the sitemap with an accurate lastmod, so they are discovered quickly.
• When you retire or move pages, remove the old URLs and redirect them, so crawlers stop spending effort on addresses that no longer describe your business.

Common mistakes

• Including staging, test or parameter URLs
• Listing both http and https, or both www and non-www versions
• Forgetting to regenerate the sitemap after a site migration
• A sitemap URL in robots.txt that returns a 404

A sitemap check is part of the technical layer of an AI visibility audit; you can request one through the free GEO audit.