Technical AEO

    XML Sitemaps and lastmod for AI Search

    How XML sitemaps and accurate lastmod dates help Google and Bing recrawl the pages AI answers draw from, with sample XML, IndexNow and a quick audit.

    Saarthak Srivastav

    Saarthak Srivastav

    · 6 min read

    Key takeaways

    • Sitemaps don't put you in AI answers, but they help the search indexes behind those answers find and recrawl your pages faster.
    • lastmod is the one optional field that matters: Google and Bing use it for crawl scheduling only when it reflects real content changes.
    • Google and Bing both ignore changefreq and priority, so you can leave them out.
    • List only canonical, indexable URLs that return 200, and split large sites into typed sitemaps under a sitemap index.
    • Google retired its sitemap ping endpoint in 2023; use robots.txt, Search Console, Bing Webmaster Tools and IndexNow instead.

    XML sitemaps won't get you into AI answers on their own, but they control how quickly the search indexes behind those answers find your new pages and notice your updates. Google's AI Overviews and AI Mode are built on Google's index, and Microsoft Copilot is grounded in Bing, so a sitemap with honest lastmod dates is one of the cheapest ways to keep AI answers working from your current content. This post covers which sitemap fields matter, how to set lastmod correctly, how to structure and submit sitemaps, and a 15-minute audit.

    A sitemap tells search engines which URLs exist and when they last changed, which is what decides how fresh an AI answer can be. When an assistant grounds its answer in a search index, it can only use what that index has crawled. If your new pricing page or updated comparison isn't recrawled, the answer keeps quoting the old version.

    Bing made this point directly in a July 2025 Bing Webmaster blog post on sitemaps in AI-powered search, describing sitemaps and lastmod as key signals for discovering and recrawling content. For the broader picture of how freshness affects answers, see content freshness for AI search.

    What search engines actually read

    Only two sitemap fields do real work today: loc and lastmod. Here is how the main tags are treated, based on Google's and Bing's published guidance.

    Tag Required? How Google and Bing treat it
    <loc> Yes The absolute, canonical URL of the page
    <lastmod> No Used as a crawl-scheduling signal, but only if it's accurate
    <changefreq> No Ignored by Google and Bing
    <priority> No Ignored by Google and Bing

    Google's June 2023 Search Central post (the one announcing the end of the ping endpoint) said it uses lastmod when the value is consistently and verifiably accurate, and that it still doesn't use changefreq or priority. Bing's July 2025 guidance says the same about changefreq and priority, and recommends lastmod in ISO 8601 format with a time component.

    How to set lastmod correctly

    Set lastmod to the date the page's main content last changed in a meaningful way, not the date your sitemap was generated. That single rule is where most sites go wrong, and Bing's guidance explicitly warns against stamping the sitemap generation time on every URL.

    Changes that justify a new lastmod:

    • Rewritten or added body content, answers or sections
    • Changed facts: prices, specs, plan limits, policies
    • New or corrected structured data
    • Significant changes to the links in the main content

    Changes that don't:

    • Footer, header or sidebar edits
    • A new copyright year
    • Rebuilding or redeploying the site
    • Fixing a typo

    The practical fix is in your CMS: store a separate content_modified_at field that editors update deliberately (or that changes only when the body field changes), and generate lastmod from it rather than from the database's generic updated_at timestamp.

    A correct sitemap entry

    <?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
      <url>
        <loc>https://www.example.com/pricing/</loc>
        <lastmod>2026-10-14T09:30:00+00:00</lastmod>
      </url>
      <url>
        <loc>https://www.example.com/compare/example-vs-competitor/</loc>
        <lastmod>2026-09-02T16:05:00+00:00</lastmod>
      </url>
    </urlset>
    

    The file is UTF-8, URLs are absolute and match the canonical exactly (including the trailing slash and https://www), and dates use the W3C datetime format.

    Structuring sitemaps for a larger site

    Split sitemaps by content type and reference them from a sitemap index, so you can see coverage and freshness for each section separately. The sitemaps.org protocol limits each sitemap to 50,000 URLs and 50MB uncompressed, and a sitemap index can list up to 50,000 sitemaps.

    <?xml version="1.0" encoding="UTF-8"?>
    <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
      <sitemap>
        <loc>https://www.example.com/sitemaps/pages.xml</loc>
        <lastmod>2026-10-14T09:30:00+00:00</lastmod>
      </sitemap>
      <sitemap>
        <loc>https://www.example.com/sitemaps/blog.xml</loc>
        <lastmod>2026-10-21T12:00:00+00:00</lastmod>
      </sitemap>
      <sitemap>
        <loc>https://www.example.com/sitemaps/docs.xml</loc>
        <lastmod>2026-10-18T08:15:00+00:00</lastmod>
      </sitemap>
    </sitemapindex>
    

    Typed sitemaps (pages, blog, docs, products, integrations) also make it easy to spot a section that isn't getting indexed in Search Console. Keep docs and help centers on subdomains in their own sitemaps, verified as their own properties.

    Only include URLs that:

    • Return a 200 status (no redirects, no 404s)
    • Are the canonical version (see canonicals and AI citations)
    • Aren't blocked by robots.txt or tagged noindex
    • You actually want cited: thin tag pages and internal search results don't belong

    Submitting and notifying search engines

    Reference your sitemap in robots.txt, submit it in Google Search Console and Bing Webmaster Tools, and use IndexNow for fast change notifications. Google announced in June 2023 that its sitemap ping endpoint would stop working, and pings to it now return a 404, so old plugins that "ping Google" do nothing.

    # robots.txt
    Sitemap: https://www.example.com/sitemap_index.xml
    

    The Sitemap: line is independent of user-agent groups, so any crawler that reads your robots.txt can find it, including AI crawlers. For crawler rules themselves, see AI crawlers and robots.txt.

    IndexNow lets you tell participating search engines, including Bing, about changed URLs the moment you publish. A minimal request looks like this:

    curl -X POST "https://api.indexnow.org/indexnow" \
      -H "Content-Type: application/json; charset=utf-8" \
      -d '{
        "host": "www.example.com",
        "key": "your-indexnow-key",
        "keyLocation": "https://www.example.com/your-indexnow-key.txt",
        "urlList": [
          "https://www.example.com/pricing/",
          "https://www.example.com/compare/example-vs-competitor/"
        ]
      }'
    

    Set it up once in your CMS or deploy pipeline so it fires only for URLs whose content actually changed. More on this in Bing Webmaster Tools and IndexNow.

    Common sitemap mistakes that hurt AI freshness

    Most sitemap problems come from automation that nobody has looked at since launch. Check for these:

    • Every URL shows the same lastmod (usually the last deploy time)
    • lastmod dates in the future, or a date format engines can't parse
    • Redirecting, 404 or noindex URLs listed
    • HTTP or non-canonical host variants mixed in
    • Important pages missing because the CMS only lists blog posts
    • Sitemap blocked by a CDN or bot rule, so crawlers get a challenge page instead of XML
    • A staging sitemap pointing at a staging domain left live after a launch

    Example: a 15-minute sitemap audit

    This example uses an illustrative site to show the process; the numbers are made up.

    Step 1: check the lastmod distribution. Pull the sitemap and count dates:

    curl -s https://www.example.com/sitemaps/pages.xml \
      | grep -o '<lastmod>[^<]*' | cut -c10-19 | sort | uniq -c | sort -rn | head
    

    If 1,180 of 1,200 URLs share yesterday's date, lastmod is being set by the build, not by content changes. That's the first fix.

    Step 2: check status codes for a sample.

    curl -s https://www.example.com/sitemaps/pages.xml \
      | grep -o '<loc>[^<]*' | cut -c6- | head -50 \
      | xargs -I{} curl -s -o /dev/null -w "%{http_code} {}\n" {}
    

    Anything other than 200 should come out of the sitemap or be fixed.

    Step 3: compare against the pages that matter. Take the 20 pages you most want cited (pricing, comparisons, top docs) and confirm each is listed with a believable date. Missing high-value pages are common when the sitemap is generated from a single content type.

    Step 4: confirm the sitemap is reachable by bots, not just by your browser, by checking CDN security events or server logs for crawler requests to the sitemap URL.

    If you're about to change URLs, sitemaps are also a core part of site migrations without losing AI visibility. For the rest of the technical checks, use the technical AEO audit checklist.

    The bottom line

    A sitemap is a small file with an outsized effect on how current AI answers about you can be. List only canonical, indexable URLs; set lastmod from real content changes and nothing else; drop changefreq and priority; split large sites into typed sitemaps under an index; and notify search engines through robots.txt, Search Console, Bing Webmaster Tools and IndexNow. Honest dates earn trust from the crawlers that feed AI search, and that trust is what gets your updates picked up quickly.

    FAQ

    Frequently asked questions

    Do AI crawlers like GPTBot or PerplexityBot use XML sitemaps?

    AI companies publish little about whether their crawlers read sitemaps. What is documented is that Google and Bing use them, and those indexes ground AI Overviews, AI Mode, Copilot and parts of ChatGPT search, so sitemaps help AI visibility indirectly.

    Should I update lastmod every time I republish a page?

    Only when the main content changes in a meaningful way. Changing lastmod for a new footer, a copyright year or a sitemap rebuild teaches search engines to ignore your dates.

    Does Google still use changefreq and priority?

    No. Google has said it ignores both, and Bing said the same in its July 2025 sitemap guidance. Leaving them out is fine.

    How many URLs can one sitemap hold?

    The sitemaps.org protocol allows up to 50,000 URLs and 50MB uncompressed per sitemap file. Larger sites use a sitemap index that points to several sitemaps.

    Do I still need to ping Google when my sitemap changes?

    No. Google deprecated the sitemap ping endpoint in June 2023, and pings now return a 404. Reference the sitemap in robots.txt and submit it in Search Console instead.

    • XML sitemaps
    • lastmod
    • crawling
    • IndexNow
    • technical SEO
    Saarthak Srivastav

    Written by

    Saarthak Srivastav

    AEO & GEO, AEO-Rank

    Saarthak Srivastav writes and reviews articles on the AEO-Rank blog, drawing on what the AEO-Rank team sees tracking AI answers across ChatGPT, Gemini, Claude, Perplexity and Grok.