A Guide to Writing a Sitemap (sitemap.xml)

1. What is sitemap.xml?

sitemap.xml is an XML-format map file of a site’s pages, used to pass the site’s page structure, update frequency, priority and other information to search engines (such as Google and Bing), helping crawlers discover and crawl pages efficiently. It is a key part of basic SEO. Sources: the official sitemaps.org protocol and the Google Search Central guide.

2. Why is sitemap.xml needed?

1. Improves indexing efficiency: it explicitly tells search engines “which pages are worth crawling”, so crawlers do not miss deep pages or dynamic pages (such as paginated pages and category pages);
2. Conveys content priority: page importance is marked with <priority> (for example, home page = 1.0, blog list = 0.8);
3. Supports rich media: it can embed images (<image:image>), videos (<video:video>) and other extended information;
4. Handles multiple languages and devices: pages for different languages or mobile devices are handled by splitting the sitemap or using multiple namespaces.

3. Basic Syntax: Core Structure and Elements

1. Basic template

XML syntax must be followed strictly. The required elements are <urlset> (root node) and <loc> (page URL). Optional elements include <lastmod> (last modified time), <changefreq> (update frequency) and <priority> (priority).

<?xml version="1.0" encoding="UTF-8"?>
<!-- Root node: declares namespaces (base protocol + extensions) -->
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
        xmlns:xhtml="http://www.w3.org/1999/xhtml"  <!-- multiple languages / rich media extension -->
        xmlns:image="http://www.google.com/schemas/sitemap-image/1.1"  <!-- image extension -->
        xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">  <!-- video extension -->

  <!-- single page entry -->
  <url>
    <!-- required: absolute page URL (use https, avoid relative paths) -->
    <loc>https://example.com/</loc>
    
    <!-- optional: last modified time (ISO 8601 format, e.g. YYYY-MM-DD or YYYY-MM-DDThh:mm:ss+08:00) -->
    <lastmod>2024-05-20</lastmod>
    
    <!-- optional: update frequency (a hint for crawlers, not enforced) -->
    <changefreq>daily</changefreq>
    
    <!-- optional: priority (0.0-1.0; 1.0 recommended for the home page, 0.1 is the minimum) -->
    <priority>1.0</priority>
    
    <!-- extension: multiple language handling (xhtml namespace) -->
    <xhtml:link rel="alternate" hreflang="zh-CN" href="https://example.com/" />
    <xhtml:link rel="alternate" hreflang="en" href="https://example.com/en/" />
    
    <!-- extension: image information (image namespace) -->
    <image:image>
      <image:loc>https://example.com/images/logo.jpg</image:loc>  <!-- image URL -->
      <image:title>Site Logo</image:title>  <!-- image title (optional) -->
    </image:image>
  </url>

  <!-- more page entries... -->
</urlset>

2. Key element reference

Element	Required	Description
<loc>	Yes	The page’s absolute URL (must include the protocol https://; relative paths such as /about are not allowed)
<lastmod>	No	Last modification time of the page (e.g. 2024-05-20T14:30:00+08:00; second-level precision helps crawlers judge)
<changefreq>	No	Update frequency for crawlers to use as a reference (possible values: always/daily/weekly/monthly/yearly/never)
<priority>	No	The page’s importance relative to the site (home page = 1.0, core category pages = 0.8, ordinary detail pages = 0.6, minimum 0.1)

4. Advanced Techniques: Handling Complex Cases

1. Splitting a large site: using a sitemap index

When the number of URLs exceeds 50,000 or the file size exceeds 50MB​, a sitemap index file (with an .xml suffix, usually named sitemap_index.xml) must be used to split the sitemap into several child sitemaps:

<?xml version="1.0" encoding="UTF-8"?>
<!-- index file root node -->
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">

  <!-- child sitemap 1: home page and core pages -->
  <sitemap>
    <loc>https://example.com/sitemap-home.xml</loc>  <!-- child sitemap URL -->
    <lastmod>2024-05-20</lastmod>  <!-- child sitemap last updated time -->
  </sitemap>

  <!-- child sitemap 2: blog posts -->
  <sitemap>
    <loc>https://example.com/sitemap-blog.xml</loc>
    <lastmod>2024-05-19</lastmod>
  </sitemap>

  <!-- more child sitemaps... -->
</sitemapindex>

2. Excluding invalid pages

Never include the following pages, or crawlers may trust the site less:

• Login and registration pages, search result pages, and 404 error pages;
• Duplicate content pages (such as print versions and aggregation pages with no substantive content);
• Pages blocked from crawling by robots.txt (such as Disallow: /admin/).

3. Handling dynamic content

• Paginated pages: such as https://example.com/blog?page=2. Make sure the URL carries no session ID and that parameters are stable (avoid ?page=2&sid=xxx);
• Category and tag pages: include them only when the content has standalone value (for example, aggregating high-quality posts); empty pages or duplicate content pages must be excluded.

5. Validation and Submission

1. Syntax validation

After generating the file, check that the XML format is correct. Recommended tools:

• XML Validator (online validation that points out where syntax errors are);
• W3C XML Validator (authoritative validation with namespace checking).

2. Submitting to search engines

• Google: submit through Search Console → “Sitemaps” → enter the sitemap.xml URL;
• Bing: upload through Bing Webmaster Tools → “Submit sitemap”;
• General method: add Sitemap: https://example.com/sitemap.xml to robots.txt (this lets crawlers discover it on their own).

3. Monitoring and maintenance

• Check “Sitemap status” in Search Console / Bing Webmaster Tools regularly, and watch for errors such as “Couldn’t fetch” and “Timeout”;
• Update the sitemap promptly after content is added or removed (tools such as Yoast SEO can trigger updates automatically).

6. Recommended Tools (no manual writing)

1. Generators

• Online tools: XML-Sitemaps.com (enter a domain and it crawls every URL on the site to generate a sitemap);
• Desktop tools: Screaming Frog SEO Spider (crawls a site and exports a sitemap, with filtering for invalid pages).

2. CMS plugins (one-click setup)

• WordPress: Yoast SEO (generates and updates the sitemap automatically, with image and video extensions);
• Shopify: SEO Manager (built-in sitemap support that excludes invalid pages automatically);
• Hexo / Hugo: themes usually ship with a sitemap generator plugin (for example Hexo’s hexo-generator-sitemap).

7. Things to Watch For

1. URL rules: every <loc> must be an absolute URL (such as https://example.com/page); relative paths (such as /page) and temporary URLs with parameters are not allowed;
2. Consistent encoding: the file must be encoded as UTF-8 (so non-ASCII text does not turn into garbage);
3. Avoid over-optimization: do not set every <priority> to 1.0, and match <changefreq> to the actual update frequency (for example, use yearly for static pages rather than daily);
4. Update regularly: check the sitemap at least once a month and make sure it matches the site’s actual pages.

Summary: the core of sitemap.xml is “accurately conveying information about valid pages”. A standard format, sensible extensions and tooling can greatly improve how efficiently search engines crawl a site and lay the groundwork for SEO rankings.