Daily ToolsSEO ToolsAI Tools

XML Sitemap Guide: Get Every Page Discovered, Crawled and Indexed

You published the page. You did the keyword research, wrote 2,000 good words, polished the meta tags. Two weeks later, it is nowhere in Google — not ranked badly, just absent. Search Console shows the dreaded status: “Discovered — currently not indexed”, or worse, Google has not found it at all.

An XML sitemap is the cheapest insurance against that limbo. It is a machine-readable list of every URL you want indexed, handed directly to search engines. This guide explains what sitemaps do and do not fix, the format rules that matter (and the two tags Google ignores), the best generator tools, and the submission routine that gets new pages found in hours instead of weeks.

What an XML sitemap actually does

Googlebot discovers pages mainly by following links. That works — eventually — but it fails predictably: new sites with few backlinks, new pages not yet linked from anywhere prominent, deep pages three-plus clicks from home, and big sites where crawl budget runs thin. A sitemap short-circuits all of that by declaring: here is every page I have, and when each last changed.

Each entry supports four tags:

  • <loc> — the full URL. The only required tag.
  • <lastmod> — when the page last changed. Google uses this to prioritise recrawling; keep it honest.
  • <changefreq> and <priority> — Google has publicly said it ignores both. Setting them is harmless tradition; skipping them is fine.

Limits: 50,000 URLs or 50MB per file — beyond that you split files and list them in a sitemap index. Blogs and small business sites will never notice.

Discovery is not indexing — why pages sit in limbo and how to fix it

What a sitemap will and will not fix

It fixes discovery: Google learns your URLs exist, fast. New posts, orphan pages, deep archives — all surfaced.

It does not force indexing. “Discovered — currently not indexed” means Google knows the URL and has chosen to wait, usually a quality or demand judgement. The cure there is better content, stronger internal links (see how our tool pages interlink), and site-level trust — not more sitemap submissions. Knowing which problem you have saves weeks of wrong debugging: not discovered → sitemap and links; discovered but not indexed → content and authority.

Sitemap hygiene rules

  • Only canonical, indexable, 200-status URLs. A sitemap full of redirects, 404s and noindexed pages teaches Google to distrust the whole file.
  • Absolute URLs with the right protocol — https, one hostname, consistent trailing slashes.
  • Escape special characters — ampersands in URLs must be &amp; in XML. (Generators handle this; hand-editing forgets it.)
  • Update lastmod when content genuinely changes — faking freshness site-wide gets the signal ignored.
  • Reference it in robots.txt with a Sitemap: line so every crawler finds it without being told — one click in our robots.txt generator.

The best sitemap tools compared

1. HN Solutions XML Sitemap Generator — best free manual generator

Our XML sitemap generator turns a pasted list of URLs into a valid sitemaps.org-compliant file instantly: it deduplicates, adds missing https://, skips and counts invalid lines, escapes characters correctly, and lets you toggle lastmod, changefreq and priority. Copy the XML or download sitemap.xml ready to upload. Free, no limits, runs entirely in your browser.

2. Rank Math / Yoast sitemaps — best for WordPress

Auto-generated and self-updating at /sitemap_index.xml — if you run WordPress, this is your primary sitemap. The manual generator complements it for landing pages outside the CMS, static microsites, or hand-picked submission lists.

3. Screaming Frog — best for large-site generation

Crawls your site and emits a sitemap of what it found. Powerful, desktop-based, free tier caps at 500 URLs.

4. XML-Sitemaps.com — the old classic

Crawls up to 500 pages free, server-side, with upsells. Fine occasionally; less transparent than pasting your own list.

5. Hand-writing the XML

Educational exactly once. The escaping rules are why generators exist.

Why choose our generator

  • Paste-and-go: your URL list becomes valid XML in one click — no crawler setup, no account.
  • Cleans as it builds: duplicates removed, protocols fixed, bad lines counted so you know what was skipped.
  • Standards-correct output including character escaping — the silent killer of hand-made sitemaps.
  • Private: your URL list (which can reveal unlaunched pages) never leaves the browser.
Submit to Search Console — the 60-second habit after every launch

How to create and submit a sitemap

  1. Open the XML sitemap generator.
  2. Paste your URLs, one per line — every page you want indexed, nothing you do not.
  3. Leave changefreq and priority unset (Google ignores them) and keep lastmod on.
  4. Generate, then download sitemap.xml and upload it to your site root.
  5. In Google Search Console → Sitemaps, enter the sitemap URL and submit. Do the same in Bing Webmaster Tools — Bing powers more assistants and AI search than people realise.
  6. Add the Sitemap: line to robots.txt, and for high-priority new pages, also use Search Console’s URL Inspection → Request Indexing.

Pro tips: after any site launch or migration, resubmit immediately — it is the fastest recovery lever you have; check the Sitemaps report monthly for “discovered vs indexed” gaps, which flag quality issues early; and keep one sitemap per logical section on bigger sites so problems localise.

Frequently asked questions

How fast does Google read a submitted sitemap?

Usually within hours; indexing decisions follow on their own schedule. Submission accelerates discovery, not judgement.

WordPress already makes a sitemap — do I need this tool?

Your plugin sitemap covers CMS content. The generator is for everything else: static pages, landing pages, subdomains, or a curated list you want to submit separately.

Should image and video URLs be included?

Standard page sitemaps list pages; media benefits from dedicated image/video sitemap extensions on media-heavy sites. Most sites never need them.

Does a sitemap improve rankings?

Indirectly: it cannot rank a page that was never indexed. Sitemaps fix the plumbing so your content quality can compete.

The bottom line

An XML sitemap is sixty seconds of setup that removes an entire failure mode from your SEO. Build one with the free sitemap generator, announce it in robots.txt, and make sure the pages it lists deserve the crawl — the density checker and meta tag generator in our free SEO tools hub handle that part.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top