All You Need To Know About Sitemaps
What is a sitemap?
A sitemap is simply a file or a list of pages on your website. Just like a map aiming to show users directions to a place, similarly a sitemap helps search engine crawlers to discover the pages on your website. A sitemap tells Google about the important pages on your website so that they can find it easily and get valuable information such as when was it last updated, how often it usually changes etc.
Why do you need a sitemap or do you actually need one?
Did you know crawlers discover pages on your website through sitemaps? This first step to having a crawler friendly website is to include a sitemap on your website. You might have read on the internet that sitemaps are not required if your site is small and has less than pages or if it is extensively linked internally. But it would never hurt to have a sitemap on your website regardless of its size.
Googlebot and other crawlers discover pages by following links from one page to another. If your pages aren’t linked to each other or from an external site, the crawlers might not be able to discover these pages. So it is ideal to have a sitemap on your website to avoid some key pages being undiscovered by bots. But do keep in mind having a sitemap doesn’t guarantee that all the URLs in it will be crawled and indexed.
Sitemaps are an essential part of any website and tell search engines about your site structure and hierarchy. As an SEO best practice, it is best to include two different types of sitemap.
Types of Sitemap
Basically there are two types of sitemaps namely XML sitemap and HTML sitemap. Let us dig deeper to understand what they are and the key differences and their purpose.
XML sitemaps are understood by and intended for the search engine bots & crawlers (like Googlebot) to use to locate and crawl through the pages of the website. They also provide vital information about the number of pages present on the website, frequency of update, when the page was last updated etc.
These are some factors that need consideration before creating a sitemap:
- Update your sitemap when new content or page is added on your website
- Ensure that all URLs / pages present in the sitemap are free of error pages (4xx error pages), redirect pages (3xx redirected URLs), canonicalised URLs and pages that are blocked from indexing (pages with ‘noindex’ meta directives)
See the screenshot below to see how an XML sitemap looks like:
HTML sitemaps are intended primarily for users to help them navigate through the website with ease. However, Google bots and crawlers can follow the links found on an HTML sitemap to discover pages too.
An HTML sitemap is a web page and usually contains links of pages arranged in a logical & hierarchical order. This gives an idea of the whole structure of the website to the user.
See the screenshot below to see how an HTML sitemap looks like:
Where can you find the Sitemap of your Website?
By default, your XML sitemaps are present in www.yourwebsite.com/sitemap.xml and one thing to keep in mind is that sometimes you may have one or more sitemaps. In such an event you need to add it to your robots file in the following format:
Since Google and Bing aren’t the only search engines out there, some other search engines find your sitemaps via your robots.txt file. It is also good to know that Google recommends keeping no more than 50,000 URLs in a single sitemap. If your website has more URLs than that, the best ideal way would be to split it into multiple, smaller sitemaps and link their URLs from the main sitemap.xml file.
How to create an XML Sitemap?
Creating an XML sitemap has never been easier. If you’re a business owner who has an online presence or if you’re a blogger and you observe that your pages are not indexed by Google, you can create an XML sitemap and submit it to Google Search Console (GSC).
If you’re using a WordPress CMS, you can easily set this up by adding a plug-in called Yoast SEO to your website. This will allow you to generate an XML sitemap which will automatically update itself as you add or remove new content and URLs to your site.
You can also create one by using free sitemap generator tools found online or use Screaming Frog (a website crawler tool which allows you to crawl 500 pages with the free version) and then submit to GSC.
How to submit an XML Sitemap to Google Search Console?
Once you have generated an XML sitemap file and uploaded to your root directory, you can submit it to the search console by following the steps below.
- Log in to the Google Search Console and add your property if you haven’t yet.
- Click on the “Sitemaps” section under the “Index” menu.
- You can submit the file by pasting the relative URL “sitemap.xml” and look for crawl errors and additional information about the coverage of your web pages on the SERPs.
It is important to note that even if you submit a sitemap, Google may or may not crawl all the URLs in it.
Once you are done with the submission in Google Search Console, it’s best to keep a close eye on it to make sure that no errors are reported from the list of URLs.
If you’ve read through this article, you would now have a fair idea of what sitemaps are and how to create them and add them to your website. Even though it falls under the technical aspect of SEO, it isn’t that as difficult to do as it used to be back in the days. Thanks to the sophisticated tools online, we can generate XML sitemaps with ease.
It is also important to note that having a sitemap on your website would add enormously to your site’s SEO performance if you don’t have this set up yet.
If you found this article to be helpful, leave us a comment and let us know.