Use Sitemap

An XML Sitemap is a file that you create in order to help search engines find pages on your site. Sitemaps are not mandatory, but they are recommended by the major search engines. For more information on Sitemaps, please see: Learn about sitemaps , Build and submit a sitemap, Sitemaps.org

If you have an XML Sitemap on your site and you wish for Siteliner to use it when crawling your site, please click the checkbox "Use Sitemap" under "More options." Siteliner will then locate your Sitemap and add any pages listed in your Sitemap to the pages it finds on its own when crawling.

In order to locate your Sitemap, Siteliner will look in the robots.txt file on your site for a line of the form:

sitemap: http://www.mysite.com/sitemap.xml

Note that you must provide the complete URL of the Sitemap in your robots.txt file. If you do not have a robots.txt file, or your robots.txt file does not contain a line identifying the location of your Sitemap, Siteliner will by default look for a file named "sitemap.xml" in your website's home directory.