sitemap.xml & ror.xml
If no one knows about your site it’s fair to assume no one will find you let alone visit you. Search engines can drive lots of traffic to your site if they know about you and what you have to offer.
Google, Bing, Yahoo!, and other search engines use programs called spiders to walk through your site and index what they find. Don’t wait for search engines to stumble across your site or miss portions of what you have. Sitemaps are text files that direct search engine spiders to and through your web site.
There are two sitemap formats: ror.xml and sitemap.xml. I use both assuming that some spiders look for one not the other. (If you look at your web site’s log file you’ll probably discover that a number of different spiders and bots crawl your site, not just the big three search engines’. Feed them all with sitemaps.
Before generating a new sitemap I always use tools to search for broken links and errors first.
The easiest way to can create a sitemap is to use this website. It generates both ror.xml and sitemap.xml files as well as a list of all the URL’s and broken links it found. This is another site that makes sitemaps and this is a sitemap validator.
Tell spiders & bots you have an ror.xml file by linking to it in the <head> of your main index page:
<link rel="alternate" type="application/rss+xml" title="ROR" href="ror.xml" />
Link to your sitemap.xml file somewhere on your site like the footer of your main index.html page. You can also put your sitemap URL in your robots.txt file on the assumption that good spiders look for and acknowledge robots.txt:
Lastly force the search engines to actually crawl your site instead of cheating and using the Open Directory Project’s summary of your site. You have little or no control over the ODP entry so why let the search engines use it? Add this line to the <head> section of your main index page:
<meta name="robots" content="noodp">