User and Google XML Sitemaps
Most people know that it is important to have a sitemap for large sites, but it is also important to ensure that an XML file exists with a listing of all files and associated update frequencies. Having a Google XML Sitemap allows you to better track errors and enables you to see search terms that people use to reach your site. When used in conjection with Google Webmasters, you will have more control over your site's performance. You need both a sitemap for your users and for Google under most circumstances. As sites grow larger and larger, this is especially important. XML sitemaps tell Google about the frequency of updates, as well as the priority of each update, for each respective file. Over time, this can be changed -- as specified in the XML file.
It is in the best interest of every listed site to help Google list each page of a website properly. When one does a "../site: domain" search, are the listed results those that were intended in the first place. Unwanted page listings can be removed, but something like this takes time. User sitemaps also must be maintained properly, as sometimes the links in them can go out of date. The World Wide Web is constantly changing -- and so must each site that is part of it. Keeping a site updated takes time and effort and each sitemap should reflect this. As well, large sites should have a robot.txt file that controls access to a site (can disallow access to a folder and its content). Most of the time, one needs at least 3 things on a website: two sitemaps and a robots.txt file.
