Extending Search Engine Optimization using Sitemaps in Drupal 6

We have discussed some of the techniques earlier in Search Engine Optimization using Sitemaps in Drupal 6 article which mainly covered XML sitemaps.

The Google spider is quite smart, but then the possibility of them missing pages on your site is also quite high. You may have pages that are not in navigation anymore or you have moved a link to a piece of content so that it’s not easily accessible. A possibility is also that your site is very big for Google to just crawl it all without completely pulling all your server’s resources. The solution to this is sitemaps.

Google News XML Sitemap

Google has created one of the most popular news sources on the Internet just by collecting and organizing news articles from other sites. It’s called Google News and if you are running a news web site then you know how powerful it can be for picking up your comment. One front page article can generate 50,000 or more visitors in an hour or two. To be listed in Google News takes more than luck. You need to write great content and proactively seek to create timely and news-worthy content. If you’ve done that and you’re still not showing up in Google news then it’s time to create a Google News XML Sitemap. The Google News sitemap generator module was originally created by Adam Boyse at Webopius and is being maintained by Dave Reid. Thanks to both of you!

Setting up the Google News sitemap generator module

Carry out the following steps to set up the Google News sitemap generator module:

1. Download the Google News sitemap module from http://drupal.org/project/googlenews and install it just like a normal Drupal module.
2. Go to http://www.yourDrupalsite.com/admin/settings/googlenews or go to your admin screen, and click the Administer | Site Configuration | Google News sitemap feed link. You’ll see a screen similar to the following screenshot:

   

3. Select the content types that you wish to show up in the news feed. If all your story content types are newsworthy, pick Story. Your blog or page content types are probably not a good fit and selecting them may hurt the chances of your content being approved by Google.
4. Click on Save configuration.
5. Point your browser to http://www.yourDrupalsite.com/googlenews.xml and double check that you can see the sitemap, as shown in the following screenshot:

   

Submitting your Google News sitemap to Google News

Once you’ve assembled your new articles for a single publication label, submit them to Google News sitemaps by carrying out the following steps:

1. Check Google News to see if your site is already included. If not, you can request inclusion by visiting the following link, http://www.google.com/support/news_pub/bin/request.py?ctx=answer. The inclusion process may take up to a few weeks, and you’ll only be able to submit a News sitemap once this process is complete.
2. If your site is already showing up in Google News then proceed. If not, you should wait a couple of weeks and try again.
3. Log in to Google Webmaster Tools by pointing your browser to http://www.google.com/webmasters/tools/.
4. On the Webmaster Tools Dashboard, click on Add next to the site you want to submit.
5. From the Choose type drop-down menu, select News sitemap, and then type the sitemap URL, in this case http://www.yourDrupalsite.com/googlenews.xml.
6. In the list, select the publication label for the articles. You can select only one label for each sitemap.
7. Click on OK.

URL list

The XML Sitemap is the ideal choice because it allows you to specify a lot of information about the content of your site. But, say for some reason that you can’t install an XML Sitemap. Maybe there’s a conflict with another module that you just have to have. Perhaps your server doesn’t have the power to handle the large overhead that an XML sitemap needs for large sites. Or, possibly you want to submit a sitemap to a search engine that doesn’t support XML yet.

Well, there is an alternative. It’s not as robust but it is a functional, albeit rudimentary, solution. Just make a list of every URL in your site and put the list in one big text document with one URL on each line. Too much work, you say? Good thing there is a Drupal module that does all the work for you. It’s called the URL list module. It’s maintained by David K. Norman. Thank you, David!

Setting up a URL list sitemap

1. Download the Sitemap module from http://drupal.org/project/urllist and install it just like a normal Drupal module.
2. Go to http://www.yourDrupalsite.com/admin/settings/urllist or go to your admin screen and click the Administer | Site Configuration | URL list link. You’ll see the URL list screen, as shown in the following screenshot:

   

3. You can adjust the settings to keep track of who accessed the URL list to submit your site to Yahoo! and to help you authenticate with Yahoo! However, you can leave all these settings untouched for now.
4. Point your browser to http://www.yourDrupalsite.com/urllist.txt (http://www.yourDrupalsite.com/?q=urllist.txt if you don’t have clean URLs installed) and you’ll see your URL list sitemap, as shown in the following screenshot:

   

You can submit this sitemap to Google, Yahoo!, and many other search engines in lieu of an XML sitemap. Just follow the same steps as defined in the Submit your XML Sitemap to Google section above but use # http://www.yourDrupalsite.com/urllist.txt as the URL.

Remember to use http://www.yourDrupalsite.com/?q=urllist.txt if Google has problems with your URL.