why listen to us about web development?

One element we focused on during the development of sho’fr was ensuring all our Company Profile listings would be optimized for inclusion in the major search engines. While optimizing this is certainly an on going process, we’ve done fairly well thus far.

As of today, we have 200+ crawled pages in Google and 216 in Yahoo. Not bad, consider we have 195 company profiles and 17 city pages. So now we wanted to share some tips for our web developer viewers at home.

okay, so where do we start?

We will first assume your content is solid, this includes meta description tags, appropriate page titles, and of course, no cheating. Incoming links to your site are the web developer’s equivalent of striking oil in your backyard. So leave comments on popular sites with a link back to your site, put your link in your signature in every forum you post on, include it at the bottom of all your outgoing emails, do everything up to (and I’m not ruling this out) tattooing it on your forehead.

But while you wait for your links to propagate the search engines, you can provide a complete list of your great content in the form of a sitemap. Specifically, a sitemap.xml file that you create, store on your public web server, and share with search engines. Unlike the popular “Sitemap” link in the footer of 95% of the websites out there, a sitemap.xml file is written specifically for search engine spiders, and thus is not pretty and is not user friendly. If you’d like to jump the gun a bit and see a live, populated sitemap, check out our Corvette sho’fr sitemap and select “View Source” in your browser.

let’s make some XML!

In a future article, I’ll detail how we built the dynamic, daily updating, sho’fr sitemap using a .htaccess trick, some PHP and cron jobs. For now, we’ll work on the manually method.

Don’t be fooled by services or downloadable applications that want to charge you in exchange for creating your sitemap. This is an easy process that can be done in any text editor or with free services online.

Let’s take a look at a simple properly formatted sitemap file, with only one URL included:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">

	<url>
		<loc>http://corvette.shofr.com/atlanta/</loc>
		<lastmod>2007-10-28</lastmod>
		<changefreq>weekly</changefreq>
		<priority>0.9</priority>
	</url>

</urlset>

Feel free to paste this into you favorite text editor and edit as we break this down line by line, so you know what you are looking at.

The first two lines are required to identify your XML file and reference the current XML protocol standard. The <urlset> tag may look different depending on your schema, but for beginners, just copy past that line from us.

The <url> and </url> tags surround each instances of a page on your site. You will be creating a new entry (and only one entry) for each page, and you will always open and close it with this tag.

The next one should be obvious. <loc> is the exact URL of the page we are talking about. Be certain you include the “http://” and while you are here, you should take a stance on the www vs no www front. Either way, stay consistent. Additionally, your sitemap will be less valuable in the long run if you direct search engines to a page that 1) doesn’t really exist or 2) gets redirected elsewhere.

The next 3 lines all reference back to the URL you just defined above.

<lastmod>2007-10-28</lastmod>
When was the last time the page was updated? It’s best to simply tag the page with a date, although time stamps do exist (2007-10-28T14:12:14+00:00) but are only necessary for very dynamic content.

<changefreq>weekly</changefreq>
How often do you expect the content on this page to change? Be honest with yourself about what parts of your site are static (your contact page) versus the pages that change often (an inventory of in stock parts). Valid options here are: “always”, “hourly”, “daily”, “weekly”, “monthly”, “yearly”, and “never”. The option “always” should be used for a page that changes every time it is loaded, while “never” should be used for archived pages.

<priority>0.9</priority>
Priority is the biggest hurdle for most sitemap authors. Yes, we all believe our content is important, however the important thing to remember here is the value is relative to the values you set for the rest of your site. 99% of the time, everyone should set their main page to 1.0. From there, you need to build a hierarchy in your head as to which pages are most important to be included in search engines. While your company history or mission statements are important to your business, they are not what your customer’s will be searching Google for. They want quick access to the bread and butter.

Use the 0.0 - 1.0 scale on your terms. For example, here at sho’fr our front page is 1.0, our city pages are 0.5, and our company profiles are 0.9. Why? Because users landing on a city page are still a click away from the content they need. This isn’t to say our city pages aren’t important. Our static content (”About sho’fr” for example) is less critical and gets 0.4. Einstein said it best: it’s all relative.

Close your </url> and continue to add additional pages. When you are done with your site, use the final close tag </urlset > and your sitemap.xml file is done!

everyone knows it’s nice to share

Great, you have a sitemap.xml file. We need to make sure everyone knows where to find it. The one blanket method to do this is via your robots.txt file. Simple add one line:

Sitemap: http://corvette.shofr.com/sitemap.xml

You can also take a more direct approach and alert each major search engine. Google and Yahoo have control panels that give you feedback on the status of your sitemap while Ask and MSN require you to submit your siteap via a specially constructed URL:

For ASK: http://submissions.ask.com/ping?sitemap=YOUR_SITEMAP.XML_HERE
For MSN: http://api.moreover.com/ping?u=YOUR_SITEMAP.XML_HERE

The most powerful tool here is the Google Webmaster Tools page. Creating a Google Account just for this tool is highly recommended.

learn more

As always, the internet gives us access to virtually every piece of information we could need. And while I hope this article has given you a quick insight into sitemaps, there are dozens of additional resources that are worth browsing (some that read like stereo instructions). A great resource is the document prepared by Google, Inc., Yahoo, Inc., and Microsoft Corporation over at www.sitemaps.org. Of course, feel free to ask questions in the comments, we will do our best to answer them (possible in a future article).