Search engines are lazy. The more that you study SEM and SEO, the more you realize that the industry’s job is to make their jobs as easy, accurate, and efficient as possible. After all, the were several billion new webpages (not websites) each day in 2008, according to Google. One can only imagine how many pages the search engine’s poor spiders need to crawl (and possibly index) now. (I wonder if the overworked bots should form a union!)
Here’s a human-based metaphor. Say that you need to get to a book (perhaps on marketing) in your house. Would you want the book to rest on a table in the next room or be stuffed inside a box in a pile of boxes somewhere in the cellar on the other side of the house? The former, of course. Well, search engines act the same way: They want to index anything and everything as quickly as possible before moving onto the next website.
This is one reason why webmasters need to ensure that their sites do not have broken links, poor spelling, duplicate content, and other things that tell Google’s spiders that a website is lacking in quality. When the electronic creatures see these things, they essentially say, “Screw this! I’m not going to waste my time on a crap website.” And then they leave without indexing parts — or even all — of your website.
There is another important principle to remember, and it relates to URL structure and website hierarchy as well. Once you are familiar with the technical aspects of URLs, you can then understand how they relate to SEO. First, it is important to know that websites need to be large in terms of width rather than height. Google’s spiders only go so deep — if you have a page that is, say, nine levels down from the home page, it will be indexed only rarely if at all.
The general rule is to have a website whose pages are never more than two or three removed from the index. (See a blog post I wrote for my company on the topic.) At this site, you will see that I have many pages that are one level from the home page — SEO-Software Reviews, Social-Media Software, and so on — and each of those pages has at most one layer underneath.
URLs, in general, should reflect the website hierarchy. Here is an example from my site here:
There is nothing else in the URL. Nothing. In my time offering SEO-consultant services, I have seen some horrible URLs. Two webmasters at two companies, for some reason, had put “home” or “index” between the root domain and the page in question. Something like:
Why is the “home” a bad thing? Well, every “/” after the root domain tells Google that it is entering a new layer of the website. This construction tells the search engine, whether it is accurate or not, that the “About Widgets” page is two layers from the home page rather than one. And “/home/” is not even a page unto itself, so it reflects a faulty website hierarchy and URL structure that raises a question in Google’s eyes. A better URL (without worrying about keywords right now) would be:
See the difference? And this is something basic. It’s just one reason of many that you need to keep SEO in mind (an article by my company) long before you even start to build or revamp a website. (Not that I’m biased.)
How to Optimize Blog URLs for SEO
As I wrote in a prior SEO guide to over-optimization, I often use this blog as a testing ground for my online marketing ideas. (That way, I do not have to worry about doing something possibility negative at my day job for our or our clients’ websites! But that’s neither here nor there.) Recently, I addressed an important aspect of website optimization that I had neglected when starting My SEO Software last year: optimizing the blog-post URLs for SEO. Since I created and now manage this site in WordPress, I went to the Permalink Settings in my Dashboard:
Originally, I had chosen the “Day and name” option, which puts the year, month, and day in the URL between the root domain and the page name. Which, of course, was bad — I cannot think of a possible context in which such an option would be preferable. As a result, the URLs of blog posts looked like:
Now, can you see why that is bad? There are many reasons to take out the junk in the middle, and those are addressed in the “for further information” links at the end. With the new setting, the URL is:
However, then I had fix all of the old posts and the links (external and internal) to them. I exported my initial XML sitemap (through a WordPress XML Sitemap plug-in) and then exported it into an Excel spreadsheet (see the left column). Then, I rewrote all the blog-post URLs (see the right column):
Next, I used the Redirection WordPress plug-in to create 301 redirects — the type of redirect that, as I am sure you know, passes PageRank or so-called “Link Juice” — from the old URLs to the new ones. It was just a two-copies-and-pastes job per URL. (If you have an extremely large website, you might want to put a cup of coffee on.) Then, I rebuilt the XML sitemap manually and resubmitted it in Google Webmaster Tools for a quick re-indexing of the new URLs. Ta-da!
Still, there are two things to keep in mind: duplicate content and keyword cannibalization as well as social-media sharing. Your blog-post URLs will no longer include specific dates and will instead only be identified by the page name (ideally, the targeted keyword). So if you write two posts about the same topic, you need to make sure that you use a different keyword each time. In addition, after I had changed the URLs, all of the buttons at the bottom listing the number of Facebook likes, Twitter tweets, and so one were reset to zero — after all, no one had shared anything at that (new) URL yet. But personally, I think it is worth it in the long run. If you write quality content, people will re-like and re-tweet again soon enough.
For more information on how to optimize URLs for SEO, I highly recommend these posts by Rand Fishkin, Ann Smarty, Garth O’Brien, and, of course, Matt Cutts. I have also written a prior search-engine-optimization URL post as well.
Samuel J. Scott, a former journalist in Boston turned Internet marketer in Israel, is the founder and publisher of My SEO Software and Director of Digital Marketing and Communications and SEO Team Leader at The Cline Group. You can follow him at Google+, LinkedIn, Facebook, and Twitter. His views here and elsewhere do not necessarily reflect those of his company and clients.