You hear it everywhere on the Internet: content is King! Nobody will dispute that high quality, unique and relevant content is one of the key elements to attracting the Search Engines' attention and getting your site's pages indexed and ranked.

But the content argument does overlook something of pivotal importance: if you target the wrong keywords and keyphrases your pages may either never get ranked at all or remain burrowed so deep down in the result pages that they will never get seen by human eyes.

Keyphrase research should precede anything else you do, because targeting the wrong search terms (or not targeting any at all) is probably the most commonly made mistake when setting up a site. We would certainly advise to thoroughly research the keyword space, before doing anything else because the selected keyphrases will have a bearing on almost all other aspects of the site development process, from actual content to meta tags, text anchors, page titles and more.

Before we start it's important to understand that a Web site doesn't have a keyword space itself: only its pages do. In plain English, each page will have a few suitable, optimal keyphrases (usually one or two) specific to that page. There are no keywords that are relevant to the "site", only to its pages.

The most frequently made mistake by Webmasters is being lured into choosing search terms that receive very high traffic. But the keywords that are used the most by searchers are often not very relevant or specific and they don't really tell what the searcher is actually looking for. Many searchers will abandon such "broad" queries for more narrow ones that yield fewer but more relevant results.

High traffic, broad keyphrases also return lots and lots of pages (often many millions) and getting good rankings with such amounts of competing pages is very difficult. No matter how much traffic the search term receives, if your page cannot be found somewhere on the first three results pages then it is unlikely to receive significant amounts of traffic.

Keyphrase research should unearth keyphrases which meet four criteria that can be grouped into two categories:

Relevancy and specificity

1. Relevancy: if your site sells books for children, then the terms "books" and "children" may appear relevant but really these are very broad terms to avoid like the plague. "Children's books" however is more relevant and could be used as a "seed" to find other, related and equally relevant keyphrases.
2. Specificity: adding so-called qualifiers to your search terms can greatly improve conversion rates. Qualifiers tell searchers what your site does: words like "free", "buy", "sell" are often used to make a relevant search term more specific. A visitor that finds your site using "buy children's books" is much more likely to actually make a purchase. If your site is location specific, then a geographical qualifier will prevent your site receiving traffic from parts of the world it cannot service. Geo-targeted terms are also easier to optimise for and you will get higher rankings on such terms.

Traffic generation

1. Search terms must receive "a good amount" of traffic. In other words, Search Engines must receive a sufficient amount of daily queries (the so called 24 hour count). The amount you should expect naturally depends on how popular or how niche the theme of your page actually is, so there is no hard and fast rule.
2. Competing with fewer pages for the same traffic rather than with more pages is, relatively speaking at least, easier. So, an ideal keyword would meet all of the above criteria and return as few results as possible.

The traffic generation criteria are the subject of our predicting Website traffic page, so we won't elaborate on this any further on this page.

So, in practice, how do you go about selecting suitable search terms? Here's a step by step guide. You need to do this page by page for every page you want to receive Search Engine traffic from. That's probably every content page and the home page, but not pages like "about us, "contact us", "site map" and similar, for which page rankings aren't important.

1. Decide what will be the theme of the page in question will be. That sounds pretty obvious perhaps but many Webmasters fail to endow the different sections of their site with a clear, unambiguous theme.
2. Brainstorm, preferably with someone who's interested in the theme of the site but isn't as "close" to it as you are, to generate a list of seed terms. It helps to try and think "outside the box" (hence your "assistant": he'll play the role of a searcher). The seed terms are intuitive and few will make it to the finishing line, as you will gradually replace them with more relevant terms. Avoid most one word search terms, concentrating on two, three and even four word terms. Note that the most frequently used words in any language are so called "skip words" and that most Search Engines don't take them into account. Google e.g. returns the same results for "books for children" as it does for "books children". Plural versus singular can return different results and different amounts of results, so you need to play around with this. Misspellings need also be taken into account. Some common "misspellings" arise from the inherent difference between US English and UK English. For instance, "search engine optimization" and "search engine optimisation" yield different and different numbers of results in Google.
3. Include brand names, if applicable. A site about earth moving equipment could use "caterpillar" as a search term because some searchers will use it in so called "targeted searches".
4. Next you need to find related terms, many of which will be more suitable than the seeds. Without help this is an impossible task to complete. Instead use a keyword suggestion tool. There are several available, including Overture's keywords suggestion tool and Google Adwords has one too. Both are free but fairly limited. Instead, use Wordtracker (see Toolbox, left), which allows you to tap into a keyword database of 360 million terms, including data on daily traffic per term and number of competing pages per term. You'll need these data to complete the analysis and make the final selection. Using the seed terms and Wordtracker's data, identify a large group (at least fifty, perhaps less if the theme of the page is very niche) and export the data into a spreadsheet, including 24 hour count and number of competing pages.
5. Using the analysis presented on the our "traffic prediction" page, select search terms with a good balance between 24 hour count and number of competing pages. The choice isn't easy and only you can make it. It's about deciding how many competing pages your page will be able to "jump". If you page has reasonable PageRank (4 or more) it should be able to get good rankings in a competitive field of results, provided content is strong and well optimised for that term. If you're set on certain very high volume terms, perhaps because you feel they are really very relevant to your page, and you haven't used qualifiers yet, then now is the time. For example "buy children's books (in) New York" returns about five times fewer results than "children's books" and is much more specific (but only if you operate in New York!). Go back to Wordtracker to find the 24 hour count and number of competing pages to check whether this search term has the right balance for you.