OK, so most business owners these days have a website. Some of them are spectacular pieces of web wizardry and others are a simple one-page affair. The wizzbangery of your website doesn't have to affect its effectiveness.

The simple process of understanding how people use the web can be an effective insight into making your website an automated marketing tool, converting your customers and raising your bottom line.

Today’s article will focus on giving you an understanding of how search engines work so you can work towards the process of generating traffic to your website through search engines.

Millions of people use search engines every day to search for goods, services, answers and information. If your website isn’t built in a way that makes it easy for search engines to read, and then rank, you're missing out on a lot of business. Even worse, if your competitors have built their websites properly, they will be benefiting from the traffic you could have been funneling to your website.

How search engines work

If we break a search engine down to its simplest parts there are essentially three components to a search engine:-

• the crawler or spider
• the index
• the algorithm

The crawler

Sometimes called a spider, bot or crawler, this is a program that 'reads' your website and takes a copy of it to store in the index. A crawler will enter your site either through a link from another website or from a submission request at a search engine's "submit your website" page.

The crawler will then read all of the content on the site (including items such as the title, description, alt and keywords tag), follow all of your menu item links to internal pages, read those and take a copy of them.

This copy is then stored in the search engine's index.

The index

The index is like the filing cabinet of a search engine. All the pages that are crawled by the search engine spiders are stored in the index. Remember, the index has millions and millions (if not billions) of pages of information in it and it’s growing at a rapid rate every day. This is why it's so easy for your website to get lost in the plethora of information available on the net.

Making sure your website can get into the index is an incredibly important part of a search engine optimisation strategy. The easiest and most simple way to find out if your website has been crawled and indexed is to use the advanced operator of site: (the word site then colon).

So, to find out if your website is in a search engine’s index type the following into a search engine’s search box:-

site:www.example.com

This will tell you how many pages are in the index of a search engine. If your website has 50 pages and the site:operator tells you that only 8 of them are indexed, then you have a problem. That’s 42 other pages that people could be finding but aren’t because they’re not indexed.

Note – search engines have some really handy site operators that tell you more information about a page or keyword term. For example, if you type define:laptop into a search engine, it will give you back a definition of what the word laptop means.
You can find a list of these operatorsin Google by clicking the links below:-

Google - http://www.google.com/help/operators.html

The algorithm

This is the secret sauce; the magic in the machine. You will never figure out how the algorithm works, however, you can manipulate algorithm your website to rank above others. This takes time, testing and skills, but it’s not as difficult as you may think.

When someone types a search term or search phrase into a search engine, the search engine software runs through the entire index and the algorithm decides which website to rank #1, 2, 3, 4 etc. in its search engines results page (SERP).

How to get your website into the index

There are many different ways to get your website into a search engine’s index. The easiest way is to get links to your website from other websites. Here’s a few ideas on how to do this:-

Getting links to your website

1. List your business with directories.
2. Write articles for other websites on the proviso that you can have a link back to your website.
3. Create a blog in conjunction with your website and use both to interlink your website.
4. Create useful, unique content that will naturally get links to it (as long as you market it). Top ten lists are great and editorial content with loads of helpful information are always a great source of natural links.
5. Approach websites that are in a complementary industry to exchange links. (If you sell lawnmowers, then a gardening website might be a good place to start.)

Use the search engine’s submission page

Most search engines have a submission page that allows you to add your website. Some of them are accessible to the public, others you will be required to create an account for.

Here are the three top engines and links to their submission pages:-

http://www.google.com/addurl/
http://search.yahoo.com/info/submit.html
http://search.msn.com.sg/docs/submit.aspx

Create an XML sitemap and feed this into the search engines

This is a more advanced technique for getting content into a search engine’s index – it’s definitely the most thorough and quickest way. If you wanted to find out more about this process, you can visit my website for a detailed article on how to set up an xml sitemap.

This should give you a basic understanding of how a search engine works.
In my next post I will focus on how to ensure your website is crawlable by search engine spiders.