Thursday, September 18, 2008

How do Search Engines Work?

How do Search Engines Work?

Most of the top-ranked search engines are crawler based search engines while some may

be based on human compiled directories. The people behind the search engines want the

same thing every webmaster wants - traffic to their site. Since their content is mainly

links to other sites, the thing for them to do is to make their search engine bring up the

most relevant sites to the search query, and to display the best of these results first.

In order to accomplish this, they use a complex set of rules called algorithms. When a

search query is submitted at a search engine, sites are determined to be relevant or not

relevant to the search query according to these algorithms, and then ranked in the order it

calculates from these algorithms to be the best matches first.

Search engines keep their algorithms secret and change them often in order to prevent

webmasters from manipulating their databases and dominating search results. They also

want to provide new sites at the top of the search results on a regular basis rather than

always having the same old sites show up month after month.

An important difference to realize is that search engines and directories are not the same.

Search engines use a spider to "crawl" the web and the web sites they find, as well as

submitted sites. As they crawl the web, they gather the information that is used by their

algorithms in order to rank your site.

Directories rely on submissions from webmasters, with live humans viewing your site to

determine if it will be accepted. If accepted, directories often rank sites in alphanumeric

order, with paid listings sometimes on top. Some search engines also place paid listings at

the top, so it's not always possible to get a ranking in the top three or more places unless

you're willing to pay for it.

Let us now look at a more detailed explanation on how Search Engines work. Crawler

based search engines are primarily composed of three parts.