Thursday, September 18, 2008



The spider’s movement across web pages stores those pages in its memory, but the key

action is in indexing. The index is a huge database containing all the information brought

back by the spider. The index is constantly being updated as the spider collects more

information. The entire page is not indexed and the searching and page-ranking algorithm

is applied only to the index that has been created. Most search engines claim that they

index the full visible body text of a page. In a subsequent section, we explain the key

considerations to ensure that indexing of your web pages improves relevance during

search. The combined understanding of the indexing and the page-ranking process will

lead to developing the right strategies. The Meta tags ‘Description’ and ‘Keywords’ have

a vital role as they are indexed in a specific way. Some of the top search engines do not

index the keywords that they consider spam. They will also not index certain ‘stop

words’ (commonly used words such as ‘a’ or ‘the’ or ‘of’) so as to save space or speed up

the process. Images are obviously not indexed, but image descriptions or Alt text or “text

within comments” is included in the index by some search engines.