Before discussing search engines as a marketing tool, it is important to understand what they are, how they work, how they have developed and how they will develop in the near future. Additionally, it is essential to know the players in the search arena and the connections between them, in order to be able to fully utilize search engine marketing.
A technical encyclopaedia, WhatIs.com, provides an accurate definition of a search engine.
“A search engine is a coordinated set of programs that includes:
In essence, the search engine bots crawl web pages and use links to help them navigate to other pages. The search engine then indexes those pages into its database. When a searcher sends a search query, the search engine compares the web pages in the index to find documents that are relevant to the search query. Based on its algorithm, the search engine returns results to the searcher in the search engine result page (SERP).
The search engine algorithm is a set of rules that a search engine follows, in order to return the most relevant results. Search engines fail to return relevant results sometimes, and that is why they need to improve their algorithm constantly. The algorithms determine the placement of web documents in the organic or natural search results, which are typically displayed on the left side of the screen in the SERPs, as illustrated in the figure 1.1.
Search engine algorithms are very closely kept industry secrets, because of the fierce competition in the field. Another reason for search engines to keep their algorithms private is search engine spam. If webmasters knew the exact algorithm of a search engine, they could manipulate the results in their favour quite easily. By testing different tactics, website owners sometimes find out elements of the algorithms and act accordingly to boost their ranking in the SERPs. Therefore, changes in the algorithms are often due to increased search engine spam, which I will talk more about in chapter 4.2.
Relevancy is an essential part of search engines. Search engine relevancy is determined by how well a document provides the information a user is looking for, as measured by the user (Sullivan 2003a). Because relevancy of the documents changes between different users and because users may have different search objectives, returning relevant results is not an easy task. Therefore, search engine algorithms can be extremely complicated with hundreds or even thousands of variables.