How Search Engines Work - The Beginners Guide to SEO
Search engines have two major functions: crawling and building an index, and providing search users with a ranked list of the websites they've determined are the most relevant. Imagine the World Wide Web as a network of stops in a big city subway system. Each stop is a unique document (usually a web page, but sometimes a PDF, JPG, or other file). The search engines need a way to “crawl” the entire city and find all the stops along the way, so they use the best path available—links. Crawling and IndexingCrawling and indexing the billions of documents, pages, files, news, videos, and media on the World Wide Web. Providing Answers Providing answers to user queries, most frequently through lists of relevant pages that they've retrieved and ranked for relevancy. The link structure of the web serves to bind all of the pages together. Links allow the search engines' automated robots, called "crawlers" or "spiders," to reach the many billions of interconnected documents on the web.