A search engine operates, in the following order: 1) Crawling; 2) Deep Crawling Depth-first search (DFS); 3) Fresh Crawling Breadth-first search (BFS); 4) Indexing; 5) Searching. Web search engines work by storing information about a large number of web pages, which they retrieve from the WWW itself. These pages are retrieved by a web crawler (also known as a spider) - an automated web browser which follows every link it sees, exclusions can be made by the use of robots.txt.

The contents of each page are then analyzed to determine how it should be indexed. Data about web pages is stored in an index database for use in later queries. Some search engines, such as Google, store all or part of the source page (referred to as a cache) as well as information about the web pages, whereas some store every word of every page it finds, such as AltaVista. This cached page always holds the actual search text since it is the one that was actually indexed, so it can be very useful when the content of the current page has been updated and the search terms are no longer in it.