Three stages
- Crawling: automated programs (crawlers, such as Googlebot) discover pages by following links and reading sitemaps, then download them.
- Indexing: the search engine analyses the page’s text, images, structured data and other signals, and may store it in its index. Not every crawled page is indexed; duplicates and low-value pages are often left out.
- Serving and ranking: when someone searches, the engine picks the most relevant, helpful results from its index, considering factors such as the meaning of the query, the relevance and quality of content, usability, and context like the searcher’s location and language.
Google states that it does not accept payment to crawl a site more often or rank it higher. Ads are separate and labelled.
What SEO actually is
Search engine optimisation (SEO) is the practice of making your site easy for search engines to crawl and understand, and genuinely useful to the people searching. It is not about tricking an algorithm. Most of the work is: helpful content that matches what people search for, a technically sound site, a good page experience, and a reputation that earns links and mentions naturally.
Key takeaways
- Search works in three stages: crawling, indexing and serving/ranking.
- Crawled pages are not guaranteed to be indexed or ranked.
- SEO means being understandable to search engines and helpful to people, not gaming the system.
Practice: Check how a search engine sees you
- Search
site:yourdomain.comto see a rough sample of indexed pages. - Note any important pages that are missing.
- Note any pages that should not appear (test pages, duplicates).
Go further: Set up Google Search Console for your site (covered later in this course) to see exact indexing data.