Guides
How search engines work, without the jargon
Eight short guides covering the whole pipeline: how crawlers discover pages, how indexes store them, how rankings are decided, how queries are understood, how SEO actually works, how spam is caught, and where all of it came from.
Start here, then read in order
How Search Engines Work: Crawling, Indexing and Ranking
The three stages every engine runs, in the order they happen — the foundation for everything else on this site.
Read the guide →Web Crawlers: How Search Engines Discover Pages
How Googlebot and Bingbot move across the web, what robots.txt really does, and how crawl budget is spent.
Read the guide →The Search Index: How Engines Store the Entire Web
Inverted indexes, postings lists, shards and tiers — how billions of pages become a millisecond answer.
Read the guide →Ranking Algorithms: How Search Engines Decide What Comes First
Relevance, links, quality, freshness and machine-learned ranking — what actually orders the results.
Read the guide →Query Processing: How Search Engines Understand What You Typed
Autocomplete, spelling correction, stemming, synonyms and search intent, from keystroke to results page.
Read the guide →SEO Basics: How to Help Search Engines Find and Rank Your Pages
The crawlable / indexable / rankable checklist, plus titles, structure, speed and measurement.
Read the guide →Spam, Quality and Trust: How Search Engines Fight Bad Results
Cloaking, doorway pages and link schemes, plus the filters, demotions and human raters that respond.
Read the guide →A Brief History of Search Engines: From Archie to AI Answers
The timeline of web search, from FTP file lists to models that answer before you click.
Read the guide →Put it to work
Search the web on Mugurdy, or read SEO Basics if you publish pages yourself.