SEO Basics: How to Help Search Engines Find and Rank Your Pages

Because crawling, indexing and ranking happen in sequence, SEO problems happen in sequence too. A page that cannot be crawled is invisible; a page that can be crawled but not indexed is invisible; a page that is indexed but ranked on page nine is invisible in practice. Fixing visibility means working through the gates in order rather than jumping to keyword ideas.

Three gates, in order

Technical foundations

• Sitemap. An XML file listing canonical URLs, referenced from robots.txt and submitted in each search console you care about.
• robots.txt. Keep it minimal and accurate. Do not block assets that render your pages, and never rely on it to hide sensitive URLs.
• Canonical URLs. Every page should declare one, and it should be the same URL you link to internally.
• HTTPS. Non-negotiable. Serve everything over TLS and redirect HTTP requests permanently.
• One URL per page. Avoid sessions IDs, unnecessary parameters and trailing-slash variants that split signals.
• Status codes. Return 404 for missing pages and 301 for permanent moves. Do not return 200 with an error message — that is a soft 404.
• Mobile first. Most crawling and most visitors are mobile; test layouts at phone widths before desktop.
• Speed. Aim for fast loading, minimal layout shift and quick response to input. These are measurable and fixable: compress images, defer scripts, cache aggressively.
• Structured data. JSON-LD that describes articles, products, FAQs or organisations helps engines understand pages and can earn richer result features.

On-page essentials

• Title. One per page, unique, descriptive, front-loaded with the topic, roughly 50–60 characters so it is not truncated.
• Meta description. A short summary, around 150–160 characters. It does not rank pages directly, but it is often the snippet — so write it as an invitation, not a keyword list.
• Headings. One <h1> that matches the page's purpose, then <h2> and <h3> for the sections below it. Headings are structure for readers and machines alike.
• URLs. Short, readable, hyphenated, and stable. Changing URLs later costs you redirects.
• Internal links. Link between related pages using descriptive anchor text. This is how crawlers learn what matters and how readers find more.
• Images. Descriptive filenames, alt text that explains the image, sensible dimensions and modern formats.
• Depth and clarity. Cover the obvious follow-up questions on the same page instead of forcing users back to the results page.

Content that deserves to rank

Ranking systems reward pages that satisfy an intent. That usually means a page has to be about one thing, complete for that thing, and honest about what it does not cover.

• Match the intent. A query asking "how" should not land on a product page.
• Be specific. Original data, first-hand experience, screenshots, tests and concrete examples are hard to imitate and therefore valuable.
• Be scannable. Subheadings, short paragraphs and lists let a reader confirm in ten seconds that the page is the right one.
• Keep it current. Update pages when facts change and show a visible review date.
• Consolidate. Two mediocre pages on one topic usually perform worse than one strong page. Merge rather than multiply.

Links, brand and off-page reality

Links remain the primary external signal of trust, and they cannot be manufactured honestly. The work that earns them is unglamorous: publish things people want to reference, be a useful presence in the communities around your subject, and make your material easy to cite. Directory submission sprees, paid link networks and comment spam are the classic ways to lose ground rather than gain it — and they are exactly what the next guide covers.

Measuring what changed

Search consoles give you the three numbers that matter most, and they are far more useful than rank-tracking theatre:

• Impressions — how often your pages appeared in results. Rising impressions mean the index is showing you more often.
• Clicks and click-through rate — whether your titles and snippets persuade. Poor CTR at good positions usually means the snippet, not the ranking, needs work.
• Average position — useful as a trend, misleading as an absolute, because it averages across queries and locations.

Change one thing at a time and give it weeks, not hours. Search results are noisy, and single-day movements rarely mean anything.

Common mistakes

• Blocking crawling while expecting indexing — the two directives are not interchangeable.
• Writing titles for machines only; they are also the headline people click.
• Publishing near-duplicate pages for every city, size or colour variant without adding value.
• Chasing a fixed "ranking factor" checklist instead of satisfying a query.
• Ignoring the search console after launch, then wondering why nothing is indexed.

Everything above follows directly from the pipeline described in How Search Engines Work. If you keep one rule, keep this one: make the page worth finding, then remove every obstacle between it and the crawler.