Technical SEO is not a separate discipline. It is the set of conditions that let the other work be seen.
Crawl budget, in plain terms
Search engines allocate a limited amount of fetching to each site. On a small site you will never notice. On a site with tens of thousands of URLs, it decides what gets refreshed.
You increase effective crawl capacity by:
- Removing URLs that should not exist (faceted combinations, tracking parameters, infinite calendars).
- Returning 404 or 410 for pages that are genuinely gone, rather than redirecting everything to the homepage.
- Keeping important pages shallow in the link graph.
- Serving fast, stable responses. Timeouts and 5xx responses waste crawl.
You do not increase it by adding more sitemaps.
Status codes that matter
- 200. Fine.
- 301. A permanent move. Use it when a URL has genuinely changed. Update internal links to point at the new URL so you are not relying on the redirect forever.
- 302. Temporary. Rarely what you want for a permanent restructure.
- 404. Not found. Correct for deleted pages.
- 410. Gone, permanently. Slightly faster to drop from the index than a 404.
- 5xx. Server failure. Crawlers back off. Persistent 5xx is the fastest way to lose visibility.
The most common self-inflicted wound is a redirect chain: A to B to C. Each hop costs a request. Collapse chains to a single hop.
Canonicals
A canonical tag says "this is the version that counts". It is a hint, but a strong one.
- Every indexable page should have a self-referencing canonical with an absolute URL.
- Duplicate content should point at the original, not at each other.
- Never canonicalise every page to the homepage. That tells the engine every page is a duplicate of one page, and the rest can disappear from the index.
- Canonical and hreflang must agree. A common international bug is a canonical pointing at the wrong locale.
Robots and sitemaps
robots.txt controls fetching. A meta robots tag or an X-Robots-Tag header controls indexing. They are different tools and people mix them up constantly.
- Use robots.txt to keep crawlers out of admin areas, search results and parameter sprawl.
- Use noindex when a page can be fetched but should not appear in results.
- Never combine "noindex" with a robots.txt block. If the crawler cannot fetch the page, it cannot see the noindex instruction.
Sitemaps should list canonical, indexable, 200-status URLs with a lastmod you actually maintain.
Speed and Core Web Vitals
Core Web Vitals are a tiebreaker, not a magic lever. Still, a slow page loses readers, and readers are the point.
- LCP (largest contentful paint) is usually an image or a hero heading. Serve it early, compress it, do not lazy-load it.
- INP (interaction to next paint) is about main-thread work. Ship less JavaScript.
- CLS (cumulative layout shift) is about reserving space for images, embeds and fonts.
Measure with field data (real users), not just lab tools. A page can pass Lighthouse and fail in the field.
JavaScript SEO
If the page is built in the browser:
- Ensure the main content, headings, links and metadata exist in the server response.
- Avoid rendering canonical tags or structured data client-side only.
- Test with a rendered fetch, not your browser.
A useful test: disable JavaScript, load the page, and read it. If you cannot tell what the page is about, neither can a crawler on its first pass.
The audit order
When someone asks you to "do a technical audit", work in this order:
- Indexability: is anything important blocked, noindexed or canonicalised away?
- Crawl paths: are important pages linked and shallow?
- Status codes: are there chains, loops, soft 404s?
- Duplication: are there multiple URLs for one page?
- Rendering: is the content in the HTML?
- Speed: what is slow, for whom, on what device?
- Structured data: does it validate and match the page?
That order matters. Fixing speed on a page that cannot be indexed is a waste of a week.