Technical SEO is the practice of building and maintaining a website so that search engines crawl, render and index it correctly. It also makes pages load fast and work on any device. The work covers robots.txt, XML sitemaps, canonical tags, redirects, Core Web Vitals, HTTPS, structured data and JavaScript rendering.
Strong content still loses rankings when a technical problem keeps search engines from reaching or trusting the pages behind it. Below is what each technical SEO lever does, the exact thresholds Google publishes for it, and where to start a technical SEO audit.
Technical SEO covers crawlability, indexing, site speed, HTTPS, mobile-friendliness and structured data.
Core Web Vitals set loading, interactivity and visual-stability targets at 2.5 seconds, 200 milliseconds and 0.1, measured at the 75th percentile.
robots.txt and XML sitemaps control what Googlebot crawls and discovers on a site.
A canonical tag and the correct redirect type stop duplicate-content and ranking-signal problems after a site change.
What Is Technical SEO?
Technical SEO is the set of website and server changes that help search engines crawl, render and index a site. It sits apart from the words on the page and the links pointing to it. The work includes robots.txt rules, sitemaps, site speed, mobile-friendliness, HTTPS and structured data, and it decides whether good content ever gets seen.
Technical SEO is one of the three branches search engine optimization work splits into. On-page SEO optimizes a page's own content and HTML elements, such as titles and headings, and off-page SEO builds authority through backlinks and mentions. Technical SEO instead fixes how the site itself behaves for crawlers, browsers and servers.
Demand for the term reflects that split. The keyword "technical seo" gets searched about 8,400 times a month in the US. That is more than four times the volume for the longer phrase "what is technical seo," at roughly 2,000 searches (Ahrefs).
| Number | What it means | Source |
|---|---|---|
| 8,400/month | US searches for "technical seo" | Ahrefs |
| 2.5 seconds | Good Largest Contentful Paint threshold | web.dev |
| 200 ms | Good Interaction to Next Paint threshold | web.dev |
| 1994 | Year the robots.txt standard was proposed | Wikipedia |
| 2011 | Year Schema.org launched | Wikipedia |
| 2019 | Year mobile-first indexing became the default for new domains | TechCrunch |
How Does Google Crawl and Index a Website?
Google finds a page through a link or a sitemap, then crawls it with Googlebot. Only after that does Google render and index the page. How often Googlebot returns is called crawl budget, set by a crawl capacity limit and crawl demand. Google raises that limit once a site handles more requests without errors (Google Search Central, updated 22 July 2026).
Crawl capacity limit is the number of simultaneous connections and the amount of time a server allows before Google's crawlers back off. Crawl demand depends on a site's size, update frequency and page quality relative to other sites. A new page gets crawled sooner when it sits close to the homepage in the site's internal links and appears in an up-to-date sitemap.
What Does a robots.txt File Control?
A robots.txt file sits at a site's root. It tells crawlers which URLs or folders they can skip, using allow and disallow rules for each named bot. The file does not remove a page from Google's index on its own. A blocked page can still appear in results without a description.
Martijn Koster proposed the robots.txt standard in February 1994 while working at Nexor. It became a de facto standard by June 1994, once most crawler operators adopted it (Wikipedia). The file still works the same way three decades later: one rule per line, read before any other request to the site.
Why Does an XML Sitemap Matter for Indexing?
An XML sitemap lists every URL a site wants indexed, giving Google a complete map instead of relying on internal links alone. Google recommends pairing a full XML sitemap with a smaller RSS or Atom feed for recent changes. New and updated pages then get discovered faster (Google Search Central Blog, 2014).
A sitemap does not force indexing on its own, since Google still evaluates quality before indexing a submitted URL. Keeping the <lastmod> date accurate stops Google from re-crawling pages that have not changed, which saves crawl budget for pages that have.
How Does a Canonical Tag Stop Duplicate Content?
A canonical tag points search engines to the one preferred URL when the same or similar content exists at more than one address. Common causes include a "www" versus non-"www" version, or a URL with tracking parameters. Google then consolidates ranking signals onto that chosen URL instead of splitting them across duplicates.
The canonical link element is described in RFC 6596, in effect since April 2012 (Wikipedia). Most content management systems add the tag automatically on every page. A missing or wrong canonical after a site migration is one of the most common technical SEO problems an audit finds.
When Do You Use a 301 vs a 302 Redirect?
A 301 redirect tells browsers and search engines that a URL moved permanently, so Google transfers the old page's ranking signals to the new one. A 302 redirect marks the move as temporary, so Google keeps the original URL as the one it indexes and ranks.
Using a 302 for a permanent change is a common mistake, since it leaves an old URL ranking long after the content moved. A chain of more than one redirect also slows Googlebot and the visitor's browser. A single, direct 301 is the right fix for a permanent URL change.
What Are Core Web Vitals in SEO?
Core Web Vitals are Google's 3 user-experience metrics. They cover Largest Contentful Paint for loading speed, Interaction to Next Paint for responsiveness, and Cumulative Layout Shift for visual stability. Google sets a good score at 2.5 seconds, 200 milliseconds and 0.1. It measures all 3 at the 75th percentile of page loads on mobile and desktop (web.dev).
INP replaced First Input Delay as the responsiveness metric in March 2024. FID measured only the first interaction, instead of every click, tap or key press during a visit. A page that fails these thresholds still gets indexed, but Google uses page experience as a tiebreaker between pages of similar relevance and authority.
Does HTTPS Affect Search Rankings?
Yes. Google announced HTTPS as a lightweight ranking signal on 7 August 2014 (Google Search Central Blog). It later made the browser padlock the default expectation for any page handling forms or logins. HTTPS encrypts the connection between a browser and a server. That also protects a site's own analytics and conversion data from tampering.
Browsers mark an HTTP page as "Not Secure" in the address bar, which erodes trust before a visitor reads a word of content. Moving a site to HTTPS needs a 301 redirect from every old HTTP URL, or Google treats the HTTP and HTTPS versions as duplicate pages.
How Does Structured Data Help Search Engines Understand a Page?
Structured data is a standard-format label, usually written in JSON-LD. It tells a search engine what a page's entities are: a product, a review, an event, an organization or a FAQ. Google and other search engines use it to build rich results, such as star ratings or FAQ dropdowns, directly in the search listing.
Bing, Google and Yahoo! launched the shared Schema.org vocabulary on 2 June 2011. Yandex joined that November (Wikipedia). The same markup now works across every major search engine. Structured data must match the visible content on the page, since Google removes markup that misrepresents what a visitor sees.
Can Googlebot Read JavaScript-Rendered Content?
Yes, but with an extra step. Googlebot crawls a page's raw HTML first and queues it for rendering. A headless Chromium then renders the JavaScript before Google indexes the final content. Google documents this three-phase crawl, render and index process directly for developers building for Search (Google Search Central).
The render step often sits in a queue for a while. Vercel's analysis of its own pages found Google rendered the median page 10 seconds after crawling it. The slowest 10% waited roughly 3 hours (Vercel). Server-side rendering avoids that wait entirely, since Google indexes the server's HTML without running any JavaScript first.
What Belongs on a Technical SEO Audit Checklist?
A technical SEO audit checks crawlability, indexing, site speed, mobile-friendliness, HTTPS, structured data and internal link structure in one pass. It usually runs with a crawler tool plus Google Search Console. It also covers two more items: hreflang tags for multi-language sites, and log file analysis. Log file analysis reads server logs to see exactly which pages Googlebot visited.
Site architecture: keep every page 3 clicks or fewer from the homepage, so link authority and crawl priority reach it.
URL structure: use short, readable, lowercase URLs with hyphens, not underscores or session parameters.
hreflang tags: mark the language and region of each page variant, so Google serves the right version to the right country.
Log file analysis: check the raw server logs for the pages Googlebot requested, which often differ from what a sitemap lists.
Mobile-first indexing: match content and structured data between the mobile and desktop versions. Google has used the mobile version for ranking every new domain since 1 July 2019 (TechCrunch).
Technical SEO FAQ
What is technical SEO in simple terms?
Technical SEO is the work that makes a website easy for search engines to crawl, index and rank. It covers crawlability, site speed, mobile-friendliness, HTTPS and structured data.
How is technical SEO different from on-page SEO?
Technical SEO fixes how a site works for crawlers and servers. On-page SEO optimizes a page's own content and HTML elements, such as titles and headings.
What are Core Web Vitals?
Core Web Vitals are Google's 3 user-experience metrics: Largest Contentful Paint, Interaction to Next Paint and Cumulative Layout Shift. Google measures each at the 75th percentile (web.dev).
Does HTTPS affect SEO rankings?
Yes. Google has used HTTPS as a lightweight ranking signal since announcing it on 7 August 2014 (Google Search Central).
What tools check technical SEO?
Google Search Console and PageSpeed Insights are free, and Screaming Frog or Ahrefs Site Audit are paid crawlers used for larger sites.
How often should a technical SEO audit happen?
A technical SEO audit happens after a major change, such as a CMS migration, a redesign or a traffic drop, and once a year otherwise.
The Bottom Line
Technical SEO is the foundation the rest of SEO stands on: content and links do not rank a page Google cannot crawl, render or trust. Start with the free checks: Search Console's coverage report, PageSpeed Insights and a robots.txt read-through. Then move to a full crawl with a tool such as Screaming Frog or Ahrefs Site Audit. Our technical SEO audit services cover every item on the checklist above and hand over a prioritized fix list instead of a bare scan.