Website Optimization: A Technical SEO Checklist

Website optimization is the engineering layer underneath every ranking, click, and conversion your SEO program earns. You can publish the best content in your market, but if Googlebot cannot crawl it, index it, and serve it fast to a mobile user, that content will not rank. This is a technical SEO checklist built from real audits — the same items we run through when a brand onboards into SoarTask and we need to know whether the site is structurally sound or structurally broken.

This is a technical SEO checklist built from real audits — the same items we run through when a brand onboards into SoarTask and we need to know whether the site is structurally sound or structurally broken.

This guide covers the eight areas that consistently move the needle in a website SEO review: crawlability, indexability, Core Web Vitals, structured data, internal links, mobile, security, and migration safety. Each section is actionable — read it, open the relevant tool, and check the item the same afternoon. Where Google has published official guidance, we cite it. Where the evidence is weaker, we say so. We do not dress up nice-to-haves as emergencies.

1. Crawlability: can Googlebot reach your pages?

Crawlability is the most fundamental layer of search optimization. If a crawler cannot reach a URL, nothing else in your SEO program matters for that URL — not the content, not the links, not the schema. Before you optimize anything, confirm that the pages you care about are reachable.

1.1 Check robots.txt for blocked paths

Your robots.txt file tells crawlers which paths they may fetch. The most common defect is not a missing file — it is an over-broad Disallow rule, often written years ago for a staging subdomain and copied into production. Pull your robots.txt and read every Disallow line against your actual URL structure.

Google Search Central documents the directive semantics explicitly: Disallow: / blocks the entire site, and Disallow: /*?* blocks every URL with a query string — which on many e-commerce sites is most of the indexable catalog. Use the URL Inspection tool in Search Console to test any specific URL against your live robots rules.

What to verify

• The file returns HTTP 200 with a valid text/plain content type.
• No Disallow rule covers a path that should be indexed.
• The sitemap is declared with a Sitemap: directive, not just footer-linked.
• Crawl-delay is not set absurdly high — on most modern hosts it is unnecessary.

1.2 Manage crawl budget on large sites

Crawl budget is the number of pages Googlebot will crawl within a given timeframe. On small sites (under ~10,000 URLs) it is rarely a constraint — do not spend time "optimizing" it. On large, dynamically generated sites — faceted e-commerce, large directories, programmatic pages — crawl budget becomes a real bottleneck and a core part of website optimization. Google's Gary Illyes has stated crawl budget is primarily a concern for sites over roughly 10,000 pages; below that, quality signals matter far more.

Practical crawl-budget levers

• Eliminate low-value URLs from the crawl surface — faceted filters, sort parameters, session IDs, print views. Block or canonicalize them away.
• Prune the XML sitemap so it only contains canonical, indexable URLs. A sitemap full of redirected or 404 URLs wastes crawler time.
• Fix crawl-depth — important pages should be reachable within 3-4 clicks from the homepage.
• Reduce duplicate URLs with consistent canonical tags and parameter handling in Search Console.

2. Indexability: once crawled, will it be indexed?

Crawlability gets the crawler to the page. Indexability determines whether that page enters Google's index and is eligible to rank. These are different problems, and conflating them is one of the most common mistakes in an SEO review.

2.1 Audit noindex and nofollow directives

Run a crawler across your site and flag every URL returning a noindex directive — whether via the <meta name="robots"> tag or the X-Robots-Tag HTTP header. Cross-reference that list against the URLs you want indexed. Mismatches are the defect.

The most frequent cause of de-indexed content we see is a staging noindex flag never removed when the site went live, or a CMS toggle ("discourage search engines") enabled during development and forgotten. Search Console's Page Indexing report tells you exactly which URLs Google sees as blocked by noindex — start there.

2.2 Canonical tags and duplicate content

Canonical tags tell Google which URL is the preferred version when multiple URLs serve similar or identical content. They are a hint, not a directive — Google can and does ignore them when they conflict with other signals. A correct canonical strategy is central to website SEO health on any site with parameters, pagination, or syndicated content.

Canonical checklist

• Every indexable page has a self-referencing canonical pointing to its own absolute URL.
• Canonical URLs use the same protocol and host as the live site.
• Parameterized variants canonicalize to the clean base URL.
• Paginated series does not canonicalize all pages to page 1 (a known anti-pattern Google warns against).
• Canonical tags are consistent between HTML and sitemap entries.

3. Core Web Vitals: measure the experience, not the score

Core Web Vitals are Google's standardized user-experience metrics for page loading and interaction. They are a confirmed ranking signal — a tie-breaker among otherwise comparable pages, not a dominant factor. Treat them as a quality floor, not a ranking lever. The goal of SEO here is to clear the thresholds and then stop obsessing.

Core Web Vitals are Google's standardized user-experience metrics for page loading and interaction. They are a confirmed ranking signal — a tie-breaker among otherwise comparable pages, not a dominant factor. Treat them as a quality floor, not a ranking lever.

The three current field metrics are LCP (Largest Contentful Paint), INP (Interaction to Next Paint, which replaced FID in March 2024), and CLS (Cumulative Layout Shift). All three are measured against real Chrome users via the Chrome User Experience Report, and surfaced in Search Console and PageSpeed Insights.

3.1 Core Web Vitals thresholds

Note that these are field-data thresholds measured at the 75th percentile across a 28-day window. Lab data from a single PageSpeed Insights run does not determine your Core Web Vitals status — it estimates it. Use lab data to diagnose causes; use field data to confirm status.

3.2 The fixes that actually move the metrics

LCP — usually an image or a server problem

• Serve the LCP image with fetchpriority="high" and preload it; use modern formats (WebP, AVIF) and correctly sized srcset.
• Eliminate render-blocking CSS and JS above the fold; reduce TTFB to under 600ms at the 75th percentile.

INP — usually long JavaScript tasks

• Break up long tasks (> 50ms) with setTimeout yielding or scheduler APIs.
• Defer or lazy-load third-party scripts (analytics, chat widgets, ad tags); avoid large hydration work on interaction-heavy frameworks.

CLS — usually unsized media and late-loading fonts

• Always specify width/height (or aspect-ratio) on images, embeds, and ads; reserve space for late-loading content.
• Preload primary fonts and use font-display: swap or optional.

You can measure all three against any URL with our PageSpeed Checker, and track them over time as part of a recurring SEO audit.

4. Structured data: help Google understand, not just crawl

Structured data is JSON-LD (preferred over microdata) that describes your content in a machine-readable vocabulary — typically schema.org. It does not directly improve rankings. What it does is enable rich results, improve entity understanding, and increase click-through rate from the SERP by making your listing more informative.

Google Search Central maintains the authoritative list of supported structured data types and their eligibility for rich results. Implement only the types that map to real content on the page, and validate every implementation with the Rich Results Test before shipping.

4.1 The types worth implementing

• Organization / WebSite — site-wide identity, sitelinks search box.
• Article / BlogPosting — for editorial content; enables headline, date, author rich results.
• Product — price, availability, reviews; critical for e-commerce.
• FAQ — only for genuine Q&A content; Google narrowed FAQ rich-result eligibility significantly since 2023.
• BreadcrumbList — reflects your site hierarchy in the SERP.
• LocalBusiness — for physical locations; pairs with Google Business Profile.

4.2 Common structured data mistakes

• Marking up content not visible on the page (manual action risk).
• Using FAQPage where questions are not present as visible HTML.
• Missing required fields — validate against schema.org, not just Google's eligibility.
• Duplicate or conflicting JSON-LD blocks from multiple CMS plugins.

Structured data is a nice-to-have that compounds. Get the basics right (Organization, BreadcrumbList, Article) before chasing niche types. A clean, validated implementation beats a sprawling, error-ridden one.

5. Internal links: the architecture that distributes authority

Internal links are how PageRank and topical relevance flow through your site. They are also one of the most over-optimized surfaces in modern SEO — exact-match anchor text repeated across hundreds of internal links is a known spam signal, and Google's algorithms do penalize it. The goal is a natural, diverse, user-helpful internal link graph, not a keyword-stuffed one.

5.1 Architecture and depth

Every important page should be reachable within 3-4 clicks from the homepage. Pages buried 6+ clicks deep receive less crawl frequency and less internal authority. Use hub-and-spoke structures (a category hub linking to related articles, which link back) to keep depth shallow and topical clusters tight.

5.2 Anchor diversity and over-optimization guardrails

This is where most internal link programs go wrong. Repeating the same exact-match anchor to the same target URL across dozens of pages looks manipulative. SoarTask's internal link guardrails flag this automatically — the production thresholds we use:

• Maximum 5 repeated anchors per target URL.
• Maximum 100 inbound links per URL (link equity dilution beyond this).
• Maximum 30 outbound links per page.
• Exact-match anchor ratio below 50%.
• Maximum 3 new links to the same URL per month (link velocity).
• Anchor diversity score above 0.5 (Shannon-inspired, 0 = identical, 1 = fully diverse).

Check your internal link graph with our Internal Link Checker, and read our broader guidance in the SEO best practices guide. For measuring the impact of internal link changes, see our SEO analytics and measurement article.

6. Mobile and security: table-stakes, not differentiators

Google uses mobile-first indexing for the vast majority of sites — it crawls and indexes the mobile version and uses that as the canonical version for ranking. This makes mobile usability a prerequisite, not a bonus. Security (HTTPS) is similarly non-negotiable: HTTP pages are flagged as "not secure" in Chrome and receive a ranking disadvantage.

6.1 Mobile-first readiness

• Content and links on mobile match desktop (do not strip content on mobile).
• Structured data is present on the mobile page, not just desktop.
• Viewport meta tag is set; tap targets are ≥ 48×48 CSS pixels; body font ≥ 16px.
• No intrusive interstitials that obscure content on mobile load (Google penalizes these).

6.2 HTTPS and mixed content

• Entire site served over HTTPS with a valid certificate; HTTP URLs 301-redirect to HTTPS.
• HSTS header is set (Strict-Transport-Security); canonical tags point to HTTPS.
• No mixed-content warnings — every subresource (images, scripts, stylesheets, iframes) is HTTPS.

These are not differentiators. Every serious site has them. If yours does not, fix them before anything else in this checklist. A site that is not HTTPS and not mobile-usable is not competing — it is forfeiting.

7. Migration safety: where most technical SEO disasters happen

Site migrations — domain changes, platform migrations, redesigns, HTTP-to-HTTPS moves — are the single highest-risk event in technical SEO. Most traffic drops we are called in to diagnose trace back to a migration where redirects were incomplete, canonicals were reset, or content was removed without a plan. A migration is not finished when the new site launches; it is finished when the post-launch recheck confirms parity.

7.1 Pre-migration checklist

• Crawl the old site and save a full URL inventory (every indexable URL, redirect chain, canonical).
• Map every old URL to a new URL in a 1:1 redirect map. Do not collapse categories without a content rationale.
• Prepare 301 redirects for every mapped URL (avoid 302s for permanent moves).
• Replicate or improve structured data, title tags, meta descriptions, and H1s on the new site.
• Update the XML sitemap and internal links to point to the new URLs (do not rely on redirects for internal navigation).
• Notify Google of the domain change in Search Console (Change of Address tool for domain moves).

7.2 Post-migration recheck checklist

• Within 48 hours: crawl the new site and confirm no noindex flags leaked from staging.
• Within 48 hours: test old URLs and confirm they 301-redirect to the correct new URL (not the homepage).
• Within 1 week: confirm the new sitemap is submitted and reported as discovered.
• Within 2 weeks: compare GSC impressions/clicks against the pre-migration baseline. A short dip is normal; a sustained drop is a defect.
• Within 4 weeks: confirm Core Web Vitals field data has not regressed and no soft-404 surge appears in the Page Indexing report.

8. The comprehensive technical SEO checklist

Below is the consolidated checklist we run through during a first-pass technical audit. Priority is marked P1 (fix first — blocks indexing or rankings), P2 (fix this quarter — degrades performance), and P3 (nice-to-have — improves quality and resilience).

Run this checklist as a recurring quarterly audit, not a one-time project. Sites drift — CMS updates reset canonicals, new content gets published with staging flags, plugins inject duplicate schema. The sites that stay healthy are the ones that recheck, not the ones that audited once and assumed it held.

Key takeaways

• Crawlability and indexability come first. If Google cannot crawl and index your pages, no other optimization matters. Fix robots.txt, noindex leaks, and canonical errors before anything else.
• Core Web Vitals are a quality floor, not a ranking lever. Clear the thresholds (LCP ≤ 2.5s, INP ≤ 200ms, CLS ≤ 0.1) and stop obsessing. Use field data to confirm status, lab data to diagnose causes.
• Structured data enables rich results, not rankings. Implement types that map to real content, validate with the Rich Results Test, and prioritize Organization, BreadcrumbList, and Article first.
• Internal links distribute authority — and can over-optimize. Keep depth shallow, anchor text diverse, and monitor exact-match anchor repetition and inbound link concentration.
• Migrations are the highest-risk event in technical SEO. Save a pre-migration crawl, write a 1:1 redirect map, and recheck against the baseline for 4 weeks post-launch.
• Mobile and HTTPS are table stakes. If your site is not mobile-usable and fully HTTPS, you are not competing — you are forfeiting.
• Audit recurrently, not once. Sites drift. A quarterly technical recheck catches defects a one-time audit misses.

Frequently asked questions

How often should I run a technical SEO audit?

For most sites, a full technical audit every quarter is sufficient, with a lightweight recheck (robots.txt, sitemap, Core Web Vitals, indexing report) monthly. Sites that publish frequently, run on a headless CMS, or have recently migrated should audit more often — drift is faster on dynamic platforms. The audit is only useful if you act on the findings, so schedule fixes into the same cycle.

Does Core Web Vitals score directly affect rankings?

Core Web Vitals are a confirmed ranking signal, but a minor one — Google describes them as a tie-breaker among pages otherwise comparable in relevance and quality. Passing the thresholds will not make a weak page rank, and failing them will not sink a strong page. Clear the bar so vitals are not a liability, then invest in content and relevance where the leverage is larger.

What is the difference between crawlability and indexability?

Crawlability is whether a crawler can fetch a URL — governed by robots.txt, server response, and crawl budget. Indexability is whether a fetched URL is eligible to enter Google's index — governed by noindex directives, canonical signals, and content quality. A page can be crawlable but not indexable (blocked by noindex), and indexable but rarely crawled (buried deep in the architecture). Both must be true for a page to rank.

How long should I keep 301 redirects live after a migration?

At minimum 6 months; ideally 12 months or longer. Google needs time to recrawl and consolidate signals onto the new URLs, and external backlinks pointing at the old URLs continue to pass value through the redirect indefinitely. Keeping redirects live is cheap; losing link equity from a prematurely removed redirect is expensive. If you own the old domain, there is no good reason to drop the redirects.

Is structured data a ranking factor?

No, not directly. Google has stated repeatedly that structured data is not a ranking factor. Its value is indirect: it enables rich results (which improve click-through rate), helps Google understand entities and relationships, and improves how your pages are represented in the SERP. Implement it for visibility and CTR benefits, not because you expect it to move your rank position.

If you want to run this checklist against your own site without assembling the tooling, SoarTask automates the audit, crawl, internal link guardrails, and measurement loop in one workflow. Start a free workspace, connect your site, and get a prioritized technical SEO report — with evidence, not opinions — in minutes. Or try the individual SEO Audit, PageSpeed Checker, On-Page SEO Checker, and Internal Link Checker tools first.