- The aim is to get the canonical version of every important page indexed, not every URL.
- Bilingual sites need separate URLs, reciprocal hreflang and genuinely translated body content.
- Core Web Vitals targets are LCP 2.5 s, INP 200 ms and CLS 0.1, judged on real-user data at the 75th percentile.
- A good report ranks fixes by impact on key pages and says how each will be verified.
Many companies that commission a technical SEO audit receive the same thing: a PDF exported from a crawler, a health score out of 100, and a list of several hundred “errors”. The marketing team forwards it to the web developer, the developer asks which of the 600 items matter, and nobody can answer. Three months later the report is still in an inbox and the site has not changed.
A credible audit is different. It explains why specific pages are or are not being found, indexed and shown, it ties each finding to how search engines actually document their behaviour, and it ends with a short, ordered plan someone can execute. This guide sets out what that audit should cover for a corporate website, with particular attention to bilingual Arabic and English sites, and how to judge the report you receive.
Start with the right question: what should be indexed?
Before any crawler runs, the auditor should agree with you which pages matter: service pages, product or category pages, case studies, location pages, the contact and quotation routes. Everything else is secondary. This matters because Google itself says not every URL needs to be indexed. Its Page indexing report documentation states that the goal is to get the canonical version of every important page indexed, and that non-indexed URLs can be fine.
That single point removes much of the noise in tool exports. Tag archives, filtered URLs, paginated lists, old campaign parameters and attachment pages often show up as “not indexed”, and in most cases that is correct. An audit that treats them as errors has not understood the site.
The checks a corporate site audit must cover
The list below is the core. Each item is grounded in documentation published by the search engines, not in a vendor’s scoring model.
1. Crawl access and robots.txt
- Does robots.txt block anything important, including CSS and JavaScript the page needs to render?
- Is robots.txt being used to hide pages from search? Google’s robots.txt introduction is explicit that it is not a mechanism for keeping a page out of Google; a blocked URL can still be indexed if other pages link to it. Pages that must stay out of search need noindex or password protection.
- Are server errors (5xx), 401/403 responses or firewall rules stopping crawlers on some sections?
2. Indexing status, read from Search Console
The audit should work from Google Search Console’s Page indexing report, not only from a third-party crawl. That report groups URLs under named reasons such as “Crawled – currently not indexed”, “Discovered – currently not indexed”, “Duplicate without user-selected canonical”, “Duplicate, Google chose different canonical than user”, “Alternate page with proper canonical tag”, “Page with redirect”, “Soft 404” and “URL marked ‘noindex'” (Google). A useful report explains which of these affect important pages and why. “Alternate page with proper canonical tag” on a duplicate URL, for instance, is usually a sign that things are working.
3. Canonicalisation and duplicates
Corporate sites generate duplicates easily: http and https, www and non-www, trailing slashes, tracking parameters, print views, the same service listed under two menu paths. Google’s canonicalisation guidance describes redirects and rel=”canonical” as strong signals and sitemap inclusion as a weak one. It also warns against using robots.txt or noindex to choose a canonical, and against giving different canonical URLs for the same page through different methods. The audit should check that canonicals, internal links and the sitemap all point to the same version.
4. Redirects and past site moves
If the site has been redesigned or moved domains, the audit should test the old URLs. Google’s site move guidance recommends server-side permanent redirects from each old URL to its true equivalent, avoiding chains, not sending many old URLs to the home page, updating internal links to the new URLs, and keeping redirects for as long as possible, generally at least one year. Redirect chains left over from two or three redesigns are worth testing on any older corporate site.
5. Arabic and English versions
This is where bilingual sites in the Gulf and Egypt most often go wrong (our guide to Arabic and English website structure covers the build side). The audit should check:
- Separate URLs per language. Google recommends different URLs for each language version rather than switching language by cookie or browser setting, and advises against automatically redirecting users by language or IP.
- Correct hreflang. Per Google’s localized versions documentation, each version must list itself and all other versions, and if two pages do not point to each other the tags are ignored. Codes use ISO 639-1 for language and optional ISO 3166-1 Alpha 2 for region (for example ar, ar-SA, en-AE); a country code alone is not valid. x-default can mark the fallback page.
- Truly translated content. Google says localised versions are only considered duplicates if the main content remains untranslated. An “Arabic” page where only the menu and footer are in Arabic and the body is still English is a real problem, and one that crawlers rarely flag.
- Language is judged from visible content. Google states that it does not use hreflang or the HTML lang attribute to detect a page’s language. Those attributes still matter for accessibility and for hreflang mapping, but they will not rescue a page whose visible text is in the wrong language.
6. Core Web Vitals, from field data
Google’s Core Web Vitals page sets the targets: Largest Contentful Paint within 2.5 seconds, Interaction to Next Paint under 200 milliseconds and Cumulative Layout Shift under 0.1. According to web.dev, these are assessed at the 75th percentile of page loads, segmented by mobile and desktop, and INP became a stable Core Web Vital in 2024, replacing First Input Delay. A credible audit reports real-user (field) data where it exists, separates mobile from desktop, and identifies the templates causing the problem, such as an oversized hero video or an unreserved banner slot, rather than listing every URL individually.
7. Structured data
Markup should describe what is visibly on the page. Google’s structured data guidelines say not to mark up content that is hidden from readers, and that structured data makes a feature possible but does not guarantee it will appear. The audit should validate Organization, Breadcrumb, Article or Product markup with the Rich Results Test and flag markup that misrepresents the page, such as FAQ schema with questions that do not appear in the text.
8. Spam-policy and security risks
The audit should look for anything that conflicts with Google’s spam policies: near-identical city or service pages built to rank for similar searches (doorway abuse), pages mass-produced with little value (scaled content abuse), hidden text, cloaking, sneaky redirects and hacked content. On WordPress sites, injected spam pages and unknown admin users are worth checking directly, not only through a crawler, and a proper WordPress care plan should catch them between audits. After a spam update, see our note on what to check after Google’s September 2026 spam update.
9. Access for AI search crawlers
Some buyers now use AI assistants to research suppliers, so the audit should state which AI crawlers can reach the site and whether that was a deliberate decision. OpenAI controls ChatGPT search (OAI-SearchBot) separately from model training (GPTBot), and Perplexity notes that a firewall or bot-protection setting may block its crawler even when robots.txt allows it, a frequent hidden cause on sites behind CDNs. Google says there are no additional requirements, special AI text files or extra schema needed to appear in AI Overviews or AI Mode, so an audit recommending them as a Google requirement is wrong. The full crawler table and how to track citations are in our guide to getting cited in ChatGPT search.
10. Bing and IndexNow
Bing runs its own index and tools, so the audit should confirm the site is verified in Bing Webmaster Tools with a submitted sitemap. If the site uses IndexNow, the key file must be hosted correctly; the protocol’s documentation notes that submitted URLs are shared with all participating engines, which its FAQ lists as Amazon, Bing, Naver, Seznam.cz, Yandex and Yep. Google is not listed there, so IndexNow does not replace Search Console.
Tool export or action plan: how to tell the difference
| Signal | Auto-generated export | Credible audit |
|---|---|---|
| Headline metric | A vendor “health score” | Indexing status of the pages that bring inquiries |
| Volume of findings | Hundreds of items, all labelled errors | A short list grouped by root cause and template |
| Data source | One third-party crawl | Search Console, field performance data, server responses and a crawl |
| Bilingual sites | “Missing hreflang” counts | Checks reciprocity, codes and whether the Arabic body is actually Arabic |
| Recommendations | “Fix all warnings” | Who does what, in what order, and how success will be verified |
| References | None, or the tool’s own help pages | Search engine documentation for each rule |
Tool scores are not Google metrics. A site can score poorly in a crawler and perform well in search, or score highly while its key service pages are excluded as duplicates. Treat any score as a symptom to investigate, not a target.
What the final report must show
Whoever writes it, ask for a report with these parts:
- Scope and data. Which domains, languages and subdomains were checked, the crawl date, and which Search Console and analytics properties were used.
- Executive summary. Five to ten findings in plain language, each with the business consequence: “the Arabic service pages are not indexed because they canonicalise to the English versions”, not “canonical issues: 214”.
- Priority matrix. Each issue rated by impact on important pages and by effort, with an owner (developer, content team, hosting provider).
- Evidence. Example URLs, screenshots of Search Console statuses, and the response codes found.
- Exact fixes. The redirect map, the corrected hreflang set, the robots.txt change, written so a developer can implement them without interpretation.
- Verification plan. How each fix will be confirmed: URL Inspection, validation in the Page indexing report, a re-crawl, fresh field data once enough new visits have been recorded.
Running the audit and acting on it
An audit is only valuable if it is followed by changes, so plan the work in this order:
- Blockers first: anything that stops important pages being crawled or indexed (robots.txt, noindex left from staging, wrong canonicals, broken redirects).
- Language and duplication second: hreflang, untranslated Arabic bodies, duplicate URL versions.
- Templates third: Core Web Vitals and structured data, fixed once per template rather than page by page.
- Visibility controls last: AI crawler access, Bing verification, IndexNow, sitemap hygiene.
Many problems found in technical audits come from how the site was built, so findings often lead back to the theme, page builder or hosting setup. If the fixes require structural changes, it can be more efficient to rebuild key templates than to patch them, which is why SEO and website development should agree the plan together.
Before you commission a new audit, run the 20-URL test from the note above yourself, using URL Inspection in Search Console. If every important page is indexed with the canonical you expect, the audit can be narrow; if not, you already know where it has to start.
Frequently asked questions
What is included in a technical SEO audit?
Crawl access and robots.txt, indexing status from Search Console, canonicals and duplicates, redirects, language versions and hreflang, Core Web Vitals, structured data, spam and security risks, AI crawler access and Bing setup, ending in a prioritised fix list.
Is a low score in an SEO tool a Google ranking problem?
Not necessarily. Tool health scores are vendor metrics, not Google measurements. Use them to find issues to investigate, then confirm impact in Search Console and real-user performance data.
Do all pages on my site need to be indexed?
No. Google says its goal is to index the canonical version of every important page and that non-indexed URLs can be fine, for example filtered, duplicate or archive URLs.
Should I block AI crawlers in robots.txt?
It depends on your goal. OpenAI and Perplexity document separate crawlers for search visibility and, for OpenAI, model training, so you can allow search crawlers such as OAI-SearchBot and PerplexityBot while blocking GPTBot. Make it a deliberate decision and check your firewall too.
How often should a corporate website be audited?
Run a full audit after any redesign, domain or platform change, and before major campaigns; in between, monitor the Page indexing and Core Web Vitals reports in Search Console regularly.
- Google Search Console Help, Page indexing report.
- Google Search Central, Understanding Core Web Vitals and Google search results.
- web.dev (Google), Web Vitals.
- Google Search Central, Tell Google about localized versions of your page.
- Google Search Central, Managing multi-regional and multilingual sites.
- Google Search Central, Site moves with URL changes.
- Google Search Central, How to specify a canonical URL.
- Google Search Central, Introduction to robots.txt.
- Google Search Central, General structured data guidelines.
- Google Search Central, Spam policies for Google web search.
- Google Search Central, AI features and your website.
- OpenAI, Overview of OpenAI crawlers.
- Perplexity, Perplexity crawlers.
- IndexNow.org, IndexNow documentation.
- IndexNow.org, IndexNow FAQ.




