Technical SEO

A health score is the most misread number in any audit tool, including this one. Horizon's crawl returns 70%, and the number is accurate. What it describes is eight pages, seven of which were readable, on one day, requested by one identity. It is not a statement about the website.

Every conclusion drawn from a crawl inherits the crawl's boundaries, and those boundaries are set by choices somebody made before the run started. Change the start URL, the page limit, the discovery mode, or the user agent, and the same site returns a different score.

None of this makes the number useless. It makes the number conditional, and the condition has to travel with it. An audit that reports 70% without reporting eight pages is not wrong so much as unfalsifiable, which is worse.

A crawl is a sample. The scope belongs in every sentence the sample is used to support.

What actually defines the sample

Discovery mode is also how orphans surface. A page present in the sitemap and absent from the link crawl has no internal path, and you learn that for free by comparing two runs you were going to do anyway.

Identity changes the answer

Servers do not treat all requesters the same. A firewall, a rate limiter, or a bot-management rule can serve one thing to a browser and something else to a crawler, and the crawler faithfully reports what it was given.

This produces a specific and expensive false positive. The crawl reports blocked pages or thin content across a template. The developer opens the same URLs in a browser, sees a working page, and concludes the audit tool is broken. Both observations are correct. They were made by different identities and the server answered them differently.

Record the identity used and whether anything about the run looked like throttling. Slow responses that get slower through a run, a burst of errors partway in, or a clean first hundred pages followed by uniform failures are all the shape of rate limiting rather than the shape of a broken site.

Say the sample size out loud

The sentence that turns a useful audit into a misleading one is almost always a generalization nobody checked. An eight-page sample can confirm that one calculator fails to render. It cannot support a claim about the technical state of a five thousand page site, and the moment somebody writes that claim down, every later decision inherits it.

The fix is a phrasing habit rather than a process. Name what was tested, then say what it showed. Seven of eight sampled pages were readable, and the unreadable one is the settlement calculator. That sentence is defensible in a year, and it is barely longer than the one that is not.

Set the scope before you run

Four decisions, made deliberately, recorded with the run. The point is that somebody else could reproduce your numbers next quarter.

1. Choose the property and the start URL.Confirm the protocol and host variant you actually mean. A crawl started on the wrong one measures a redirect chain and calls it a site.
2. Set the page limit and the discovery mode.Decide whether you are testing what the site links to, what it claims in a sitemap, or a list you supplied. Each answers a different question and the answers rarely agree.
3. Record the crawler identity and any access limits.The user agent and anything that blocked or throttled the run. Without this a firewall result and a site result are indistinguishable in the report.
4. Check that representative templates were reached.A sample that missed the product template, or the location pages, cannot say anything about them. Confirm coverage before you read severity counts.

Indexing issue is not a diagnosis. It is a category containing at least six independent controls, each with a different failure, a different fix, and a different place to verify it. Collapsing them into one label is how a team spends a sprint on the wrong one.

The controls also interact, and some combinations cancel each other out in ways that look like the tool is lying. Most of the confusing cases in this lesson are two correct-looking settings that contradict.

Name the control that failed, not the symptom. Six controls, six different fixes.

Six controls, six different jobs

The header directive deserves its own mention because it is invisible in the place people look. A team checks the rendered head, finds no noindex, and concludes the page is indexable. The X-Robots-Tag is in the response header, set by a CDN rule somebody added two years ago, and it overrides everything they just examined.

The blocking mistake, in two forms

The most common self-inflicted indexing problem is blocking a crawler from seeing the very instruction you want it to obey. It happens in two versions and both look like decisive action.

Blocking a noindex page in robots.txt. The page carries noindex, so somebody also disallows it to be thorough. The crawler now cannot fetch the page, which means it cannot see the noindex, which means the URL can persist as a bare listing indefinitely. The robots rule defeated the removal it was meant to reinforce.

Blocking a redirecting URL in robots.txt. Horizon retires an obsolete city page and redirects it to a consolidated destination. Disallow that old URL and the crawler never requests it, never sees the redirect, and never passes anything along. The redirect is correct and unreachable.

The rule underneath both: a control only works if the crawler is allowed to observe it. Robots.txt governs the request, and everything else governs what happens once the request succeeds. Use robots.txt to prevent crawling you do not want, never to enforce an instruction that lives inside the response.

Work the controls in order

Same discipline as module 1: stop at the first control that fails, because everything below it is unobservable until that one clears.

1. Check the status and the full redirect chain.Not just the final code. A chain of three hops, or a 200 that becomes a 503 under crawl load, are different problems from the 404 the report shows.
2. Check robots.txt against the exact path.Including query strings, and including the rule order, because a later allow can override an earlier disallow in ways that are easy to misread.
3. Check both robots directives.The meta tag in the rendered head and the X-Robots-Tag in the response header. Checking only the first is the single most common miss in this lesson.
4. Compare declared canonical against selected canonical.URL Inspection reports what the engine chose. When it disagrees with your declaration, the disagreement is the finding and the page will not accumulate its own signals.
5. Verify something points at the URL.Internal links and sitemap membership. A perfectly indexable page nothing links to is not an indexing problem, and fixing indexing controls will not help it.

Search engines render JavaScript. That has been true for years, and the myth that they do not causes damage in both directions: teams rebuild working sites to escape a problem they never had, and teams dismiss genuine rendering failures because rendering is supposedly solved.

The real question is narrower. Does the useful content survive when something goes wrong, and can it be read without an interaction nobody is going to perform? Horizon's settlement calculator answers no to both, and it is the one critical finding in the entire crawl.

Rendering is a resilience question, not a JavaScript question. Schema describes what is visible; it does not create it.

Rendering is about resilience

Compare the initial HTML against the rendered result on the templates that matter. Where they differ, ask what happens if the difference never arrives: a blocked resource, a script error, a slow third party, a timeout under load.

Horizon's calculator is the worked case. The instructions and the explanatory content only exist after scripts run. When they fail, the page is an empty shell. A person sees a blank box and leaves, and a crawler stores a page with nothing on it. Both outcomes are bad and neither is fixed by markup.

The fix is to make the useful content present without requiring the interactive layer to succeed. Server-render the explanation, keep the tool as the enhancement, and the page degrades into something still worth reading instead of into nothing.

Performance is a user claim before it is a search claim

Name the experience whenever you raise a performance issue. The calculator takes four seconds before anyone can type a number is a sentence somebody can prioritize. The score is 61 is a sentence that starts an argument about the score.

Structured data has two requirements

The first is that it matches what a person can actually see on the page. Markup describing reviews that do not appear, prices that are not shown, or an author nobody is credited as, is a misrepresentation, and it is the version of schema work that gets sites into trouble.

The second is that it maps to a documented feature the page is eligible for. Valid markup for something no engine displays is syntactically correct and operationally pointless, and it still costs maintenance every time the template changes.

What schema cannot do

Structured data does not improve a page. It describes one. When the calculator's content is missing because rendering failed, adding markup produces a machine-readable description of a page that still does not work for the person who arrived.

It also buys eligibility rather than an appearance, and eligibility is not a ranking. This distinction matters when reporting: shipping schema is completed work, and it belongs in the work layer of the measurement tree rather than in the visibility layer, until something actually changes in a report.

Test what a machine and a person both get

Six checks per important template. Template rather than page, because these problems are almost never limited to one URL.

1. Compare initial HTML with rendered content.On the templates that carry the business, not on the homepage alone. The gap between the two is where every rendering finding lives.
2. Check resource access and script errors.A blocked stylesheet or a failing third-party script produces exactly the symptoms people attribute to crawlers not running JavaScript.
3. Ask what remains when the script fails.This is the resilience question and the one that decides whether the finding is critical. A page that degrades to nothing is a different severity from one that degrades to a readable article.
4. Read field and lab data separately.Field describes your visitors, lab lets you compare a change against itself. Mixing them produces claims neither one supports.
5. Validate schema against the visible page.Every claim in the markup should be findable by a person reading the page. Anything else is a description of a page you did not build.
6. Confirm the feature is documented and eligible.Valid markup for an appearance that does not exist is maintenance with no upside, and it will break silently the next time the template changes.

An audit returning 340 issues has not given you 340 pieces of work. It has given you a symptom list, and symptom lists sorted by severity label produce queues where a site-wide template fault sits below a batch of missing alt attributes because there are more of the latter.

The work of this lesson is turning that list into a small number of causes, ordered by what they cost the business, each with a way to tell whether the fix worked.

Issue count is not priority. Group by root cause, order by affected importance, and close only on re-measured evidence.

Group by root cause first

Most large issue counts collapse dramatically once grouped. Two hundred pages missing a meta description is usually one template that stopped generating them, which is one fix and one deploy. Counting it as two hundred items makes it look like the biggest problem on the site, and prioritizing by count guarantees it gets worked first.

Group before you estimate anything. Effort estimates on ungrouped symptoms are meaningless, because the same fix is being estimated repeatedly. The grouped version usually reveals that the enormous list is a dozen causes, and that three of them are one line of template code.

Attach the affected important URLs to each group rather than the full list. Important means pages that serve the outcome from module 1. A root cause touching four hundred pages nobody visits ranks below one touching the six pages that produce consultations.

What actually sets the order

Confidence is the input people leave out, and it is what stops a queue filling with confident guesses. A cause you are seventy percent sure about, with a cheap test available, should usually be tested before it is scheduled.

Closing requires the same scope

A fix is not done when it ships. It is done when a repeat test, using a comparable scope, shows the original evidence changed. That means recrawling with the same start URL, the same limit, the same discovery mode, and the same identity as the run that found it.

Change the scope and the comparison is void. A hundred-page crawl after an eight-page one will produce a different health score whether or not anything was fixed. Somebody will report that number as an improvement, and it will be arithmetic rather than progress.

Write acceptance criteria that a different person could evaluate. The calculator renders its instructions and explanation with JavaScript disabled is testable by anyone. The calculator is fixed is not, and it is how items get closed while the problem survives.

Build the remediation queue

The output is a short ordered list where each row names a cause, its affected important pages, and the observation that will close it.

1. Group identical root causes.Do this before reading counts or estimating effort. It is normal for a three-figure issue list to become a dozen causes, and the shape of the queue changes completely once it does.
2. Attach affected important URLs and the evidence.Important against the module 1 outcome, not against the total. The evidence is what stops the fix being re-diagnosed by whoever picks it up.
3. Order by the five inputs, with confidence visible.Write the reason for the position. A priority nobody can explain gets re-argued in the next planning meeting and usually loses to whatever is loudest.
4. Assign an owner and testable acceptance criteria.Testable means somebody other than the author could run the check and get a definite answer. Anything softer closes on opinion.
5. Recrawl the same scope and close on the evidence.Same start URL, limit, discovery mode, and identity. If the evidence did not change the way you predicted, the item is not closed, whatever shipped.