Appendix D — Decision trees

The Technical SEO Reference · each tree's evidence lives in the noted chapter

D.1 Keep out of Google: crawl-block vs noindex vs auth (→ Ch 7 §7.7, Ch 3)

flowchart TD
    A[Content must not appear in Google] --> B{Is it confidential /<br>must no one reach it?}
    B -- yes --> C[Authentication / password.<br>Only real protection — robots.txt and<br>noindex do nothing for humans]
    B -- no --> D{Must the URL never<br>show in results?}
    D -- yes --> E[noindex meta/header —<br>URL MUST stay crawlable<br>never combine with robots disallow]
    D -- no, just save crawl cost --> F[robots.txt disallow —<br>⚠️ URL can still be indexed<br>from links, without content]

D.2 The removal ladder (→ Ch 7 §7.8, Ch 14 §14.9)

flowchart TD
    A[Content must come OUT of Google] --> B{Emergency?}
    B -- yes --> C[Removals tool: hides in ~hours<br>⚠️ TEMPORARY — ~6 months]
    C --> D[…then make it permanent below]
    B -- no --> D{Does the content still exist?}
    D -- gone --> E[404 / 410 — drops on recrawl]
    D -- stays public --> F[noindex, crawlable]
    D -- stays private --> G[Auth-gate it]

D.3 Faceted navigation (→ Ch 10 §10.8)

flowchart TD
    A[Filter/facet URLs exist] --> B{Do any facet pages<br>earn search demand?}
    B -- no --> C{Control point available?}
    C -- robots.txt --> D[Disallow parameter patterns +<br>allow the canonical listing<br>first-tier per Google]
    C -- implementation --> E[Fragment-based filters —<br>no crawl impact at all]
    B -- some --> F[Whitelist crawlable facets:<br>& separator, stable order,<br>404-in-place for empty/nonsense combos]
    F --> G[Second-tier only if needed:<br>canonical to unfiltered slower;<br>nofollow needs 100% anchor coverage]

D.4 Choosing a redirect (→ Ch 16 §16.1)

flowchart TD
    A[URL must send users elsewhere] --> B{Permanent move?}
    B -- yes --> C[301/308 server-side —<br>target becomes canonical]
    B -- no --> D{Why temporary?}
    D -- outage/variant test --> E[302/307 — source stays<br>canonical and in results]
    D -- A/B test --> F[302 + canonical on variants —<br>never 301, never noindex]
    A --> G{Server config impossible?}
    G -- yes --> H[Instant meta refresh = permanent<br>JS redirect = last resort<br>renders may fail]

D.5 Planned downtime (→ Ch 16 §16.9)

flowchart TD
    A[Site must pause] --> B{How long?}
    B -- hours–2 days --> C[503 + Retry-After, static page<br>⚠️ robots.txt must NOT 503]
    B -- weeks --> D[Indexable 200 placeholder home;<br>limit functionality, keep URLs alive]
    B -- prefer zero impact --> E[Stay up: disable transactions,<br>banner + structured-data availability]
    A --> F[NEVER: Removals tool, noindex,<br>403/404-ing the site for maintenance]

D.6 AI-bot policy matrix (→ Ch 5 §5.9)

flowchart TD
    A[Decide per crawler class] --> B{Google Search visibility<br>incl. AI Overviews/AI Mode?}
    B -- keep --> C[Allow Googlebot; tune with snippet<br>controls / GSC gen-AI toggle only]
    B --> D{Gemini training/grounding?}
    D -- opt out --> E[Disallow Google-Extended —<br>zero Search impact]
    A --> F{Third-party AI search<br>ChatGPT/Perplexity/Copilot?}
    F -- want citations --> G[Allow their search bots<br>OAI-SearchBot, PerplexityBot,<br>Claude-SearchBot…]
    F -- protect content --> H[Disallow training bots<br>GPTBot, ClaudeBot, CCBot…<br>⚠️ user-triggered fetchers may<br>ignore robots.txt anyway]

D.7 hreflang debugging (→ Ch 11 §11.13)

flowchart TD
    A[Wrong-locale URL ranking] --> B{Annotations reciprocal<br>incl. self-reference?}
    B -- no --> C[Fix return links —<br>ignored per-pair otherwise]
    B -- yes --> D{Codes valid?<br>no UK/EU, no region-only,<br>no es-419}
    D -- no --> E[Fix codes — reserved codes<br>are ignored SILENTLY]
    D -- yes --> F{Variant's Google-selected<br>canonical = itself?}
    F -- no --> G[Cross-locale canonicalization:<br>same-language canonicals,<br>differentiate content, check cluster membership]
    F -- yes --> H{Annotations point at<br>200, indexable, canonical URLs?}
    H -- no --> I[Handshake broken —<br>annotate final URLs only]
    H -- yes --> J[Check head validity + method conflicts;<br>then consider translated-results confounder]