Technical SEO

If you'd rather have someone handle this for you, Matthew Boyle has worked on technical SEO with a range of business owners and can advise on growing leads, traffic, and sales through your Google Maps presence. One project was a site with a billion pages and a failed migration which he fixed. Get in touch to find out how he can help.

Robots.txt is a file that sits on your website and tells search engine crawlers which pages they can and cannot access. Get it wrong and you can accidentally block Google from crawling your entire site and wonder why your rankings have disappeared.

It is more common than you think. A developer makes a change, a website migration goes slightly wrong, someone copies a staging site configuration to live. Suddenly your robots.txt is telling Google and other search engines to go away and it quietly obliges.

To check yours, type your domain into a browser followed by /robots.txt — for example yourwebsite.com/robots.txt. If you see a line that reads Disallow: / under User-agent: * your whole site is blocked. That is bad. Google also has a free Robots.txt Tester inside Google Search Console. It lets you test individual URLs against your current rules and see exactly what crawlers can and cannot access. Worth checking, especially if you have recently moved platforms, redesigned your site, or had any developer work done.

Want it broken down for you? Try ELI5 my robots.txt — paste yours in and get a plain-English, line-by-line explanation of exactly what it's telling search engines and AI bots.

A favicon is the small icon that appears in a browser tab next to your page title. It sounds minor but it's a trust signal — a site without one looks unfinished. It also appears in bookmarks and on mobile when someone saves your site to their home screen.

Most website builders will have a favicon upload option in their settings. You need a square image, ideally 512x512 pixels, saved as a PNG. If you don't have a logo, even a single letter or simple icon works fine. There are free tools like favicon.io that will generate one for you in seconds and give you the right file formats.

Make sure your images are an optimised reasonable file size, the right dimensions for where they're used, and proper alt text for accessibility.

Huge, unoptimised images slow your page down badly, and page speed is something Google factors into ranking. Slow-loading pages also mean visitors leave before they've even seen what you offer.

You don't need anything technical to fix this. There are free online tools (like TinyPNG or Squoosh) that compress images in seconds with barely any visible quality loss, or most website builders have a built-in compression option you can switch on. Either way, check your images aren't several megabytes each — it's one of the easiest wins available.

Compressing the image keeps the image normal to the human eye but reduces the size for machines loading the image saving time and space.

Think of a heading hierarchy like a book. H1 is the title, H2s are chapters, H3s are sections within those chapters. That structure helps a human skim a page, and it does the same job for crawlers.

Search engines and AI bots aren't reading your page the way a person does, they're parsing the document structure to understand what matters and how it's organised. One H1 per page that states what it's about. H2s for your main sections. H3s for anything nested under those. Skip levels or use headings for styling rather than structure, and you make it harder for a robot to make sense of the page, even if it looks fine to a person.

Google Search Console lets you inspect any URL on your site and see how Google actually renders it — the page as Googlebot sees it, not as you see it in your browser. Bear in mind there are thousands of different devices, screen sizes, and resolutions out there, and the way your page looks on your laptop is not how everyone else sees it, and not how Google sees it either.

If the rendered version looks broken, missing content, or nothing like the real page, something is wrong. JavaScript not rendering, content blocked by robots.txt, or a crawl error — the inspection tool will surface it. If something looks visually off, treat it as a problem that needs investigating. It's one of the most useful diagnostics available and it's free.

What It Is FAQ accordions are interactive web elements that expand to reveal answers when a user clicks on a question. They allow pages to store a dense amount of structured information in a clean, compact layout.

However, many websites build these accordions using dynamic JavaScript that loads the text into the Document Object Model (DOM) only after an interactive click event occurs. Without that user interaction, the text does not exist in the rendered page code.

Why It Matters Search engine crawlers and LLM scrapers generally browse pages in a static state—they do not trigger click events or interact with collapsed elements as a human user would.

If your FAQ content requires a click to physically load into the code, search engines and AI models will treat that content as invisible. This means valuable long-tail keywords, direct answer targets, and rich context won't be indexed or used to train AI models on your brand.

And a really key important part, the type of quesiton you include in your FAQs are by definition freuqntly asked questions and are going to be the EXACT same type when prompted by the user in their LLM tool. It can be some of your most important helpful content and its hidden.

How to Check and Fix You can audit your website's FAQ components in under a minute using Chrome DevTools:

Open a page on your site with collapsed FAQs using Google Chrome.

Click to open one of the questions and copy a unique phrase from inside the answer.

Right-click anywhere on the page and select Inspect to open DevTools.

Select the Elements tab, press Ctrl + F (or Cmd + F on Mac), paste the text, and press Enter to confirm the text string appears in the source code.

Reload the page while keeping DevTools open, without clicking or interacting with the page.

Search for the exact same text string in the Elements tab again.

If the text string is missing on your second search, the component is only injecting text on a click event. To correct this, the hidden content should be rendered directly into the HTML upon initial page load, using standard CSS properties (like display: none or visibility: hidden) to handle the visual expand/collapse behavior.

Google keeps changing how it crawls, renders and indexes websites. A technical setup that was fine two years ago could now be holding your site back.

The Art of SEO by Eric Enge, Stephen Spencer and Jessie Stricchiola is a comprehensive reference that covers the technical side in depth. Google's Search Off the Record podcast gives you behind-the-scenes insight from Google's own Search team, including episodes on crawlers and page size limits. On Reddit, r/TechSEO is the go-to community for the technical side of SEO.

Stay switched on, or you risk fixing problems the old way.