Your Site Got Redesigned, But Google Keeps Crawling Old Links

Hi everyone, this is Neo.

A friend who runs a B2B export business recently came to me in a panic: “Neo, I just spent a fortune redesigning my site, and now GSC (Google Search Console) is reporting a pile of 404 errors every day. Googlebot is crawling those old links I deleted ages ago like crazy, my crawl budget is being wasted, and my new pages are getting indexed slower. What do I do?”

Trust me, 90% of independent site operators have hit this exact problem. After a redesign, migration, or cleanup of outdated content, Google acts like a nostalgic old friend — it keeps visiting URLs that no longer exist, and your GSC 404 count spikes.

Don’t stress. This is not only normal, it’s easier to fix than you think. Today I’m walking you through a real case study and handing you the complete solution step by step, so you can put this headache behind you for good.

Why Is Google So “Sentimental”? Understanding the Mechanism First

Before we start, let’s get clear on why Google is so persistent. In short, there are three main reasons:

  1. Historical indexing: Your site has existed on the internet for a long time, and Google’s database has long recorded all your old URLs. Unless you give it an explicit signal that those links are dead, it’ll keep coming back to check out of habit.
  2. Backlinks: Other websites may still link to your old pages. As long as those external links exist, Googlebot will follow them, even if the page is no longer on your site.
  3. Internal links: Somewhere on your site — an old blog post or a product description — you might still have links pointing to deleted pages.

So simply deleting pages in your backend is nowhere near enough. You have to tell Google, in a language it understands: “This place has moved. Stop coming here!”

The core idea here isn’t to “block” those 404s — it’s to “channel” them. These three steps are the most effective method I’ve found in practice.

Step 1: 301 Redirect — The Best Choice, Protecting Both Authority and Traffic

If your old pages just changed addresses and the content still exists on your new site (or there’s a highly similar replacement), a 301 permanent redirect is the way to go.

  • When to use it? When old content has a corresponding new version.
    • B2C example: Your e-commerce site had an old product URL yourshop.com/product-a, and you optimized it to yourshop.com/best-product-a-for-summer.
    • B2B example: A machinery equipment site upgraded its old product model page your-b2b-site.com/model-x100.html to your-b2b-site.com/products/model-x200-plus.
  • Why it’s good: A 301 tells Google and the user’s browser: “This page has permanently moved to a new address.” Its biggest advantage is passing almost all of the SEO authority (ranking power) and user traffic accumulated by the old URL to the new one.
  • How to do it:
    • Apache server (.htaccess file): 
      Redirect 301 /old-page.html /new-page.html
    • Nginx server (nginx.conf file): 
      location /old-page.html {
    return 301 /new-page.html;
}
    • WordPress sites: Use a plugin like “Redirection” or “Rank Math” and set it up easily through a graphical interface.

Step 2: 410 Gone — The Strongest “Permanently Deleted” Statement

If the old page is completely obsolete with no replacement — say, an outdated job posting or a finished event page — the 410 Gone status code is a better choice than 404 Not Found.

  • Why better than 404?
    • 404 Not Found means “not found for now.” Google assumes it might be a temporary glitch and will keep coming back to check.
    • 410 Gone is an unmistakable signal: “This page has been permanently deleted. Never come back.” When Google receives a 410, it removes the URL from its index much faster, saving your precious crawl budget.1
  • How to do it?
    • Apache server (.htaccess file): 
      RewriteEngine On
RewriteRule ^path/to/your/old-page\\.html$ - [G]
    • Nginx server (nginx.conf file): 
      location /path/to/your/old-page.html {
    return 410;
}
    • WordPress sites (functions.php file): 
      add_action('template_redirect', function() {
    if (is_404()) { // You can set more precise conditions as needed
        global $wp_query;
        $wp_query->set_404();
        status_header(410);
        include(get_query_template('404'));
        exit();
    }
});

Step 3: The Final Push in GSC to Speed Things Up

After the server-side setup, we need to give things a nudge in GSC (Google Search Console) so Google processes it quickly.

  1. Update your Sitemap: Make sure your sitemap.xml only contains current, accessible URLs. Then resubmit it via GSC > “Sitemaps”. It’s like handing Google a brand-new treasure map.

  2. Use the “Removals” tool for bulk processing:

    • Let’s clear up a common misconception: Many people think GSC’s removal tool supports complex regex, but it actually doesn’t.
    • The right approach: Use the “prefix” and “wildcard *” features instead. This is especially effective for bulk invalid URLs auto-generated by CMS platforms like WordPress (e.g., /feed/ or /trackback/).
    • Real-world example: Say your site redesign left a pile of dead RSS links ending in /feed/.
      1. Go to GSC > “Indexing” > “Removals”.
      2. Click “New request”.
      3. Choose “Remove all URLs with this prefix”.
      4. Enter https://www.yourdomain.com/*/feed/ in the input box.
      5. Submit the request. This one simple move asks Google to block hundreds or thousands of invalid links matching that pattern in a single shot — the effect is immediate.

  3. Verify the fix: After you’ve set up the 301s or 410s and submitted requests through the removals tool, wait a week or two, then go to the GSC “Pages” > “Not found (404)” report and click “Validate fix”. This tells Google: “I’ve solved the problem — come check it out!”

Bonus: Finishing Touches

Once the core three steps are done, a little cleanup work will keep your site healthier.

  • Clean up internal links: Use a crawler like Screaming Frog or Sitebulb to crawl your entire site, find all “dead links” pointing to 404 pages, and fix them right away.
  • Optimize your 404 page: Even with our best efforts, users can still land on a nonexistent page. Build a user-friendly custom 404 page with a search box, a link back to the homepage, or a list of popular articles — it can win back a chunk of that traffic.
  • Be patient: SEO has no magic wand. Google’s index updates take time; depending on your site’s size and authority, this process can take a few weeks or even a month or two.

Summary

Alright, let’s recap the core steps from today:

  1. Content has a replacement -> set up a 301 redirect
  2. Content is obsolete -> serve a 410 status code
  3. Bulk invalid links -> remove them with GSC wildcards
  4. Finally -> update your sitemap and validate the fix

The 404 problem after a site redesign is fundamentally a communication problem with search engines. As long as you clearly convey your intent in the right technical language (301/410), this “old friend” Google will eventually understand and update its memory.

Handling these issues doesn’t just solve your immediate headache — it’s a deep technical SEO pass on your site that helps it run faster and more reliably in the future.

If your situation is more complex, or you’re not sure how to do this on your server type, drop a comment below and let’s figure it out together!

References:

  • Google Search Central Blog: Do 404s hurt my site? 1