There was a time when the internet was just a wee little one when it was important to have a web domain that people could remember. It seems like so long ago! People were writing URLs down in notebooks, people were buying up easy to remember domains and selling them for a profit, Myspace was still cool… different times.

Now, the most important thing for your business and your brand is search equity. How easily can someone find you on the web? You want to be top of the list, the first result, in any search query, and here is a quick tip on how to regain that search equity.

The tools that you will need:

• Screaming Frog - paid subscription
• Google Search Console

Once you have your tools, let’s start the process...

STEP 1: Verify the canonical version of the domain in Search Console.

Here’s what I mean… Imagine your site’s homepage can be accessed through multiple URLs:

• https://sitename.com/
• https://www.sitename.com/
• http://sitename.com/
• http://www.sitename.com/

If your site already uses canonical tags, take a look into the source code of one of the pages to see which of the above patterns it matches. Once you know the format of the canonical version of the homepage go ahead and verify it in Google Search Console.

For the example case let’s assume that the canonical version of the homepage is https://www.sitename.com/

STEP 2: Crawl the site with Screaming Frog.

Open the Screaming Frog application, and set it in the Spider mode:

Before starting the crawl, go to Configuration > API Access > Google Search Console, connect to your account, and select the Google Search Console property of the URL you verified in the previous step. In the same window, navigate to the General tab, and check the “Crawl New URLs Discovered in Google Search Console” box:

With the above box checked, Screaming Frog will discover and crawl the site’s landing pages that appeared in Google Search Results for the period of time selected. Speaking of which… In the “Date Range” tab, select the last 6 months, any further than that and the data won’t be as relevant:

Click OK, and start the crawl of your site.

STEP 3: Export the CSV and Manipulate the Data

Once Screaming Frog has finished crawling the site, export the contents of the Internal HTML tab as a CSV:

Open the CSV and set these filters to the following columns.

The “Status Code” column should only show URLs that returned 404:

The filter of the “InLinks” column should show only those URLs with no internal links:

Finally, and most importantly, the “Clicks” column of the CSV should be set to anything but 0 and (Blanks), as shown in the following screenshot:

The Result. You will end up with a list of valuable URLs that are used to drive clicks to your site from Google Search results, that are harming you because they are currently returning a 404 error. The next step from here would be to set each of those 404 URLs to 301-redirect to the most relevant valid (200 OK) pages of your site.

Now that you’re done, let’s unpack what we did on a more conceptual level. These 404 landing pages, at one point, used to drive traffic to your site. They were valuable to your search equity, but for whatever reason, the pages no longer exist. However, their search equity did cease to exist, and is still somewhere there, so by setting them to point to valid 200 OK pages we can pass that search equity back to the site.

A landing page that drove traffic in the past is likely to have inbound links from other external sites. So these pages are pretty valuable, and we want to find them and make them point elsewhere where it makes sense in your funnel.

Of course, like anything else in the digital world, there are other, more comprehensive ways of fixing this issue, but this shortcut works great for those who would rather concentrate on managing their inventory and creating new products instead of needless time spend in the wormhole of the Google Popularity contest.

Would you like to outsource your SEO? We are here to help.