404 –Not Found is an error message that many Internet users have most likely come across at least once during their Internet browsing history. Most likely this has come in the form of a 404 error page.
But what exactly does it mean?
404 error is an HTTP status code that indicates that the page you were attempting to reach on the website cannot be reached on their server. This error often occurs when you mistype the URL into a browser, or when you follow a broken link from another website. It may also happen simply because the page was either renamed or deleted.
How Is My Page Effected?
It’s important to identify these 404 error pages as soon as possible because it can indirectly have negative effects on your organic rankings. (Organic traffic is generated from users when they click on your result in search engines, as opposed to paid advertisements. The more views, clicks, and interactions that your website receives from users, the better the rankings that it will receive!)
Google recognizes that the pages are no longer available and removes them from their search index, meaning that they will no longer appear on Google Search Engine Result Page (SERP). It also voids any link equity that those broken links acquired in PageRank, deeming the links useless.
Websites with numerous broken links results can also result in poor user experience. This is because users aren’t as keen to use sites that are unable to provide information in an efficient manner. It signifies poor site quality and enables users to lose confidence in your site/brand. Ultimately, this will result in lower rankings on search engines.
Evidently, 404 error pages can be detrimental to your website’s SEO. Continue reading to learn how to quickly identify these broken pages and how to fix them to best optimize your website traffic.
How to Quickly Identify Broken Pages
Step 1: Log in to Google Webmaster Tool. On the sidebar, go to Behavior > Site Content > Landing Pages. It will then display all of the traffic generated from each landing page.
Next, you would want to adjust the time period on the upper right-hand corner to encompass the entire site history vs. the auto-generated 30-day time period. Viewing the entire site history traffic since its birth will allow you to better identify and remedy the error pages.
Click Custom and adjust the date. (I adjusted the date to go as far back as July 2015 to present day. If you’re unsure of when your website had first began, continue playing with the timeline until you reach the beginning start date of the website and then click Apply.)
Now, you want to filter by organic traffic. Click Add Segment, check Organic Traffic, and click Apply. You will now see the data solely from Organic Traffic. Next, you want to export the data into Excel.
Step 2: Export the data into Excel. Before you do so, you will need to adjust some of the filters. At the bottom of the page, it will display the number of results that organic traffic generated. Next to Show rows, adjust the number to encompass the entire data set. For example, considering we have 1339 results, we can adjust it to 2500. Once the data loads, underneath Landing Pages, click Export and choose Excel (XLSX).
Your Excel data sheet will look similar to this, where it outlines the landing pages URL, the number of sessions, new users, etc.
Step 3: Input your links into Screaming Frog to identify any broken links. Your Excel data sheet will look similar to this, where it outlines the landing pages URL, the number of sessions, new users, etc.
Now you will go into Screaming Frog. At the top of the application, click Mode > List. Feed the URLs into the application by clicking Upload> Paste or Enter Manually. Remember, the URLs have to begin with http:// or https:// in order for Screaming Frog to accurately crawl the links.
(Hint: Use formula CONCATENATE in Excel to include the https:// or http:// at the beginning of the link)
Once it is 100% completed, go to the Response Codes tab, filter by Client Errors (4xx) > Export.
You now have a list of all of the page URLs that have a 404-Not Found error message!
Step 4: Identify how much traffic has been brought upon these 404 pages over the years.
You will now need to cross-reference the 404 Response Code excel sheet with the Landing Page Summary Excel Sheet by creating a VLOOKUP formula.
(Note: Ensure that the URLs mirror each other on both excel sheets. For example, https://databricks.com/spark/developer-resources/ on the 404 Response Code sheet will not be able to match with /spark/developer-resources/ on the Landing Page Summary sheet.)
What to Do with Your 404 Error Page(s)
Now that you have identified your error pages, what should you do with it? There are several actions that you can take.
Restore Old Pages
For the pages that consist of outdated or old content, consider recreating and updating them, especially if they have a history or generating high traffic onto your website. Old content provides you an opportunity to connect with users and continue generating a good user experience by directing to them to other more updated pages on your website.
301 Redirects
For content that has been moved or rewritten as new content, 301 redirects are often used to guide its users to the correct location. A 301 redirect is a permanent redirect that points the users from one URL to another. It allows the browser to understand that the intended visited page has “moved permanently” to a new location. 301 redirects are beneficial to your website because you are able to salvage the existing link equity (ranking power) from the old URL to the redirected page. Keep in mind that it’s important to redirect users to a page with relevant content related to the original intended link, and not to a page that you would prefer to generate higher traffic.
Marketers should also be aware of the differences between using 301 and 302 redirects. A 302 redirect is a temporary redirect. Because it is considered temporary, search engines will continue indexing the old URL while correspondingly indexing the new one. This can result in the search engine reading it as duplicate pages. It runs the risk of having the old URL presented solely in the search engine results, omitting the new URL. Link equity plays an enormous factor as well, in which it would be divided between the old and new URLs. This ultimately affects the search rankings for both pages. In short, the best occasions to use 302 redirects is when you are updating a web page while consistently providing users with a positive site experience. Do this when gathering client feedback on a new page without impacting the site ranking, or when conducting A/B testing.
Inability to properly redirect pages to the updated/related URL can result in a soft 404. A soft 404 occurs when a user visits a page that is no longer available (either it has been moved or deleted) but it fails to return a HTTP 404 status code. It can also occur when the page is redirected to an unrelated link. Google compares it to “a giraffe wearing a name tag that says ‘dog.’ Just because it says it’s a dog, doesn’t mean it’s actually a dog. The content of the page is entirely unrelated to the HTTP response returned by the server.” Search engines can view it as a real page and may crawl and have the content indexed, impacting your rankings.
Customize Your 404 Page
If you want to altogether remove a page on your website, you can do so but you should customize your 404 error page. Oftentimes, visitors do not know why a 404 error page exists or how to get around it. You can customize the 404 error page to explain to users that the page no longer exists. Then, you can use it to direct them to other areas within your site. The options can include the Home page, a popular product page, or a page that has related content. You can also include a Site Map to allow users to better navigate your website. Additionally, you can use a search box to allow users to directly search for the content they wanted.
As content continues being generated and websites expand, it is important to consistently track your 404 error pages. This protects your organic search rankings and mitigate the negative effects that it has on your SEO. Good luck!
