The time users spend searching for something on google is valuable and should not be squandered. Your website might be at the top of google for your intended search, but having soft 404s displayed can decrease the chance of users finding your quality content and unique pages won’t be crawled as frequently.
Whilst you might have a good looking website up front, it’s important to know the technical aspects of how google crawls your website.
Soft 404s are non-existent pages which have been crawled by google and are displaying codes other than 404 or 410s. These pages will have no content but are still displayed in search listings. These create a poor user experience as it eats into the possible pages google could be displaying. You can use google webmaster tools to help list all your soft 404s.
Ben Wood, the director of marketing services at hallam shows how fixing your 404 pages can increase search efficiency and positively affect your rank. You can fix soft 404s through 5 simple steps:
- Create an error page (404.html or 404.php) on the first level of the website (root-directory).
- Open the .htaccess file – or create one if you do not yet have one (also in the root-directory).
- Type in “ErrorDocument 404” followed by the relative path to the error page.
- Save and request a page that does not exist, like www.your-domain.com/98899351.
- You should now see the contents of the error page 404.html.
This will tell google not to crawl these pages as well as not index them. A helpful tool to test your pages is ‘Fetch as Google‘ to verify whether a URL is returning the correct code.
Update: [21/08/2017] Using google’s search console, I was able to locate of list of my soft 404s. Unfortunately due to my forum using a free hoster, I have restricted access to files in order to fix these (I have however notified the support staff).