4xx errors are frustrating for website visitors as they occur when a webpage is inaccessible for various reasons. The four types of 4xx errors are the 400 error, 401 error, 403 error, and the well-known 404 error. Each error has a specific cause and can be resolved differently.
Types of 4xx Errors and their Causes
The 400 error, or bad request error, happens when a user tries to access a server location with certain restrictions. On the other hand, the 401 error, or unauthorized error, occurs when users provide incorrect credentials or invalid login details. The 403 error, or forbidden error, arises when a particular webpage is not allowed to get a request accepted from the user or the client server. The 404 error, or not found error, appears when a user attempts to access a broken link or a non-existing page on the website.
Impact of 404 Errors on SEO
The 404 error can be detrimental to a website’s SEO. When users encounter this error, it leads to negative user experience, which can lead to negative SEO. Google’s database may identify that the website is not providing an optimal experience, and it can negatively affect the website’s SEO ranking. Fixing 404 errors is crucial to avoid draining the website’s SEO value.
How to Fix Blocked due to other 4xx Errors
If you encounter the “Blocked due to other 4xx issue” status, it could be due to a client-side error that may not be reproducible in your browser. Nonetheless, this issue may still affect some of your website visitors, making it crucial to address. Here is a step-by-step guide to help you resolve the problem.
Analyze and fix “Blocked due to other 4xx issue” using Google Search Console
To analyze the “Blocked due to other 4xx issue” URLs, access the Page indexing (Index Coverage) report in Google Search Console and navigate to the status page. Check the number of affected pages to understand the extent of the problem and look for patterns in the URL structure to analyze them further.
Filter out the URLs for a given path by clicking on the inverted pyramid symbol, and then use the URL Inspection tool to research each of the affected pages. However, if Google cannot define a referring page or sitemap, consult your developers for possible reasons behind the issue as they know your website’s history and files dependency.
Check for “Submitted URL Blocked due to other 4xx issue”
Checking for “Submitted URL Blocked due to other 4xx issue” is an essential part of monitoring your website’s health in Google Search Console. Although this status was previously a separate report, you can now filter out all affected URLs to “All submitted URLs” in the upper left corner of the report to look for URLs added to your sitemaps.
To ensure that your sitemap files only include pages that respond with a 200 status code, you must remove any affected pages as a priority. Even if you deliberately made some of your pages respond with “Blocked due to other 4xx issue,” like “Gone (410),” adding these pages to your sitemap could waste your crawl budget.
Moreover, if you are managing a large website, you should perform a full crawl of your sitemap to ensure that you get a complete picture of what is happening with your submitted pages. Keep in mind that the Page indexing report shows only a sample of 1000 pages, which may not be enough to understand the whole problem.
Analyzing Internal Linking
Properly monitoring internal linking is crucial for SEO, particularly in the case of “Blocked due to other 4xx issue” pages. While Google states that pages serving 4xx HTTP status codes (except 429) don’t waste crawl budget, having many internal links pointing to 4xx pages can impact how Google crawls your website. This is because Googlebot may interpret these pages as important, leading to wasted resources in trying to access content that it can’t see or access.
Checking internal linking to 4xx pages is especially crucial if some of these pages, such as “Gone (410),” are expected to remain on your site for the long-term. If you can’t track the referring pages in the URL Inspection tool, a full crawl of your website using an SEO crawler like Screaming Frog or Deepcrawl can be useful.
To troubleshoot improper internal linking, you can either remove broken links to 4xx pages (often 404 or 400 pages) or replace the links from 4xx pages with related pages that respond with a 200 status code. However, if you want to keep the internal linking to these pages intact, there are other tactics you can use to address the “Blocked due to other 4xx issue” pages.
When assessing the scale of the problem and deciding on the next steps, you have two options. You can ignore “Blocked due to other 4xx issue” if there are only a few issues that don’t significantly affect your website and don’t require immediate attention or if it’s caused by your deliberate actions, such as setting up the 410 HTTP status code. However, if the number of affected pages is significant or constantly growing, and you don’t know how to approach these issues or what further SEO problems they may contribute to, a thorough analysis of the “Blocked due to other 4xx issue” pages is necessary.
By properly analyzing internal linking and addressing “Blocked due to other 4xx issue” pages, you can improve your website’s crawlability and avoid wasting resources while prioritizing urgent SEO issues.
Identify the Response Code Causing “Blocked Due to Other 4xx Issue”
To address the “Blocked due to other 4xx issue” on your website, you must first identify the response codes triggering the error. Unfortunately, Google Search Console does not provide this information, but you can use Chrome DevTools to simulate the Googlebot user agent and check the status code returned by each page.
Simply open the ‘Network’ tab, go to the ‘Network conditions’ tab, unclick ‘Use browser default’ in the User-agent section, and select ‘Googlebot Smartphone’ from the drop-down list. Alternatively, you can use the Link Redirect Trace tool or analyze your server logs.
Once you have identified the specific response code, you can take appropriate action to address it, such as correcting URL spelling mistakes for the “Bad request (400)” error or modifying the .htaccess file for “Blocked due to access forbidden (403).”
Redirecting “Blocked due to other 4xx issue” Pages
Redirecting your page using a 301 redirect can be a good practice if you cannot fix a specific issue that you’re struggling with, especially when the affected URL is an essential part of your website structure, valuable for your business, and has many backlinks pointing to it.
To redirect your pages in an SEO-friendly way, you can read our guides to redirects and troubleshooting the “Redirect error” issues in Google Search Console.
However, keep in mind that sometimes the data for “Blocked due to other 4xx issue” in Google Search Console may not be accurate and up-to-date. Therefore, always double-check the current HTTP response codes for your affected URLs before taking action.
Changing “Blocked Due to Other 4xx Issue” Pages to 404 or 410
If you can’t fix a specific 4xx error or a “Blocked due to other 4xx issue” page no longer serves your website’s strategy, changing the status to 404 or 410 may be a solution. After Googlebot knows that a page doesn’t exist, it will limit its crawling to a minimum, and a user-friendly 404 page can enhance user experience.
To approach “Blocked due to other 4xx issue” properly, refine your website’s technical SEO strategy to avoid uncontrolled chaos. Though these issues rarely cause significant SEO problems as random individual cases, they may lead to bigger problems on a larger scale. Therefore, carefully decide your SEO priorities when managing a large website.
Preventing 4xx Errors
To prevent 4xx errors, website owners can check the server access log for unauthorized access or forbidden errors. They can also use various tools such as SEMrush, Screaming Frog, and other marketplace tools to identify and rectify pages that generate 404 errors or do not exist. Finally, creating a custom 404 landing page that provides helpful information to the users can prevent 4xx errors and improve the website’s overall user experience.
Conclusion
In summary, understanding the different types of 4xx errors, their causes, and their impact on SEO is essential for website owners. By implementing preventive measures and fixing 404 errors, website owners can improve the website’s user experience, prevent negative SEO effects, and maintain their website’s SEO value.
FAQ
What is “Blocked due to other 4xx” ?
It means that the server responded with a 4xx status code, indicating that access to the page/URL/resource you were requesting has been blocked.
What are the common 4xx status codes?
The most common 4xx status codes include:
- 400: Bad Request
- 401: Unauthorized
- 403: Forbidden
- 404: Not Found
- 429: Too Many Requests
What might cause Blocked due to other 4xx?
The “Blocked due to other 4xx” error can be caused by a variety of different things, including but not limited to:
- Invalid user credentials
- Invalid API keys or tokens
- Not having the right permissions to access the resource
- Server-side issues such as resource not found
How can it be fixed?
The first step should be to identify what is causing the “Blocked due to other 4xx” error. This could involve examining server logs, or checking any API keys, credentials, or user permissions. Once the cause is identified, the fix may involve replacing or updating credentials, or re-configuring the server.
Is the “Blocked due to other 4xx” error always fixable?
Not necessarily. If the cause of the error is a coding bug on the server side, then the issue might be trickier to diagnose and fix. In some cases, the error may also be caused by an issue with the client’s browser, in which case the client may need to update or reconfigure the browser.