Get page not found (404) for correct links

.htaccesscodeigniterhttp-status-code-404

I have a very strange issue: Google is giving me that some links in my sitemap are not accessible and are giving 404 errors. However, I have tried to open the pages in question and it works fine.

The website has been built using codeigniter and here is a sample of links

Now if you try to open the link above, it will work fine. but if you try to curl -I:

curl -I http://widwebway.com/en/solutions
HTTP/1.1 404 Not Found

I have made some routes and rewrite rules to exclude the controller name from the URL. In other words, the correct link is

curl -I http://widwebway.com/en/widwebway/view/solutions
HTTP/1.1 301 Moved Permanently
Date: Wed, 29 Feb 2012 18:05:01 GMT
Server: Apache/2.2.14 (Ubuntu)
Location: http://widwebway.com/en/solutions
Vary: Accept-Encoding
Content-Type: text/html; charset=iso-8859-1

Apache error log gives

File does not exist: /var/www/myweb/en, referer: http://widwebway.com/en/solutions

my .htaccess is:

ErrorDocument 404 /index.php
<IfModule mod_rewrite.c>
  RewriteEngine on
  RewriteRule ^ar/widwebway/view/([a-z_]{1,50})[/]*$           /ar/$1 [L,R=301,NC]
  RewriteRule ^en/widwebway/view/([a-z_]{1,50})[/]*$           /en/$1 [L,R=301,NC]
  RewriteRule ^en/widwebway[/]*$           /en/home [L,R=301,NC]
  RewriteRule ^ar/widwebway[/]*$           /ar/home [L,R=301,NC]
</IfModule>

Any idea of how comes that the page is there but it gives 404 on curl or Google bot?

Best Answer

Here's what I found: The browser is actually getting a 404 as well on http://widwebway.com/en/solutions

When I go to: http://widwebway.com/en/widwebway/view/solutions I get:

Request URL:http://widwebway.com/en/widwebway/view/solutions
Request Method:GET
Status Code:301 Moved Permanently

The move is directed to: http://widwebway.com/en/solutions This file /en/solutions doesn't actually exist, but with your rewrite rule you're telling the browser to display the contents of /en/widwebway/view/solutions when it sees /en/solutions

From Chrome Network Inspection:

Request URL:http://widwebway.com/en/solutions
Request Method:GET
Status Code:404 Not Found

Try using the PT flag instead. since the /en/solutions isn't a real file. This should get you away from the 404 errors Example from Apache docs

Alias /icons /usr/local/apache/icons
RewriteRule /pics/(.+)\.jpg /icons/$1.gif [PT]
Related Topic