Magento – Google Sitemap generates errors

magento-1.9sitemaps

When I manually Generate sitmap.xml from admin's Catalog>Google Sitemap, it throws the following error in the browser:

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, webmaster@TheDomain.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.

Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request.

Apache error_log

[Fri Dec 25 10:15:12 2015] [error] [client HostIP] client denied by server configuration: /home/DIRECTORY/public_html/MAGENTO/app/etc/local.xml  
[Fri Dec 25 10:26:07 2015] [error] [client MyIP] Script timed out before returning headers: index.php, referer: https://TheDomain/sitemap/index/key/e6...22/  
[Fri Dec 25 10:31:35 2015] [error] [client MyIP] Script timed out before returning headers: index.php, referer: https://TheDomain/cms_page/index/key/56...9c/  
[Fri Dec 25 10:41:07 2015] [error] [client MyIP] Script timed out before returning headers: index.php, referer: https://TheDomain/sitemap/index/key/e6...22/  
[Fri Dec 25 10:46:35 2015] [error] [client MyIP] Script timed out before returning headers: index.php, referer: https://TheDomain/cms_page/index/key/56...9c/  
[Fri Dec 25 10:47:40 2015] [error] [client HostIP] client denied by server configuration: /home/DIRECTORY/public_html/MAGENTO/app/etc/local.xml  
[Fri Dec 25 10:47:52 2015] [error] [client HostIP] client denied by server configuration: /home/DIRECTORY/public_html/MAGENTO/app/etc/local.xml  
[Fri Dec 25 11:13:04 2015] [notice] SIGUSR1 received.  Doing graceful restart    
[Fri Dec 25 11:13:05 2015] [notice] Digest: generating secret for digest authentication ...  
[Fri Dec 25 11:13:05 2015] [notice] Digest: done  
[Fri Dec 25 11:13:06 2015] [warn] RSA server certificate CommonName (CN) `www.DOMAIN.com' does NOT match server name!?  
[Fri Dec 25 11:13:06 2015] [notice] Apache/2.2.26 (Unix) mod_ssl/2.2.26 OpenSSL/1.0.1e-fips DAV/2 mod_bwlimited/1.4 configured -- resuming normal operations

This could be a browser time out issue!

Can this command be executed from shell ssh?

Best Answer

Ok, seems to be a timeout problem, but could be a write permissions problem too, so just to be sure:

Try to switch to webserver user ie: $ su www-data
Then try to write a file in magento root folder (as same level as index.php) ie: $ touch [magento_root]/test-file

If these validation steps are ok, try to change the php timeouts (you can change these values just for admin request) here you will find a guide about changing php timeouts and other limitations.

However you would be able to perform sitemap.xml generation automatically if you have a system cronjob configured and pointed to magento cron.php/sh file, and properly configured in backend:

Related Solutions

Magento – Magento Google sitemap Generation

To address one of your initial comments:

"I have set Product URL Suffix(html) and Category URL Suffix(html) in the configuration.

This suffix are not showing in sitemap.xml."

We noticed this bug introduced in EE 1.13. We requested a patch and received the following. I don't believe this is EE specific so hopefully it helps a bit. Also, just upgraded a EE 1.12 site to 1.14 and noticed all the .html extension were dropped from sitemap.xml as well. Yes, having .html is old school and really not of any seo value these days but this bug can really suck if it goes unnoticed.

Warning ... I did not look at the referenced file in community. Use this as a guide, do not try to apply the patch blindly to your community store. Or if you do, at least test it very thoroughly before going live.

__PATCHFILE_FOLLOWS__
diff --git app/code/core/Mage/Sitemap/Model/Sitemap.php app/code/core/Mage/Sitemap/Model/Sitemap.php
index 2213a4f..734bb91 100644
--- app/code/core/Mage/Sitemap/Model/Sitemap.php
+++ app/code/core/Mage/Sitemap/Model/Sitemap.php
@@ -151,11 +151,13 @@ class Mage_Sitemap_Model_Sitemap extends Mage_Core_Model_Abstract
          */
         $changefreq = (string)Mage::getStoreConfig('sitemap/category/changefreq', $storeId);
         $priority   = (string)Mage::getStoreConfig('sitemap/category/priority', $storeId);
+        $urlSuffix  = Mage::helper('catalog/category')->getCategoryUrlSuffix($storeId);
+        $urlSuffix  = ($urlSuffix) ? '.' . $urlSuffix : '';
         $collection = Mage::getResourceModel('sitemap/catalog_category')->getCollection($storeId);
         foreach ($collection as $item) {
             $xml = sprintf(
                 '<url><loc>%s</loc><lastmod>%s</lastmod><changefreq>%s</changefreq><priority>%.1f</priority></url>',
-                htmlspecialchars($baseUrl . $item->getUrl()),
+                htmlspecialchars($baseUrl . $item->getUrl() . $urlSuffix),
                 $date,
                 $changefreq,
                 $priority
@@ -169,11 +171,13 @@ class Mage_Sitemap_Model_Sitemap extends Mage_Core_Model_Abstract
          */
         $changefreq = (string)Mage::getStoreConfig('sitemap/product/changefreq', $storeId);
         $priority   = (string)Mage::getStoreConfig('sitemap/product/priority', $storeId);
+        $urlSuffix  = Mage::helper('catalog/product')->getProductUrlSuffix($storeId);
+        $urlSuffix  = ($urlSuffix) ? '.' . $urlSuffix : '';
         $collection = Mage::getResourceModel('sitemap/catalog_product')->getCollection($storeId);
         foreach ($collection as $item) {
             $xml = sprintf(
                 '<url><loc>%s</loc><lastmod>%s</lastmod><changefreq>%s</changefreq><priority>%.1f</priority></url>',
-                htmlspecialchars($baseUrl . $item->getUrl()),
+                htmlspecialchars($baseUrl . $item->getUrl() . $urlSuffix),
                 $date,
                 $changefreq,
                 $priority

Magento – SQLSTATE[HY000] : General error: 2013 Lost connection to MySQL server during query in Magento

General error: 2006 MySQL server has gone away is a very generic error. It can be fired due to very different reasons. Usually, it happens with slow (until timeout) or very expensive queries

Looking your code trace, it seems it is just Magento default code (no extensions involved), so I bet your database is bigger than your MySQL resources can handle, or maybe your MySQL configuration is not correctly tuned to run a Magento application

In both cases, your hosting provider should give you an answer or recommendation

Best Answer

Related Solutions

Magento – Magento Google sitemap Generation

Magento – SQLSTATE[HY000] : General error: 2013 Lost connection to MySQL server during query in Magento

Related Topic