Magento – There was a problem with reindexing process after Magento update

indexproduct-pricesreindexshell

I have read several topics about this, but none if these topics seem to help me. I'm a beginner when it comes to Magento, so most of these topics don't really say much to me. I recently updated from Magento 1.6.2 to 1.9.2.1 and I noticed the following message:

One or more of the Indexes are not up to date: Product Prices. Click here to go to Index Management and rebuild required indexes.

Naturally, I went to Index Management and tried to reindex the only index that required reindexing: Product Prices.
After doing so, it showed this error: There was a problem with reindexing process

I've tried emptying the cache and locks folder and I also tried reindexing through shell with the command php shell/indexer.php -reindexall, which gave me the following errors below. I have no idea what to do, since I don't understand these errors.

Product Attributes index was rebuilt successfully in 00:00:01
Product Prices index process unknown error:
exception 'PDOException' with message 'SQLSTATE[21S01]: Insert value list does not match column list: 1136 Column count doesn't match value count at row 1' in /home/users/spovnftp/sportinngerritsen.nl/lib/Zend/Db/Statement/Pdo.php:228
Stack trace:
#0 /home/users/spovnftp/sportinngerritsen.nl/lib/Zend/Db/Statement/Pdo.php(228): PDOStatement->execute(Array)
#1 /home/users/spovnftp/sportinngerritsen.nl/lib/Varien/Db/Statement/Pdo/Mysql.php(110): Zend_Db_Statement_Pdo->_execute(Array)
#2 /home/users/spovnftp/sportinngerritsen.nl/app/code/core/Zend/Db/Statement.php(291): Varien_Db_Statement_Pdo_Mysql->_execute(Array)
#3 /home/users/spovnftp/sportinngerritsen.nl/lib/Zend/Db/Adapter/Abstract.php(480): Zend_Db_Statement->execute(Array)
#4 /home/users/spovnftp/sportinngerritsen.nl/lib/Zend/Db/Adapter/Pdo/Abstract.php(238): Zend_Db_Adapter_Abstract->query('INSERT INTO `ca...', Array)
#5 /home/users/spovnftp/sportinngerritsen.nl/lib/Varien/Db/Adapter/Pdo/Mysql.php(504): Zend_Db_Adapter_Pdo_Abstract->query('INSERT INTO `ca...', Array)
#6 /home/users/spovnftp/sportinngerritsen.nl/app/code/community/OrganicInternet/SimpleConfigurableProducts/Catalog/Model/Resource/Eav/Mysql4/Product/Indexer/Price/Configurable.php(133): Varien_Db_Adapter_Pdo_Mysql->query('INSERT INTO `ca...')
#7 /home/users/spovnftp/sportinngerritsen.nl/app/code/core/Mage/Catalog/Model/Resource/Product/Indexer/Price/Configurable.php(48): OrganicInternet_SimpleConfigurableProducts_Catalog_Model_Resource_Eav_Mysql4_Product_Indexer_Price_Configurable->_prepareFinalPriceData()
#8 /home/users/spovnftp/sportinngerritsen.nl/app/code/core/Mage/Catalog/Model/Resource/Product/Indexer/Price.php(385): Mage_Catalog_Model_Resource_Product_Indexer_Price_Configurable->reindexAll()
#9 /home/users/spovnftp/sportinngerritsen.nl/app/code/core/Mage/Index/Model/Indexer/Abstract.php(143): Mage_Catalog_Model_Resource_Product_Indexer_Price->reindexAll()
#10 /home/users/spovnftp/sportinngerritsen.nl/app/code/core/Mage/Index/Model/Process.php(212): Mage_Index_Model_Indexer_Abstract->reindexAll()
#11 /home/users/spovnftp/sportinngerritsen.nl/app/code/core/Mage/Index/Model/Process.php(260): Mage_Index_Model_Process->reindexAll()
#12 /home/users/spovnftp/sportinngerritsen.nl/shell/indexer.php(168): Mage_Index_Model_Process->reindexEverything()
#13 /home/users/spovnftp/sportinngerritsen.nl/shell/indexer.php(216): Mage_Shell_Compiler->run()
#14 {main}

Next exception 'Zend_Db_Statement_Exception' with message 'SQLSTATE[21S01]: Insert value list does not match column list: 1136 Column count doesn't match value count at row 1, query was: INSERT INTO `catalog_product_index_price_final_idx` SELECT `inner`.`entity_id`, `inner`.`customer_group_id`, `inner`.`website_id`, `inner`.`tax_class_id`, `inner`.`orig_price`, `inner`.`price`, `inner`.`min_price`, MAX(inner.max_price) AS `max_price`, `inner`.`tier_price`, `inner`.`base_tier` FROM (SELECT e.entity_id AS `entity_id`, pi.customer_group_id AS `customer_group_id`, cw.website_id AS `website_id`, pi.tax_class_id AS `tax_class_id`, pi.price AS `orig_price`, pi.final_price AS `price`, pi.final_price AS `min_price`, pi.final_price AS `max_price`, pi.tier_price AS `tier_price`, pi.tier_price AS `base_tier` FROM `catalog_product_entity` AS `e`
 LEFT JOIN `catalog_product_super_link` AS `l` ON l.parent_id = e.entity_id
 INNER JOIN `catalog_product_entity` AS `ce` ON ce.entity_id = l.product_id
 INNER JOIN `catalog_product_index_price_idx` AS `pi` ON ce.entity_id = pi.entity_id
 INNER JOIN `core_website` AS `cw` ON pi.website_id = cw.website_id
 INNER JOIN `core_store_group` AS `csg` ON csg.website_id = cw.website_id AND cw.default_group_id = csg.group_id
 INNER JOIN `core_store` AS `cs` ON csg.default_store_id = cs.store_id AND cs.store_id != 0
 CROSS JOIN `cataloginventory_stock` AS `cis`
 LEFT JOIN `cataloginventory_stock_item` AS `cisi` ON cisi.stock_id = cis.stock_id AND cisi.product_id = ce.entity_id
 LEFT JOIN `catalog_product_entity_int` AS `tad_status` ON tad_status.entity_id = ce.entity_id AND tad_status.attribute_id = 84 AND tad_status.store_id = 0
 LEFT JOIN `catalog_product_entity_int` AS `tas_status` ON tas_status.entity_id = ce.entity_id AND tas_status.attribute_id = 84 AND tas_status.store_id = cs.store_id WHERE (e.type_id='configurable') ORDER BY IF(IF(cisi.use_config_manage_stock = 0 AND cisi.manage_stock = 1,cisi.is_in_stock, 1), 1, 0) DESC, pi.final_price ASC, pi.price ASC) AS `inner` GROUP BY `inner`.`entity_id`,
        `inner`.`customer_group_id`,
        `inner`.`website_id` ON DUPLICATE KEY UPDATE `tax_class_id` = VALUES(`tax_class_id`), `orig_price` = VALUES(`orig_price`), `price` = VALUES(`price`), `min_price` = VALUES(`min_price`), `max_price` = VALUES(`max_price`), `tier_price` = VALUES(`tier_price`), `base_tier` = VALUES(`base_tier`), `group_price` = VALUES(`group_price`), `base_group_price` = VALUES(`base_group_price`)' in /home/users/spovnftp/sportinngerritsen.nl/lib/Zend/Db/Statement/Pdo.php:235
Stack trace:
#0 /home/users/spovnftp/sportinngerritsen.nl/lib/Varien/Db/Statement/Pdo/Mysql.php(110): Zend_Db_Statement_Pdo->_execute(Array)
#1 /home/users/spovnftp/sportinngerritsen.nl/app/code/core/Zend/Db/Statement.php(291): Varien_Db_Statement_Pdo_Mysql->_execute(Array)
#2 /home/users/spovnftp/sportinngerritsen.nl/lib/Zend/Db/Adapter/Abstract.php(480): Zend_Db_Statement->execute(Array)
#3 /home/users/spovnftp/sportinngerritsen.nl/lib/Zend/Db/Adapter/Pdo/Abstract.php(238): Zend_Db_Adapter_Abstract->query('INSERT INTO `ca...', Array)
#4 /home/users/spovnftp/sportinngerritsen.nl/lib/Varien/Db/Adapter/Pdo/Mysql.php(504): Zend_Db_Adapter_Pdo_Abstract->query('INSERT INTO `ca...', Array)
#5 /home/users/spovnftp/sportinngerritsen.nl/app/code/community/OrganicInternet/SimpleConfigurableProducts/Catalog/Model/Resource/Eav/Mysql4/Product/Indexer/Price/Configurable.php(133): Varien_Db_Adapter_Pdo_Mysql->query('INSERT INTO `ca...')
#6 /home/users/spovnftp/sportinngerritsen.nl/app/code/core/Mage/Catalog/Model/Resource/Product/Indexer/Price/Configurable.php(48): OrganicInternet_SimpleConfigurableProducts_Catalog_Model_Resource_Eav_Mysql4_Product_Indexer_Price_Configurable->_prepareFinalPriceData()
#7 /home/users/spovnftp/sportinngerritsen.nl/app/code/core/Mage/Catalog/Model/Resource/Product/Indexer/Price.php(385): Mage_Catalog_Model_Resource_Product_Indexer_Price_Configurable->reindexAll()
#8 /home/users/spovnftp/sportinngerritsen.nl/app/code/core/Mage/Index/Model/Indexer/Abstract.php(143): Mage_Catalog_Model_Resource_Product_Indexer_Price->reindexAll()
#9 /home/users/spovnftp/sportinngerritsen.nl/app/code/core/Mage/Index/Model/Process.php(212): Mage_Index_Model_Indexer_Abstract->reindexAll()
#10 /home/users/spovnftp/sportinngerritsen.nl/app/code/core/Mage/Index/Model/Process.php(260): Mage_Index_Model_Process->reindexAll()
#11 /home/users/spovnftp/sportinngerritsen.nl/shell/indexer.php(168): Mage_Index_Model_Process->reindexEverything()
#12 /home/users/spovnftp/sportinngerritsen.nl/shell/indexer.php(216): Mage_Shell_Compiler->run()
#13 {main}
Catalog URL Rewrites index was rebuilt successfully in 00:00:21
Category Flat Data index was rebuilt successfully in 00:00:00
Category Products index was rebuilt successfully in 00:00:00
Warning: cannot open /proc/net/dev (No such file or directory). Limited output.
Warning: cannot open /proc/net/dev (No such file or directory). Limited output.
Catalog Search Index index was rebuilt successfully in 00:00:07
Stock Status index was rebuilt successfully in 00:00:00
Tag Aggregation Data index was rebuilt successfully in 00:00:00
Default Values (MANAdev) index was rebuilt successfully in 00:00:00
Lucene Search Index index was rebuilt successfully in 00:00:00

Best Answer

It really looks like your upgrade didn't fully run. It could just be that index table, but it's better to run the following (to be on the safe side).

Do this on a dev environment first, and make sure you do files/db backups

First thing you should do, is replace the app/code/core with a fresh install (make sure you don't have any local override in app/code/local/Mage). Same applies for /lib.

Then run the Magento Repair Tool to check your database: http://www.magikcommerce.com/blog/how-to-repair-a-database-with-magento-repair-tool/

And fix any issue that you can spot.

Related Solutions

Magento – Magento 1.8 reindex all

The catalog_product_index_eav table has a foreign key contraint named

FK_CAT_PRD_IDX_EAV_ENTT_ID_CAT_PRD_ENTT_ENTT_ID

Looking at this table's definition

CREATE TABLE `catalog_product_index_eav` (
  `entity_id` int(10) unsigned NOT NULL DEFAULT '0' COMMENT 'Entity ID',
  `attribute_id` smallint(5) unsigned NOT NULL DEFAULT '0' COMMENT 'Attribute ID',
  `store_id` smallint(5) unsigned NOT NULL DEFAULT '0' COMMENT 'Store ID',
  `value` int(10) unsigned NOT NULL DEFAULT '0' COMMENT 'Value',
  PRIMARY KEY (`entity_id`,`attribute_id`,`store_id`,`value`),
  KEY `IDX_CATALOG_PRODUCT_INDEX_EAV_ENTITY_ID` (`entity_id`),
  KEY `IDX_CATALOG_PRODUCT_INDEX_EAV_ATTRIBUTE_ID` (`attribute_id`),
  KEY `IDX_CATALOG_PRODUCT_INDEX_EAV_STORE_ID` (`store_id`),
  KEY `IDX_CATALOG_PRODUCT_INDEX_EAV_VALUE` (`value`),
  CONSTRAINT `FK_CATALOG_PRODUCT_INDEX_EAV_STORE_ID_CORE_STORE_STORE_ID` FOREIGN KEY (`store_id`) REFERENCES `core_store` (`store_id`) ON DELETE CASCADE ON UPDATE CASCADE,
  CONSTRAINT `FK_CAT_PRD_IDX_EAV_ATTR_ID_EAV_ATTR_ATTR_ID` FOREIGN KEY (`attribute_id`) REFERENCES `eav_attribute` (`attribute_id`) ON DELETE CASCADE ON UPDATE CASCADE,
  CONSTRAINT `FK_CAT_PRD_IDX_EAV_ENTT_ID_CAT_PRD_ENTT_ENTT_ID` FOREIGN KEY (`entity_id`) REFERENCES `catalog_product_entity` (`entity_id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='Catalog Product EAV Index Table';

we can see the foreign key definition is

CONSTRAINT `FK_CAT_PRD_IDX_EAV_ENTT_ID_CAT_PRD_ENTT_ENTT_ID` 
FOREIGN KEY (`entity_id`) 
REFERENCES `catalog_product_entity` (`entity_id`) 
ON DELETE CASCADE ON UPDATE CASCADE

This means for every entity_id row in catalog_product_index_eav, there needs to be an identical, corresponding entity_id value in catalog_product_entity.

The root of your problem is for some reason (either a rouge extension, errors caused by randomly typing in SQL from the internet, or performing data updates with disabled indexes), Magento's indexing attempts to update data in catalog_product_index_eav that violates this rule. The next step is identifying what Magento's doing so you can fix the data.

If we look at your call stack, this looks like a good place to start debugging

#10 app/code/core/Mage/Catalog/Model/Resource/Product/Indexer/Eav/Abstract.php(54): 
Mage_Index_Model_Resource_Abstract->syncData()

Jumping to that source file, we see the following bit of code

public function syncData()
{
    $this->beginTransaction();
    try {
        /**
         * Can't use truncate because of transaction
         */
        $this->_getWriteAdapter()->delete($this->getMainTable());
        $this->insertFromTable($this->getIdxTable(), $this->getMainTable(), false);
        $this->commit();
    } catch (Exception $e) {
        $this->rollBack();
        throw $e;
    }
    return $this;
}

As part of its indexing process, Magento tries to sync data from an "index table" (getIdxTable), to a "source" (getMainTable) table.

public function insertFromTable($sourceTable, $destTable, $readToIndex = true)
{
    //...
}

For this particular index, the index table is catalog_product_index_eav_idx, and the source table is catalog_product_index_eav.

Note: Be careful with your terminology around here, things are confusingly named. The "Source" table is the table we're copying to. (I believe it's called the source table because it's the "source" a normal Magento system will query from when it needs information)

So, Magento is trying to sync a row from catalog_product_index_eav_idx to the table catalog_product_index_eav. However, this causes the previously mentioned foreign key error. This leads us to two possible conclusions

The catalog_product_index_eav_idx has entity_id rows that do not exist in catalog_product_entity.
The catalog_product_index_eav table has (through previous manipulation with index checks turned off) entity_id rows that do not exist in catalog_product_entity.

So, your mission here is to figure out which entity_id rows in catalog_product_index_eav and catalog_product_index_eav_idx don't exist in catalog_product_entity, and manually delete said rows (from catalog_product_index_eav and catalog_product_index_eav_idx).

If it were me, and my catalog_product_entity table wasn't too large, I'd start with the following queries (these are untested, as I don't have any Magento tables with the above invalid data states)

SELECT * 
FROM catalog_product_index_eav_idx 
WHERE NOT (entity_id IN (SELECT entity_id from catalog_product_entity));

SELECT * FROM catalog_product_index_eav 
WHERE NOT (entity_id IN (SELECT entity_id from catalog_product_entity));

Good luck!

Magento – Index Management Problem

your problem is the "a foreign key constraint fails" message. It means your database is damaged and you have references to entries, which dont exist anymore.

This can happen by misconfigured databases or usage of phpmyadmin for database export.

Caution, to solve this problem some knowledge is needed. Because if you make an error, you can damage your database very strong.

Make a complete Backup of the database
use a database tool of your choice
do a truncate on the catalog_product_flat_1 table
Try the reindex again

Best Answer

Related Solutions

Magento – Magento 1.8 reindex all

Magento – Index Management Problem

Related Topic