Magento – Magento dataflow import (stop indexing on the fly)

csvimportindexingmagento-1.9reindex

There are, basically, two ways when importing product via csv file in Magento.
a) import
b) dataflow import

a) option, works faster, I assume that is because this option want index all the data on the fly, you have to do that after the import is done.

is there a way i can stop dataflow import to stop indexing on the fly. I think this will speed up the process of importation. Very important.

Best Answer

Try to set the indexes mode to Manual Update instead of Update on Save.

Also setIsMassupdate product flag is used in DataFlow import to prevent indexer to be triggered upon each product save method call.

Related Solutions

Magento – Products occasionally disappear

60,000 products * 40 store views = 2,400,000 products

I'm honestly not surprised you've got catalogue consistency issues. Magento doesn't scale well at all with large catalogues combined with large numbers of store views.

Your issue is almost entirely caused by an index not being complete, so your theories of 1, 1a and 1b are all correct.

We've seen this on a LOT of stores where the catalogues are large and re-indexing is performed.

The fixes for most of these stores was to optimise the indexing process for each index. We swapped MySQL out for SOLR - which gives an almost immediate reduction in population time. There are some other aspects to SOLR that serve to benefit you, such as the index not being dropped entirely until the previous process is committed; and that updates again don't remove data until committed.

Although, with 2.4 Million products - you're still going to struggle, unless you make some significant application-level changes.

My advice would be to take a proper look at why you are re-indexing, and at what frequency - and try and find some alternatives to this process that can perform in real-time.

I think you'll struggle to get a definitive answer on SE because of how tailored the answer needs to be to your store/business.

Magento – Why is the csv file not importing successfully in the Dataflow Profile even when it was verified as valid in Check Data

Please check and make sure that you don't have extra carriage returns/line feeds on the end of your csv file. Open the file, and delete any blank lines/rows.

Often times, when you export from Excel, it will add extra rows to the export that are nothing more than blank lines.

Best Answer

Related Solutions

Magento – Products occasionally disappear

Magento – Why is the csv file not importing successfully in the Dataflow Profile even when it was verified as valid in Check Data

Related Topic