I have imported more than 1M of products for two stores. I am trying to indexing those products but it is taking long time through Admin and SSH. "Catalog URL Rewrites, Product Flat Data, Catalog Search Index" this are taking so much time to complete. Is there is any way that i can perform indexing on those products ?
Thanks in Advance.
Best Answer
Magento has a notoriously slow indexing engine that recently received a much-needed refresh in the Enterprise Edition 1.13 (though, at the time of this writing it is currently being overhauled for some issues with SEO).
In this answer, I have provided some direction on the types of issues you typically encounter when managing a site of this magnitude:
Is Magento the right platform for 1M products?
While your store may have 1M unique products, based on the number of pricing rules, stores, store views, tiered prices, etc., your indexes may have > 1M index entries. I have seen index processes run for 10+ hours. At this size, the flat catalog is not helpful, and in some documented cases, it actually negatively impacts performance.
At this scale, one would hope that you have the financial backing or annual volume to consider the Enterprise solution, as it addresses these indexing issues specifically. Otherwise, while it is possible to run a Community Edition store with a catalog of this size, all of your hardware needs scale horizontally -- meaning, you need larger and larger hardware to accomodate.
At this time, there are no fixes planned for indexing improvement in Community Edition.
Edit:
There are some community modules which tout that they improve index speed - I have no personal experience with any of them:
http://www.magentocommerce.com/magento-connect/asynchronous-reindex-4388.html
http://sourceforge.net/apps/mediawiki/magmi/index.php?title=Magmi_Magento_Reindexer
Ecomdev has a freely-available indexer specifically for UrlRewrites:
https://github.com/IvanChepurnyi/EcomDev_UrlRewrite