If you only want to update a single product attribute without running the indexer, use the following method. It will be possible to update a couple of hundred products per second that way.
First, add a new product attribute estimated_time_of_arrival.
The, in your update script:
$product->addAttributeUpdate(
'estimated_time_of_arrival',
$date,
$storeId // numeric store id value, e.g. Mage::app()->getStore()->getId()
);
If it's a global scope attribute, the $storeId doesn't even matter.
If you don't care about the store ID scope, you can also use:
$product->setData('estimated_time_of_arrival', $date)
->getResource()
->saveAttribute($product, 'estimated_time_of_arrival');
Both methods are equivalent (the latter will use $product->getStoreId()if the attribute has a website or store view scope).
Of course, if you use that attribute in some way that requires indexing, you will have to reindex after the updates.
Best Answer
Check which events set the indexes to invalid
Magento docs show the events for every index (see Events that Trigger Reindexing):
Further you can add some debug code in
Mage_Index_Model_Resource_Process::updateStatus()This should show you a stack trace about what is setting the indexes to invalid. In my case it were API requests of 'catalogProductUpdate'
You can also check the exception.log if you'll find any locks of the 'index_process' table. This tables holds the data you see in System > Index Management
Creds for the debug code go here