I've managed to stabalize the issue as follows:
Step 1: Rewrite the Catalog URL model (Using your own module: How To)
Note: If you overwrite the core file without using a rewrite this will
render your instance of Magento incapable of future upgrades.
As per Jahnni's solution on the MagentoCommerce boards (no longer active with new board), app/code/core/Mage/Catalog/Model/Url.php [ around line 807 Mage_Catalog_Model_Url::getProductRequestPath() ]
From:
if ($product->getUrlKey() == '' && !empty($requestPath)
&& strpos($existingRequestPath, $requestPath) === 0
)
To:
if (!empty($requestPath)
&& strpos($existingRequestPath, $requestPath) === 0
)
Step 2: Truncate
Truncate the core_url_rewrite table
Step 3: Reindex & Flush Caches
Initiate the re-indexing process on Core URL Rewrites.
Thereafter, you'll want to flush the Magento cache & storage cache.
System → Cache Management → Flush Magento Cache
System → Cache Management → Flush Cache Storage
Voila, you're all set. You'll notice if you re-run the indexer, the table should stay constant in size (unless you've added more products inbetween or if you have duplicate category names).
Turns out it was an external job running from cron. We created a script to remove special pricing from products. The problem was that the script was not loading the entire product object. When it removed the pricing and saved the product the url_key was removed as well.
The job grabs a collection of all the products with expired special pricing. First I removed the addAttributeToSelect from the collection, but that didn't work and I'm not fully sure why. So my end solution was to loop through each product in the collection and use the id to load the individual product again, makes the changes and save.
If anyone comes upon this keep in mind that it comes down to a product being modified and saved so look into all the places you are doing that in your custom modules or scripts.
Best Answer
Disabling URL rewriting in the backend refers to something else, namely web server rewrites. This means, without them, URLs are in the form
instead of
It is not intended to disable the catalog rewrite feature, but if you want to prevent the index from being written, you can rewrite the model
catalog/indexer_urland overridereindexAll()and_processEvent(Mage_Index_Model_Event $event)with empty methods.