Magento – How to do a solr search programmatically

So, as this gets a bit long in the comments and you'll anyway have to debug it on your own i'll write down the main problem of this error message and how you could continue to debug it.

/solr/update?wt=jsonstore_id:13600 looks for an input document to index. So calling this URL directly will always end up in this "missing content stream" error.

As already mentioned in the comments every request to Solr in Magento runs through Apache_Solr_Service::__sendRawPost in the lib directory. This is not Magento specific code, it's an implementation of the official PHP Solr Client published by Apache. You can also find some small docs over here: https://code.google.com/p/solr-php-client/

You mentioned that this position is called 3 times, so you should check when exactly the error code 400 is returned. I would recommend to use XDebug and check the response which is initialized also in that same method.

When the update action is called Magento should submit a list of documents (products on Magento site) which Solr has to index. You can also check the current index status with a tool called luke on your solr server: /solr/admin/luke/ This gives you information about the amount of indexed documents and terms and the latest update time, which helps a lot to debug this stuff and see if indexing was successful.

Besides that, what version of EE do you use? There are anyway some open bugs in 1.12 (I don't know if they fixed it in 1.13) and we provided some patches, which might help you to run in further problems: https://github.com/sitewards/SolrPatches

Sorry that I can't give you a direct solution, but with this information it should be possible to debug that stuff on your system and find the error pretty fast.

I'm running into this issue while upgrading from EE 1.13.0.2 to 1.14.1.0 right now. We experience this when bulk updating product attributes and stock in cronjobs. In 1.13, the jobs take ~3 seconds and ~90 seconds respectively. In 1.14, it's more like ~10 minutes and I-don't-want-to-know-how-long.

There is an EE patch PATCH_SUPEE-4945_EE_1.14.0.1_v2.sh regarding the slow product save. You can request it from the support.

Another tip I found was to only update the rows which are not set to 0 already (of course only change the core file temporarily to test if it has any effect for you):

diff --git a/app/code/core/Enterprise/CatalogSearch/Model/Index/Action/Fulltext/Refresh.php b/app/code/core/Enterprise/CatalogSearch/Model/Index/Action/Fulltext/Refresh.php
index c6273a1..95e6d4c 100644
--- a/app/code/core/Enterprise/CatalogSearch/Model/Index/Action/Fulltext/Refresh.php
+++ b/app/code/core/Enterprise/CatalogSearch/Model/Index/Action/Fulltext/Refresh.php
@@ -668,7 +668,7 @@ class Enterprise_CatalogSearch_Model_Index_Action_Fulltext_Refresh
     protected function _resetSearchResults()
     {
         $adapter = $this->_getWriteAdapter();
-        $adapter->update($this->_getTable('catalogsearch/search_query'), array('is_processed' => 0));
+        $adapter->update($this->_getTable('catalogsearch/search_query'), array('is_processed' => 0), array('is_processed != 0'));
         $adapter->delete($this->_getTable('catalogsearch/result'));

         $this->_app->dispatchEvent('enterprise_catalogsearch_reset_search_result', array());
diff --git a/app/code/core/Mage/CatalogSearch/Model/Resource/Fulltext.php b/app/code/core/Mage/CatalogSearch/Model/Resource/Fulltext.php
index ee8b1c3..1d89146 100755
--- a/app/code/core/Mage/CatalogSearch/Model/Resource/Fulltext.php
+++ b/app/code/core/Mage/CatalogSearch/Model/Resource/Fulltext.php
@@ -299,7 +299,7 @@ class Mage_CatalogSearch_Model_Resource_Fulltext extends Mage_Core_Model_Resourc
     public function resetSearchResults()
     {
         $adapter = $this->_getWriteAdapter();
-        $adapter->update($this->getTable('catalogsearch/search_query'), array('is_processed' => 0));
+        $adapter->update($this->getTable('catalogsearch/search_query'), array('is_processed' => 0), array('is_processed != 0'));
         $adapter->delete($this->getTable('catalogsearch/result'));

         Mage::dispatchEvent('catalogsearch_reset_search_result');

And last, there was a recommendation to add an index to the is_processed column:

ALTER TABLE `database`.`catalogsearch_query` ADD INDEX `IDX_CATALOGSEARCH_QUERY_IS_PROCESSED` (`is_processed`) COMMENT '';

I've tried all of them and while they brought minor performance improvements none of them got me near the performance of EE 1.13.

An easy fix (on the surface) would be to add back

if (!$this->_isFulltextOn()) {
    return $this;
}

at the beginning of execute() for these classes:

• Enterprise_CatalogSearch_Model_Index_Action_Fulltext_Refresh
• Enterprise_CatalogSearch_Model_Index_Action_Fulltext_Refresh_Changelog
• Enterprise_CatalogSearch_Model_Index_Action_Fulltext_Refresh_Row

If I do that, the code following isn't executed because Solr is configured to be used. As the core team deprecated this method for 1.14 it's ugly though and I'll try to stay away from that at all costs.

I'm only investigating this since yesterday so I hope I can follow up with a proper solution.

Update 09.02.2015

I found out by creating a xdebug profile dump that the communication between Magento and Solr takes up most of the time System > Configuration > Advanced > Index Management > Index Options > Catalog Search Index is set to Update on Save. Setting the Catalog Search Index to Update when scheduled significantly improves the speed.

Update 03.03.2015

In the meanwhile, the Enterprise support explained why $this->_isFulltextOn() is deprecated:

We removed $this->_isFulltextOn() from execute method because we refactored current mysql_fulltext indexer to encapsulate actual indexing work and adapt Catalog Search SOLR Index to use new Mview based indexer model.

The Enterprise_CatalogSearch module implemented new indexer model which utilizes changelog for partial reindexing. Currently when SOLR is used as catalog search engine, the catalogsearch_fulltext indexer falls back to use old indexer model. We utilize the new indexer model in Enterprise_CatalogSearch module when SOLR is set as catalog search engine.

Therefore, if merchant wants to skip fulltext indexation while saving products, please ask him/her to change index mode to update on schedule.

So the official solution pretty much is to change the Index Mode to Update when scheduled. We're using it for a few weeks without problems now. If your cron is running every minute you'll experience only a minor delay until the search is updated.