Magento – How to deal with the Sales Order API not having a limit capability

apiorders

Working on an Order API integration for a community module. Wanted to avoid rewriting the Mage_Sales_Model_Order_Api model, so instead I implemented an observer of the sales_order_collection_load_before event that limits the result set to a hard limit. Went with 250, that's the limit Shopify uses, seems like a sane number if you're going to hard code something.

If there are API integrations that depend upon the order API not having any limit, then they could run into issues. For example, if someone has an Order API integration that pulls in the full number of orders on a monthly basis (or even a daily basis, the time range is somewhat arbitrary), then it would now only pull in 250 orders.

While it's possible to filter the result set on a date range, any date range that you pick would be essentially arbitrary. A day's worth of orders might be way too small and result in an unnecessarily high number of API hits for smaller stores, and for larger stores, it might be way too large of a range. It really is necessary to be able to limit an API response on an absolute number of results, and that's what most modern API's do.

Has anyone found any good solution to this?

Best Answer

A fix was found in the comments but I'm adding a formal answer so it can be marked as solved and people searching in the future can see easily.

Solution provided by Ashley Schroeder on twitter

Summarised answer:

We back off progressively if we detect big responses. i.e try for 24hours, then back off to 6*4hours. ...
If the connection drops or gets too big we bail out and fall back, it's recursive so if 4hour is too big we split again.

Also thats for fetching historical data, for frequent real time window is typically 10-20minutes, same splitting applies though

Related Topic