Magento – Importing Orders from a CSV file

This opensource project Magmi

will import 75products per second on my system , compared to 1 product per 2minutes on magento's import dataflow.

Keep in mind when you upload these files that you make sure no one else can access it. ( ht access etc )

Surprised no answers with so many votes/views, so I'll bite:

1. This would be dependant on the old POS system, massage the data during import.
2. Familiarize yourself with Varien_Io, particularly Varien_Io_File. Since you'll most likely be dealing with such a large collection of data, keep in mind to use streams such as StreamReadCsv and StreamWriteCsv. More details on a "stream". Without a stream or linear read/write you may run into memory issues with other load/write methods.

With the above said here is an example: (source Atwix.com)

/**
 * Generates CSV file with product's list according to the collection in the $this->_list
 * @return array
 */
public function generateMlnList()
{
    if (!is_null($this->_list)) {
        $items = $this->_list->getItems();
        if (count($items) > 0) {

            $io = new Varien_Io_File();
            $path = Mage::getBaseDir('var') . DS . 'export' . DS;
            $name = md5(microtime());
            $file = $path . DS . $name . '.csv';
            $io->setAllowCreateFolders(true);
            $io->open(array('path' => $path));
            $io->streamOpen($file, 'w+');
            $io->streamLock(true);

            $io->streamWriteCsv($this->_getCsvHeaders($items));
            foreach ($items as $product) {
                $io->streamWriteCsv($product->getData());
            }

            return array(
                'type'  => 'filename',
                'value' => $file,
                'rm'    => true // can delete file after use
            );
        }
    }
}

As for importing orders, this example has helped most: (Source: pastebin)

<?php

require_once 'app/Mage.php';

Mage::app();

$quote = Mage::getModel('sales/quote')
    ->setStoreId(Mage::app()->getStore('default')->getId());

if ('do customer orders') {
    // for customer orders:
    $customer = Mage::getModel('customer/customer')
        ->setWebsiteId(1)
        ->loadByEmail('customer@example.com');
    $quote->assignCustomer($customer);
} else {
    // for guesr orders only:
    $quote->setCustomerEmail('customer@example.com');
}

// add product(s)
$product = Mage::getModel('catalog/product')->load(8);
$buyInfo = array(
    'qty' => 1,
    // custom option id => value id
    // or
    // configurable attribute id => value id
);
$quote->addProduct($product, new Varien_Object($buyInfo));

$addressData = array(
    'firstname' => 'Test',
    'lastname' => 'Test',
    'street' => 'Sample Street 10',
    'city' => 'Somewhere',
    'postcode' => '123456',
    'telephone' => '123456',
    'country_id' => 'US',
    'region_id' => 12, // id from directory_country_region table
);

$billingAddress = $quote->getBillingAddress()->addData($addressData);
$shippingAddress = $quote->getShippingAddress()->addData($addressData);

$shippingAddress->setCollectShippingRates(true)->collectShippingRates()
        ->setShippingMethod('flatrate_flatrate')
        ->setPaymentMethod('checkmo');

$quote->getPayment()->importData(array('method' => 'checkmo'));

$quote->collectTotals()->save();

$service = Mage::getModel('sales/service_quote', $quote);
$service->submitAll();
$order = $service->getOrder();

printf("Created order %s\n", $order->getIncrementId());

With the example you have now will be resource heavy, as there are Mage::getModel(... calls in foreach loops which is bad practice, and will most likely either timeout, or fill up memory rather quickly. Especially if you have this wrapped in another foreach/while.

This...

foreach ($products as $productId=>$product) {
  $_product = Mage::getModel('catalog/product')->load($productId);

Should look like:

$_product = Mage::getModel('catalog/product');
foreach ($products as $productId=>$product) {
  $_product->load($productId);

I would not attempt to try and relate every CSV bits of data to Magento objects. It would be madness and a bit of overkill, keep with resource model entry points of $model->load(EntityId).

Also note if you are attempting to import over 100k+ orders I would be concerned for performance after the large imports as its necessary to keep MySQL tuned to handle such large volumes, not too mention if I'm not mistaken sales objects are still EAV based, and do not perform well under high volume/traffic. There is a reason Magento Enterprise has a Sales Order Archive module to pull old data out of the "transactional" sales order tables to prevent bloated/stale data that isn't needed for taking orders.

To Wrap: I would bring up the requirements and needs of the business to store such large data, if its purely reporting there are better alternatives to suite this than Magento.