Magento – Tax calculation – Checkout

This is fixed on 1.8 CE Alpha - http://www.magentocommerce.com/knowledge-base/entry/ce-18-later-release-notes#ce18-1800tax-round

Specifying that prices display in the web store excluding tax and setting a 20% tax rate (or discount rate) now calculates the grand total correctly.

As this was a known issue and 1.8 is in Alpha at the moment, it can stand to reason that once 1.8CE goes gold these changes roll up into 1.13EE in a minor version update.

I am not sure if this would help you somehow, but I think that you can calculate with these two files, or modify the functions:

Tax subtotals file calculation is done here:

magento\app\code\core\Mage\Tax\Model\Sales\Total\Quote\Subtotal.php

public function collect(Mage_Sales_Model_Quote_Address $address)
    {

There you have round functions used:

Further magento takes these functions from this file:

\magento\app\code\core\Mage\Tax\Model\Calculation.php

Check these functions:

public function round($price)
    {
        return Mage::app()->getStore()->roundPrice($price);
    }

    /**
     * Round price up
     *
     * @param   float $price
     * @return  float
     */
    public function roundUp($price)
    {
        return ceil($price * 100) / 100;
    }

    /**
     * Round price down
     *
     * @param   float $price
     * @return  float
     */
    public function roundDown($price)
    {
        return floor($price * 100) / 100;
    }

I am not sure if this is the right answer, but hopefully this can get you somewhere. Because magento pricing is quite complex.