Here are the specs for the chip/chip family that our vendor is using.
For our product we are asking them to perform a calculation by multiplying a number by a factor that has a decimal point…for example: 1.07, 0.93, etc.
We are then asking them to round to the nearest integer.
However, the number that they calculate sometimes have a rounding error. Ex: On my calcuator 56 * 2.36 = 132.16…rounded to 132. However, the microcontroller will return an answer of 133 displayed on an LCD of our product.
My question is, how can this be fixed? I've been asking the vendor many times and they can't seem to get it right.
Is calculating floating point number really hard for microcontrollers that don't have that native function? (Related question: DIY FP – Implementing floating point math on a microcontroller without a hardware FPU)
UPDATE
The range for the results are 1 to 780.
Best Answer
You don't say anything about the range of your numbers or the required precision. But you often don't need floating point for this, fixed point often will do. Just work with integers and place a virtual decimal point at a certain position. A 16-bit number could for instance have 8 bits left of the decimal point, and 8 bits after. This will give you a range from 0 to 255 (I'm assuming unsigned numbers), with a precision better than 0.01.
Rounding is then simple: just look at the most significant decimal, the bit directly right of the decimal point. If that's a zero, then the integer part should be left untouched. If it's a one, add 1 to the integer part.
edit
There's a zillion ways of rounding: round up/down, round to nearest, round to zero, banker's rounding,... Be sure you and the vendor speak the same language.