Floating-point: “The leading 1 is ‘implicit’ in the significand.” — …huh

I know of no way to way to make normal floating points deterministic in .net. The JITter is allowed to create code that behaves differently on different platforms(or between different versions of .net). So using normal floats in deterministic .net code is not possible.

The workarounds I considered:

1. Implement FixedPoint32 in C#. While this is not too hard(I have a half finished implementation) the very small range of values makes it annoying to use. You have to be careful at all times so you neither overflow, nor lose too much precision. In the end I found this not easier than using integers directly.
2. Implement FixedPoint64 in C#. I found this rather hard to do. For some operations intermediate integers of 128bit would be useful. But .net doesn't offer such a type.
3. Implement a custom 32 bit floatingpoint. The lack of a BitScanReverse intrinsic causes a few annoyances when implementing this. But currently I think this is the most promising path.
4. Use native code for the math operations. Incurs the overhead of a delegate call on every math operation.

I've just started a software implementation of 32 bit floating point math. It can do about 70million additions/multiplications per second on my 2.66GHz i3. https://github.com/CodesInChaos/SoftFloat . Obviously it's still very incomplete and buggy.

You've identified the number of bits for each portion of the representation, so you're already halfway there. There are:

• 2^1 = 2 possibilities for the sign
• 2^8 = 256 possibilities for the exponent bits, of which two are reserved for non-normals: an exponent field of all zeros is used for floating-point zeros and subnormals, while an exponent field of all ones is used for infinities and nans
• 2^23 = 8388608 possibilities for the mantissa (note that 'significand' is generally the preferred term).

Multiplying, that gives 2 * 2^23 * (2^8 - 2), or equivalently 2^32 - 2^25, possibilities altogether. So there are 2^32 - 2^25 = 4261412864 distinct normal numbers in the IEEE 754 binary32 format. The two zeros are not, technically, normal numbers, but if you wanted to include them in the count you'd get 2^32 - 2^25 + 2 instead.

And yes, this generalises directly to all the other IEEE 754 binary interchange formats. I'll leave it to you to find the numbers for double precision, half precision, quadruple precision etc.

Just for fun, here's a complete breakdown:

• 2 zeros (sign 0 or 1, exponent and significand fields zero)

• 2^24 - 2 subnormal numbers (sign 0 or 1, exponent field zero, significand field nonzero)

• 2^32 - 2^25 normal numbers (as above)

• 2 infinities (sign 0 or 1, exponent field all ones, significand field zero)

• 2^23 - 2 signaling NaNs (sign 0 or 1, exponent field all ones, significand field nonzero but with zero first bit)

• 2^23 quiet NaNs (sign 0 or 1, exponent field all ones, significand field has 1 as first bit)