Computers have trouble storing fractional numbers where the denominator is something other than a solution to 2^x. This is because the first digit after the decimal is worth 1/2, the second 1/4 (or 1/(2^1) and 1/(2^2)) etc.
Why deal with all sorts of rounding errors when the computer could have just stored the decimal part of the number as another whole number (which is therefore accurate?)
The only thing I can think of is dealing with repeating decimals (in base 10), but there could have been an edge solution to that (like we currently have with infinity).
Best Answer
There are actually modes of numbers that do that.
Binary-coded decimal (BCD) arithmetic has the computer work in base 10. The reason you run into this rarely is that it wastes space: each individual digit of a number takes a minimum of four bits, whereas a computer could otherwise store up to 16 values in that space. (It can also be slower, but it's possible to have hardware-accelerated BCD math that works just fine.). This is, in fact, exactly what most calculators do, which is why there are certain classes of rounding problems you'll never hit on a $5 Casio that will eat your lunch on a desktop computer.
The other route you can take is to use rational numbers--that is, a numerator and a denominator, stored as integers. This is actually available in nearly all languages, is exact, and allows you to store everything in native binary formats. The problem is that, at the end of the day, users probably do not want to see fractions like 463/13, nor even 35 and 8/13. They want to see 35.615..., and the moment you get there, you face all the typical problems. Add in that this format takes even more space, and can be significantly slower than floating point arithmetic, and you'll find no computers use this format by default.
So: computers can do what you want, but it's slow and it wastes space, so they only do it when they really have to. The rest of the time, the speed and space savings of floating point are a better trade-off.