I'm running windows server 2008 64, but all open-source stuff (mysql, ruby, python, R) is either not that well-tested or has problems. And of course, if you have to compile packages, you are on your own.
I guess it's obvious that FOSS will run better on linux; maybe OSX too. But there are wide differences in terms of distros. Which one has the best reputation for x64? This is for a desktop use (doing scientific simulation, needs lots of ram).
I don't particularly like macs, but could consider it if they have demonstrably better x64 support.
Best Answer
They're all fine for doing what you want to do. Pick one that someone you can ask for help knows.