I'm not an expert with Xeons these days, but don't Nehalem based Xeons require RAM in pairs of 3 not 2? The newer Nehalem based boards require memory to be installed in a very specific order with specific requirements (registered, ECC, etc. etc.). I would check the arrangement of your DIMMs in accordance with how many CPUs you have installed with the manual. Just my first guess.
Wikipedia has a fairly good explanation of rank (link). I'd say RamCity (a vendor for Kingston memory) has a more succint explanation on ranks (link):
A memory rank is, simply put, a block
or area of data that is created using
some or all the memory chips on a
memory module.
A rank must be 64 bits of data wide;
on memory modules which support Error
Correction Code (ECC), the 64-bit wide
data area requires an 8-bit wide ECC
area for a total width of 72 bits.
Depending on how memory modules are
engineered, they can contain one, two,
or four areas of 64-bit wide data
areas (or 72-bit wide areas, where 72
bits = 64 data bits and 8 ECC bits).
The article goes on mentioning price variation:
Why do the single- and dual-rank
memory modules vary in price?
In general, single-rank memory modules
are built using x4 (“By 4”) DRAM chips
and are more expensive than dual-rank
memory modules (which are built using
x8 DRAM chips); both module types have
the same number of chips but the x4
DRAMs are more expensive than x8
DRAMs. Dual-rank memory modules may
limit future upgradeability and
capacity of servers when using PC2700
or PC2-3200 memory. This tradeoff
between memory cost and capacity is
important to consider when purchasing
memory modules for Intel
Lindenhurst-based servers.
In terms of performance, I'd refer to wikipedia:
The ranks cannot be accessed
simultaneously as they share the same
data path.
So to sum up everything, it appears that ranks have more to do with density and pricing than actual performance. Granted, I'm working off of generalized statements from a vendor and wikipedia, I don't think most people put much effort into researching ranks. All that matters (for most server admins) is that RAM have matching ranks. I don't think it's an actual specification or requirement but it helps keep some consistency and keeps memory interchangeable within a number of similar servers.
Keep in mind that most servers are upgradeable and RAM density has a large part in factor. It's best (albeit more expensive) to get the more dense RAM for servers to make room for future upgrades.
Best Answer
The maximum number of domains on an M5000 is four.
http://www.sun.com/servers/midrange/m5000/specs.xml