Linux – New RAM on HP DL 380 causes errors in 64 bit CentOS

centosecclinuxmemory

I added new HP memory to a server to go from 8 to 32GB. Now I have this error message appearing in /var/log/messages every few seconds.

    Jan  8 20:13:08 vmware01 kernel: EDAC MC0: CE row 2, channel 2,
 label "": (Branch=1 DRAM-Bank=6 RDWR=Read RAS=13788 CAS=2840,
 CE Err=0x2000 (Correctable Non-Mirrored Demand Data ECC))

I've googled for the error message and had no luck. Does anyone have any idea what it means and how to fix it?

Both the BIOS and the operating system see all 32 GB.

Any help with be greatly appreciated.

Best Answer

The error message basically means that the memory you install is faulty. The system has found that the memory was corrupt and the ECC was able to correct the single bit error and alerted the OS to the fact that the memory is unhappy.

The DRAM in question is the DRAM in Bank 6 (most probably dual channel memory) so look at that location for the faulty ram.

Related Solutions

Linux – ECC chipkill errors: which DIMM

In addition to using the EDAC codes, you can use the CLI only HP utilities to determine this while the machine is online. The cli versions are far more lightweight than the web based ones and do not require you to open ports or have a daemon constantly running.

hpasmcli will give you the cartridge and module #'s of the failed modules. A little quicker than analyzing EDAC.

Example:

hpasmcli -s "show dimm"

DIMM Configuration
------------------
Cartridge #: 0
Module #: 1
Present: Yes
Form Factor: 9h
Memory Type: 13h
Size: 1024 MB
Speed: 667 MHz
Status: Ok

Cartridge #: 0
Module #: 2
Present: Yes
Form Factor: 9h
Memory Type: 13h
Size: 1024 MB
Speed: 667 MHz
Status: Ok

Cartridge #: 0
Module #: 3
Present: Yes
Form Factor: 9h
Memory Type: 13h
Size: 1024 MB
Speed: 667 MHz
Status: Ok

Cartridge #: 0
Module #: 4
Present: Yes
Form Factor: 9h
Memory Type: 13h
Size: 1024 MB
Speed: 667 MHz
Status: Ok

Status will change for failed modules.

Linux – Monitoring physical RAM errors on Linux

Most modern servers of any reasonable quality have an IPMI module which will report bad RAM (usually via SBE (single-bit error) messages from ECC RAM -- You are using ECC RAM in your servers, right?). The IPMI module also monitors and reports on a bunch of other useful stuff.

You can monitor the IPMI module using a variety of network monitoring systems (if you have a management network for the IPMI NICs) or using ipmitool which is available on most Unix systems. Many vendors (Dell and IBM for sure) also have specialized tools that interrogate the IPMI module for on-line diagnostics. Check with your hardware vendor for more details.

Best Answer

Related Solutions

Linux – ECC chipkill errors: which DIMM

Linux – Monitoring physical RAM errors on Linux

Related Topic