Linux – NUMA CPU numbering in Linux

linuxmulti-corenuma

I have access to two NUMA servers. One of them is Dell R720 and has these CPUs:

$ cat /proc/cpuinfo |grep Xeon|sort|uniq -c
     24 model name  : Intel(R) Xeon(R) CPU E5-2630L v2 @ 2.40GHz

The other is a HPE DL360 Gen8 and has these CPUs:

$ cat /proc/cpuinfo |grep Xeon|sort|uniq -c
     24 model name  : Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz

At work where we have many HPE Gen9 servers, I have been used to CPU numbering (socket0, socket1, socket0 HyperThreads, socket1 HyperThreads). It seems the HPE DL360 Gen8 uses this numbering:

$ cat /proc/cpuinfo |grep physical.id|uniq -c
      6 physical id : 0
      6 physical id : 1
      6 physical id : 0
      6 physical id : 1

But the Dell R720 server uses different numbering:

$ cat /proc/cpuinfo |grep physical.id|uniq -c
      1 physical id : 0
      1 physical id : 1
      1 physical id : 0
      1 physical id : 1
      1 physical id : 0
      1 physical id : 1
      1 physical id : 0
      1 physical id : 1
      1 physical id : 0
      1 physical id : 1
      1 physical id : 0
      1 physical id : 1
      1 physical id : 0
      1 physical id : 1
      1 physical id : 0
      1 physical id : 1
      1 physical id : 0
      1 physical id : 1
      1 physical id : 0
      1 physical id : 1
      1 physical id : 0
      1 physical id : 1
      1 physical id : 0
      1 physical id : 1

My question is, what causes this difference? The servers have two slightly different kernel versions:

Dell R720:

$ uname -a
Linux dell 4.10.0-33-generic #37~16.04.1-Ubuntu SMP Fri Aug 11 14:07:24 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

HPE DL360 Gen8:

$ uname -a
Linux hpe 4.11.0-14-generic #20~16.04.1-Ubuntu SMP Wed Aug 9 09:06:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Is this caused by different kernel versions? Or by different CPUs? Or by different motherboards / BIOSes?

Edit: I updated the kernels on both machines and rebooted, so now both machines use exactly the same kernel version. Nevertheless, the difference is still there.

Best Answer

Stop the grepping and uniq and run lscpu and lstopo --of png > server.png and visualize the results...

[root@LA_Specialty ~]# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                24
On-line CPU(s) list:   0-23
Thread(s) per core:    2
Core(s) per socket:    6
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 62
Model name:            Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz
Stepping:              4
CPU MHz:               3501.000
BogoMIPS:              7013.88
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              25600K
NUMA node0 CPU(s):     0-5,12-17
NUMA node1 CPU(s):     6-11,18-23

enter image description here