Linux Scheduler (not using all cores on multi-core machine) RHEL6

central-processing-unitlinuxmulti-threadingscheduler

I'm seeing strange behavior on one of my servers (running RHEL 6). There seems to be something wrong with the scheduler.

Here's the test program I'm using:

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>

void RunClient(int i) {
  printf("Starting client %d\n", i);
  while (true) {
  }
}

int main(int argc, char** argv) {
  for (int i = 0; i < 4; ++i) {
    pid_t p_id = fork();
    if (p_id == -1) {
      perror("fork");
    } else if (p_id == 0) {
      RunClient(i);
      exit(0);
    }
  }

  return 0;
}

This machine has a lot more than 4 cores so we'd expect all processes to be running at 100%.
When I check on top, the cpu usage varies. Sometimes it's split (100%, 33%, 33%, 33%), other times it's split (100%, 100%, 50%, 50%).

When I try this test on another server of ours (running RHEL 5), there are no issues (it's 100%, 100%, 100%, 100%) as expected. What's causing this and how can I fix it?

Thanks

Best Answer

You seem to be reinventing the wheel, there are various CPU torturing programs available already, such as stress. I bet the C compiler optimizes your program in such way it does not have to constantly burn CPU during the test run.

Try with

stress -c 4 -t 60

That would start 4 CPU intensive processes, and would run the test for 60 seconds. I bet you will see four cores running at 100%. Though if you have MUCH more cores than 4 (say, 16), then the result may vary, unless you pin the stress command to use specific cores with taskset.