Linux – How to periodically collect hardware performance counters in linux

linuxperformance-monitoring

I'm trying to monitor various hardware performance counters of a running system, however it seems that current PMC tool e.g. oprofile or perf tool can only record counters after the program finished, e.g.
~>perf record <command>;

~>perf report

the perf record returns only after the target program terminates, I'd like to periodically collect these counters like what vmstat does.

Edit: I want to collect hardware performance counters, e.g cache-misses,instructions,branch-misses, these counters are available in modern process.

Best Answer

It seems that current perf don't support Sar style output. However a very close emulation is possible with the help of watch command.

To collect counters periodically for the whole system, use the '-a' option. Below is an example of reporting the general hardware performance counters for every 60 seconds

watch -n 60 sudo perf stat -a sleep 60

It gives the output as

Every 10.0s: sudo perf stat -a sleep 10                                                                                               Wed May 11 11:19:36 2011


 Performance counter stats for 'sleep 10':

       19956.091380  task-clock-msecs         #      1.995 CPUs
              26802  context-switches         #      0.001 M/sec
               1871  CPU-migrations           #      0.000 M/sec
                951  page-faults              #      0.000 M/sec
         4582605668  cycles                   #    229.634 M/sec  (scaled from 69.93%)
         1993617795  instructions             #      0.435 IPC    (scaled from 81.07%)
          374028141  branches                 #     18.743 M/sec  (scaled from 81.34%)
           19071123  branch-misses            #      5.099 %      (scaled from 79.77%)
          100724660  cache-references         #      5.047 M/sec  (scaled from 18.66%)
            5461106  cache-misses             #      0.274 M/sec  (scaled from 20.23%)

       10.001116547  seconds time elapsed

The usage of sleep command is a bit counter intuitive at first glance. Here the perf stat will NOT collect counters specific to sleep command ,instead sleep command simply act as a way to tell the perf stat the duration of collection(I realize that from the example in the perf wiki ) Note to collect system wise counters, administrative privilege is required, so the sudo before perf stat is necessary.

If you simply need collect counters specific to a running process(e.g., with pid 2785), then a sample command is

watch -n 60 perf stat -p 2785 sleep 60

Note this time, sudo is NOT required.

Related Topic