Linux – Calculating Total I/O Cost of a Process

iolinux

I know that iotop lets me measure the disk bandwidth used by any or all processes in real time. iotop -a measures the accumulated disk bandwidth, which is closer to what I want.

However, when the process I run terminates, it disappears too soon in iotop for me to be able to see exactly how many I/Os the process used in total since it started. Is there a way to record the total I/O usage of a process when it ends, and perhaps have this saved to some file for further analysis?

Note that I am only looking for answers exclusive to Linux, specifically Ubuntu.

Best Answer

Try pidstat. Use it like this: pidstat -d -e command

pidstat is able to report statistics for Linux tasks. The -d instructs pidstat to gather IO stats. pidstat will stop and print the report once the command finished.

Related Solutions

Move Running Process to Screen – How to Do It

Have a look at reptyr, which does exactly that. The github page has all the information.

reptyr - A tool for "re-ptying" programs.

reptyr is a utility for taking an existing running program and attaching it to a new terminal. Started a long-running process over ssh, but have to leave and don't want to interrupt it? Just start a screen, use reptyr to grab it, and then kill the ssh session and head on home.

USAGE

reptyr PID

"reptyr PID" will grab the process with id PID and attach it to your current terminal.

After attaching, the process will take input from and write output to the new terminal, including ^C and ^Z. (Unfortunately, if you background it, you will still have to run "bg" or "fg" in the old terminal. This is likely impossible to fix in a reasonable way without patching your shell.)

Linux – Determining Which Process Is Causing Heavy Disk I/O

iotop (link) for starter ;) I haven't seen you posting an output of it.

1: I have experienced almost the same situation with a logging filesystem and atime - however with more writes.

Try to remount with noatime and turn off filesystem logging (later for testing only) in order to see if it's filesystem based and as said, iotop if it's process based.

2: I guess this partition is not part of a just-rebuilding raid array, is it?

3: If you are having a lot of very small files (a lot smaller than the actual block device blocksize and/or the filesystem blocksize), and you are reading those small files, you end up reading entire blocks from the system, and most of those blocks will be read for nothing.

4: If nothing helps above, you can always get the list of files accessed by executing

echo 1 > /proc/sys/vm/block_dump

please note that it degrades system performance by a lot. Instructions are available in my previous post here

Best Answer

Related Solutions

Move Running Process to Screen – How to Do It

reptyr - A tool for "re-ptying" programs.

USAGE

Linux – Determining Which Process Is Causing Heavy Disk I/O

Related Topic