Linux – How to prioritize SSH to work in the event of high CPU load

The CPU can and will be used for other processes, provided there is at least one process that is ready to receive CPU time. There's the rub - you can have an I/O-bound system with every process waiting for I/O to complete, and as there is nothing waiting for CPU time, there is no reason to schedule (and utilize) CPU for anything other than the kernel's activities...hence the term, I/O wait.

Try running vmstat 1 and see if there are numbers greater than 0 in the "b" column (2nd column) on a regular basis. If so, you're probably I/O-bound. Seeing it occasionally isn't a big deal, seeing it all the time with numbers in the 2-3 range is tolerable but not desireable, and seeing more than 5+ means you're probably way too busy (although that depends on how much I/O your system can accomodate, so it can be more or less, depending). The "b" means "processes blocked", as in, "the number of processes scheduled to run, but were blocked, pending the completion of an I/O".

Follow-up:

There is a known bug with heavy I/O and the newer schedulers on the 2.6 series of kernels. Try changing your scheduler to see if it has an impact.

How often did you run that logging cronjob? Maybe you should run it more often, because CPU usage doesn't peak instantly, you have to see an increase somewhere. Alternatively, you could use atop to monitor resource load (including CPU load) overtime.