Electronic – Coax cable with CAN bus

You need to be familiar with Transmission Line Theory to understand the deeper physics in play here. That said, here's the high-level overview:

How important termination is to your system is almost exclusively determined by how long the bus wires are. Here length is determined in terms of wavelengths. If your bus is shorter than one wavelength over 10, the termination is irrelevant (practically) since there is plenty of time for the reflections introduced from an impedance mismatch to die out.

Length defined in wavelengths is a strange unit on first encounter. To convert to standard units you need to know the velocity of the wave and it's frequency. Velocity is a function of the medium it travels through and the environment surrounding the medium. Usually this can be estimated fairly well through the dielectric constant of the material and assuming free-space surrounding that medium.

Frequency is a little more interesting. For digital signals (such as those in CAN), you are concerned with the maximum frequency in the digital signal. That is well approximated by f,max = 1/(2*Tr) where Tr is the rise time (defined 30%-60% of the final voltage level, conservatively).

Why it's 120 is simply a function of the design limited by physical size. It isn't specifically important which value they picked within a broad range (for example, they could have gone with 300 Ohms). However, all devices in the network have to conform to the bus impedance, so once the CAN standard was published there can be no more debate.

Here's a reference to the publication (Thanks @MartinThompson).

The coax picture and equations are refered to unbalanced source/load and coax. With CAN you have balanced, twisted pair configuration, where noise is picked up equaly in both conductors, therefore noise voltage is eliminating each other. No current is passing in the shield, the shield is just a Faraday cage screening twisted pairs from noise. It has to be on earth potential in one point. You have best results if the shield is tied on the ground on both sides, but usually we tend not to connect shiled on both sides, as the ground loop current can flow trough shield - this if ground potentials of both sides is different.