Your UTP curve is only qualitative, i.e. doesn't show actual numbers. Your STP cable will show similar behavior: attenuation increases with distance, and so does crosstalk. The shielding won't prevent that. The qualitative image will look the same.
Quantitatively, expect the STP to have a higher attenuation due to capacitive coupling with the shielding's ground.
You claim to be familiar with the Fourier analyses.
Take a look at this signal:
Here we have a sinusoidal signal summed together with a DC signal. The AC signal has voltage of 1 V peak to peak, while the dc component is 2 V. For each point on the graph, what happened is that the value of the cosinusoidal signal was summed with the 2 V DC level.
If we do Fourier transform of the signal, we'd get a value at the frequency of 0, which represents the DC component and the value at the frequency of the AC signal itself.
A "pure" AC signal goes from Vpeak to -Vpeak and is centered around zero. If the signal's average value is not zero, then it has "DC bias", that is to say that it can be represented as a sum of a DC signal and an AC signal that is centered around zero.
Next, a bit about phone lines and its cabling: Traditional POTS uses only two lines, one of which is ground and the other is traditionally said to be -48 V DC. The negative voltage then summed with the actual signal which carries the data.
ADSL "basically" works same way as fax modems and voice calls: The -48 V DC is summed together with the AC signal which carries the data. At both telephone central office and at subscriber's location filters are used to separate the voice traffic from ADSL signalling. This is possible since ADSL and telephone work in different frequency bands, so they do not overlap. You can then "simply" filter out different bands and send them to different devices.
EDIT:
The way right to think about this is that in reality there's no "DC" and "AC" voltage (in fact, terms DC and AC aren't really all that well defined and the conventions depend on region). Those terms are simply abstractions that are used to simplify circuit analyses.
Let's say that you have grounded line G and line with voltage A. Now imagine that it's possible to measure voltage of the A line extremely quickly and that you associate one voltage measurement with one point on time axis. You'd get something that looks like this:
Each measurement point tells you "exact" voltage at the time of the measurement. You don't get two lines because there's only one actual voltage and its the voltage that's shown.
Best Answer
The conservative answer has to be "no" - phone lines are "balanced" in that the impedance presented by each wire to ground is constant. Coax cable is "unbalanced" and the impedance presented by the screen can be vastly different to the impedance presented to ground by the inner core.
The telephone cable's natural characteristic impedance is around 600 ohm whereas the coax cable might be 50 ohms or 75 ohms and it could cause data symbol reflections and therefore an excessive bit error rate when attached to a system expecting 600 ohm.
However, being less conservative, if the coax cable is short (say no more than a few metres) then you might get away with it being inserted.