When you match to complex valued loads the matching for zero power reflection states that the impedance seen from your complex-valued load (\$100+50j\$) has to be its complex conjugate (\$100-50j\$).
This is because that way we would be satisfying the max. power theorem, and, at the same time, getting rid of the imaginary part of your load.
Impedance matching is tricky, but the role of a quarter wave transmission line is to map from one impedance to another. The actual impedance of the line will not match either the input or the output impedance - this is entirely expected.
However at a given frequency, when a correctly designed quarter wave line is inserted with the correct impedance, the output impedance will appear to the input as perfectly matched. In your case, the transformer will make the \$20\Omega\$ impedance appear as if it is a \$100\Omega\$ impedance meaning no mismatch. Essentially it guides the waves from one characteristic impedance to another.
The easiest way to visualise this is on a Smith chart, plot the two points 0.4 (\$20\Omega\$) and 2 (\$100\Omega\$). Then draw a circle centred on the resistive/real axis (line down the middle) which intersects both points. You will find that this point is located at 0.894 (\$44.7\Omega\$) if your calculations are correct. This is shown below at \$500\mathrm{MHz}\$, but the frequency is only important when converting the electrical length to a physical length.
What a quarter wave transformer does is rotate a given point by \$180^\circ\$ around its characteristic impedance on the Smith chart (that's \$\lambda/4 = 90^\circ\$ forward plus \$90^\circ\$ reverse).
Exactly why it does this is complex. But the end result of a long derivation is that for a transmission line of impedance \$Z_0\$ connected to a load of impedance \$Z_L\$ and with a length \$l\$, then the impedance at the input is given by:
$$Z_{in}=Z_0\frac{Z_L+jZ_0\tan\left(\beta l\right)}{Z_0+jZ_L\tan\left(\beta l\right)}$$
That is an ugly equation, but it just so happens if the electrical length \$\beta l\$ is \$\lambda/4\$ (\$90^\circ\$), the \$\tan\$ part goes to infinity which allows the equation to be simplified to:
$$Z_{in}=Z_0\frac{Z_0}{Z_L}=\frac{(Z_0)^2}{Z_L}\rightarrow Z_0=\sqrt{\left(Z_{in}Z_L\right)}$$
Which is where your calculation comes from.
With the quarter wave transformer in place, the load appears as matched to the source. In other words, the transformer matches both of its interfaces, not just the input end.
You can also see from this equation why the transformer only works for a single frequency - because it relies on the physical length being \$\lambda/4\$. You can actually (generally using advanced design tools) achieve an approximate match over a range of frequencies - basically a close enough but not exact match.
Best Answer
Continuous varying impedances are used all the time for impedance matching. If you have a very capacitive part of a trace (for example, where a large component pad might be), you can have a relatively inductive transition before or after it to "balance" it out.
What will end up happening is that the reflections will "stack up" but, instead of being at one point (a VSWR peak), it will be moderately spread out. You can still imagine it discretely, but in small steps.
And also remember, if you have a small reflection point, any backward reflection after THAT will be reflected slightly FORWARD, and so on.
Anyway, the good gents at http://www.microwaves101.com/encyclopedia/klopfenstein.cfm always have a nice, in depth explanation.
edit: I didn't completely answer your question. "How it would look" is dependent a bit on how you are describing it. In the frequency domain, what you'll probably get is a VSWR that is "de-Q'd". You'll go from a nice sharp peak at midband to a more gradual, broader band response.
In the time domain....well, I don't work with the time domain as much but I would imagine you would have a lower amplitude, longer pulsewidth "ringing" or reflection.