Electronic – Regulating voltage with low headroom

The input must not affect the output, it's a closed loop system, which means that in wide range of input voltage, it will regulate the output to what you configured.

And since i haven't seen such unregulated DC/DC for a long time, i suggest to first understand how do you connect it and what are you measuring. Specifically, you have to measure on the module itself to ensure regulation. The wires must be thick enough to hold charge current. Of course, in the end the measurement should be taken near the phone. If the cable is poor, you may need to raise the DC/DC output voltage.

EDIT: I didn't see OP's edit where he says he already tried this. Anyway I'll leave this answer up for future readers.

You are correct that most voltage references cannot both source and sink current. One option is to either use a shunt reference or to pre-load a series reference:

^{simulate this circuit – Schematic created using CircuitLab}

In this circuit, normally the reference IC provides 13 mA to R1. If the actual load (connected at OUT) needs to source current, that current will flow through R1 and U1 will just reduce its output current. I've shown both a filter capacitor (C1) to maintain stability of the reference/regulator, and bypass capacitance (C2) to take up most of the switching current and maintain steady voltage at the load under transients.

However, pre-loading is fairly wasteful of power (about 26 mW burned in the pre-load resistor, plus the extra current through the series reference). Andd 10 mA is a lot to ask from a precision reference circuit.

But if ~50 mW doesn't bother you, and you don't need sub-1% accuracy, it should work okay and doens't cost much.