Electronic – Using 3.3V or 2.5V for ADC and powering Op Amp from 3.3V or 5V

This should work, but usually there's also a resistor from the non-inverting input to ground:

If \$\dfrac{R4}{R2} = \dfrac{R3}{R1}\$ then:

\$V_{OUT} = \dfrac{R_{3}}{R_{1}} \times (V_{IN+} - V_{IN-})\$

To minimize offset error both inputs have to see the same impedance, and therefore \$R1 = R2\$ and \$R3 = R4\$.
Omitting R4 will only give a scaling factor for the voltage on the non-inverting input, namely:

\$V_{OUT} = 2 \times V_{IN+} - V_{IN-}\$

but changing the setting for the lower potmeter should definitely have a result in the output. Did you measure the voltages on both inputs?

What happens if you set the lower voltage to 2.5V and the upper to 1V? The inverting input should also be 2.5V, and the output 4V. What do you measure?

Note: especially the lower voltage follower is not necessary in your version; the opamp's input current is low enough to be negligible for most uses, and by the way, you're connecting the potmeter's wiper to an exactly same input!

Further reading
differential amplifier tutorial (interesting site overall!)

Yes, a voltage divider is fine in theory. How much it effects signal quality is in large part dependent on what you consider a quality signal. Is this HiFi audio, a digital data stream, voice audio, RF, something else?

There are several issues with resistive voltage dividers you should be aware of:

1. The voltage divider will load the source signal. You need a divider that puts out 1/4 of the input signal. Any divider with the top resistor 3x the bottom will do that.

   

   In this case R1 = 3*R2. The impedance looking into the divider from the source will be R1+R2. You have to make sure this is high enough to not load the source signal to change its characteristics to the point you care. For example, if R1=30kΩ and R2=10kΩ, then the divider will load the source with 40kΩ.

2. Consider the output impedance. This is most of what Steven was talking about. With a perfect voltage source (0 impedance) driving the divider, the output impedance is R1//R2. With the example values above, that would be 30kΩ//10kΩ = 7.5kΩ. As Steven mentioned, this needs to be considered when connecting to a microcontroller A/D. It's not as much a issue of loading the divider output as that the A/D needs some finite impedance to charge its internal holding cap in finite time. At high impedance, the little leakage current of the A/D pin times the impedance also produces enough offset voltage to corrupt the A/D reading. Due to these issues, microcontroller manufacturers specify a maximum impedance for driving a A/D input. In old PICs with 8 or 10 bit A/Ds, this was generally 10kΩ. This is less in some newer faster A/Ds or at higher resolution like 12 bits. Some of the dsPIC family require only a few 100 Ωs or less.

3. Frequency response. There is always some stray capacitance. The various stray capacitances will cause low and high pass filters. The final result is unpredictable since the stray capacitance is unpredictable. Using the 30kΩ and 10kΩ example again, the output impedance is 7.5kΩ. If this were loaded with 20pF, for example, then you'd have a low pass filter with about 1 MHz rolloff. If the signal is audio, no problem. If it is a fast digital signal, that could be a serious problem.

   One way to deal with this is to add deliberate capacitance as small as possible but several times the expected stray capacitance so that the total capacitance becomes predictable. The capacitance accross each resistor must be inversely proportional to that resistance. For example, here is a nicely ballanced voltage divider:

   

   At low frequencies, the resistors dominate and divide the signal by 4. At high frequencies, the capacitors dominate and divide the signal by 4. The crossover where the resistive and capacitive actions are about equal is 53 kHz in this example.

   By the way, this is how dividing scope probes work. A "10x" probe divides the signal by 10. Since it needs to do that accross the scope's whole frequency range, a little capacitance is added to each resistor. The stray capacitance can never be exactly known and there will be some part tolerance anyway, so one of the capacitors is made variable. This is what the "probe compensation" adjustment is. This adjustment turns a small trim cap of a few pF. With a square wave in, you can see easily see the point where the capacitive and resistive dividers match.

   One drawback of this capacitive and resistive approach is that the impedance of the divider goes down at high frequencies. While this approach is useful for properly dividing higher frequencies, it also loads them much more than just two resistors would. There is no free lunch.

Hopefully you can see some of the issues and tradeoffs now. If the impedances don't work out, then you need to consider some kind of active buffering like Steven already described. That has it's own set of issues, like offset voltage, frequency response, and gain error if gain is not just 1, but those are for another thread.