You're correct; this will output 100 mV as drawn. The reason for this is the amplifier outputs whatever voltage is required to set its inputs equal. Since one input is fixed at 100 mV, the other one must be driven to 100 mV. This happens when the output is at 100 mV. Since every node in the circuit is at 100 mV, no current will flow through the resistors (well, ignoring any current flowing in to the op amp input pins).
Since this is a jfet input op-amp be aware that input voltages slightly over the + rail supply can quickly damage the part. Your circuit seems protected as is, but if you did any substantial poking/probing you might want to verify that the part in the circuit is still ok.
While this op-amp is listed as being a rail to rail part it doesn't absolutely reach the rails. Per the spec the low end will only go to within 5mv of the - rail and 10mv from the + rail. (See the spec sheet section "Output Characteristics", page 18.) Other odd things happen when the output is very close to either power rail.
A potential source of larger errors may be due to the input error voltage when the output is within 300mv of either power rail. (See spec sheet figure 13, page 12). While the error is normally in the uV range your minimum output of about 30mv would go well off the chart on the high end. With a 10k load you would need to keep the output at about 120mv above the - rail to minimize the error, (I'm extrapolating the chart between RL=20k to 2k). This chart uses an example with +5v-5v supply rails, using only +5v-0v might be even worse.
Also be sure you don't have any significant AC noise on your inputs. If you were expecting all DC outputs maybe you debugged with a DVM on DC. Use a scope to check for AC noise. Just a few mV of noise would be very significant at your lowest input levels. If there is any significant AC coming in you could put caps across the 10k feedback and the 10k going to GND, (of the diff amp). The lower the noise frequency the larger cap values would need to be used to filter it.
You may want to decrease the 2.47v reference a small bit to keep the lowest output voltage farther away from the - rail (0v). Since you say your 2.47v reference is buffered by another op-amp you could put a multi-turn pot ahead of that input to give you an accurate way to calibrate the output voltage range.
Too large a cap on the final output (going to the A/D input) might also cause problems for this op-amp.
Best Answer
First we must start with some basic assumptions, we assume that the inputs of the circuit are fed from ideal voltage sources and that the op-amp is an ideal op-amp. The latter means that the circuit is linear and we can apply the principle of superposition to calculate the two gains..
For the inverting gain we treat things as if the + input of the circuit (not the + input of the op-amp) is connected to ground. The bottom two resistors have no affect since the op-amps inputs are high-impedance and the top two resistors act just like a normal inverting amplifier.
-Rf/Rg = -20/5 = -4
For the non-inverting gain we treat things as if the - input of the circuit (not the - input of the op-amp) is connected to ground.
When we do this we find that the non-inverting gain is made up of two stages. First we have a voltage divider formed by the bottom two resistors in the diagram. Then we have the op-amp along with the top two resistors in the diagram acting as a non-inverting amplifier. So the overall non-inverting gain works out to
(30/(20+30)) * (1+(20/5)) = (3/5) * 5 = 3