TL081 is not a particularly good choice for low DC voltages (it's better for AC coupled designs). Precision op-amps have offset voltages in the 10's of microvolts or lower, and there are even auto-zero op-amps that have negligible offset voltage, an drifts in the tens of nV/°C (at some significant cost in other characteristics). It's also quite noisy (25nV/\$ \sqrt{Hz}\$), but at least has a typical flicker noise corner frequency in the 1Hz range. Aside from high input Z, it has basically one really nice advantage, it's really cheap, and widely available, which is why I've actually designed such a 37-year-old op-amp into a new product recently.
A good general purpose (low noise, capable of handling +/-15V supplies, low distorion for AC signals) precision op-amp suitable for mV levels might be the OPA209A.
You can certainly null the offset voltage of your TL081 out using a trimpot as shown on the datasheet, but it won't stay that well nulled for long. A 10°C change will typically change the offset voltage by 100uV, and about one out of every two will be worse (there's no guarantee how much worse, but a guess would be most are better than +/-30uV/°C). An OPA209 is going to be roughly an order of magnitude better.
There are probably going to be better (and many worse) choices for any given application, all things considered. It's amazing what performance you can get for a a few dollars, so it's worth looking around rather than trying to make a silk purse of a sow's ear.
Just to give you an idea of the kind of (in) accuracy you could get, consider that the gain of the TL081 is only guaranteed to be >15,000, so a gain of 1000 amplifier could have a gain error in the 6% range even without the input offset error (which would be very temperature dependent, and has a -3dB corner of something like 20Hz. Cascading two \$\sqrt {1000}\$ gain amplifiers would help with that (null only the first one).
If the range is, say +/-5mV input, frequency is 0.001 to 1Hz and required accuracy 1% of FS + 50uV**, it might be typically *** okay in a lab environment with a light output load, if you null it after warm-up.
** Instrumentation type specification- it means the output could be as much as +/- 100mV from the ideal value with any input, so a 1mV input could give you 900mV or 1100mV.
*** "Typically" means that one chip might be okay, and the next might not meet the requirements. Guaranteed value is probably 10 times worse.
Can you please explain what "mismatches in the differential input
stage" is?
All op-amps have an input voltage error term. It's just like a small millivolt battery being placed in series with one of the inputs. If you build an amplifier with a gain of ten, the output voltage from the op-amp becomes offset from where you would expect it to be by 10 x the mV offset I just mentioned.
It is specifically this static error that the offset-voltage nulling circuit intends to eradicate. But, also remember that the offset voltage that is trying to be nulled, will drift with temperature (and time) and these drifting effects cannot be "cancelled" without re-positioning the potentiometer in your circuit.
Best Answer
Okay, so you've got a transimpedance amplifier (inverting) so there's a virtual ground at the inverting input. That virtual ground might be a few mV one way or the other from actual ground, depending on the Vos. If it's a true current input you'll just see the same offset at the output. You can subtract the offset after the amplifier or do this:
You'll need two suitably(do the math) stable references (plus and minus). Say you have +/-2.5V and you want to be able to adjust the offset by +/-5mV. You can connect a pot across the references (so +/- 2.5V appears at the wiper) then divide that down appropriately say with 100K and 200 ohms (100K should be much less than the pot element resistance) , then apply that voltage to the non-inverting input. If the offset is entirely due to the internal offset of the op-amp the virtual ground will then be exactly 0v when the output is adjusted to 0mV (assuming negligible bias current).
By the way, the offset adjust terminals on an op-amp (where available) should not be used to compensate for offsets that are external to the op-amp, as the are not derived from a reference, the voltage will tend to be something like proportional to absolute temperature (not very stable).