You seem to have the whole circuit in LTspice anyway. A start-up analysis will tell you most things you want to know. Replace your "big" (45 V) DC source with a source that has a pulse definition, i.e. one that starts at 0 V and steps to 45 V within a short time (say 10...100 ns), after a short time (say 1 µs). That way, all the capacitors will be initialized for an unpowered circuit, and you see your regulator doing it's very best to charge the output capacitor. Using this setup, you get the whole picture: First, the uncharged output capacitor produces a dead short across your output, so you see your regulator starting at its max. current. Once the voltage at your output capacitor reaches the desired value, you will also be able to observe any possible overshoot.
An alternative approach would be to include a current source (actually, sink) at the output, stepping between 0 A and your max. desired output current.
As a rule of thumb, I would start with 1000 µF per 1 A of max. designed output current and try (".step param") values below and above (10 µF, 47 µF, 100 µF, 470 µF; 4.7 mF, 10 mF). Also, things won't become too critical: Your pass transistor is an NPN, and this design is basically stable anyway (as opposed to an LDO, which uses a PNP pass transistor). A stability analysis of your circuit might really be a good idea; even though your schematic looks a lot like a linear regulator with a common collector pass transistor at first glance, you really have a common emitter circuit, and those tend to be unstable. The reason is that the output impedance of a common collector amplifier is roughly the transistor's base driving impedance, divided by the transistor's beta and this value does not change in any significant way when the load varies, and it is low. On the other hand, a common emitter ampifier's output impedance is defined by the load itself, which stays within a certain range at best, but can't be designed into the voltage regulator itself, of course. (*)
Here's a source with a really good explanation about a linear regulator's stability, but we have to swap "PNP" and "NPN" in our example, because we are not (!) dealing with the same circuit here. For the "ususal" way the pass transistor is wired in linear regulators, the quote is: "The PNP transistor in an LDO regulator [...] is connected in a configuration called common emitter, which has a higher output impedance than the common collector configuration in the NPN regulator." (National Semiconductor - now TI - app'note AN-1148, section 9)
(*) Had to edit my first version of the answer because I had overlooked some important issues. As can be seen in some comments to other posts, the problem has to do with repairing vintage lab equipment, and you can never learn enough from fixing stuff. Here's an excerpt from Jim Williams' article "The Importance of Fixing", as published in the book ART & SCIENCE OF ANALOG CIRCUIT DESIGN:
Oh how I like the part about fooling yourself...
Although we've had a few related questions before (see my comments), none of them seem to directly address this question.
why would the manufacturer spec a minimum ESR?
Remember that a linear regulator is a kind of feedback circuit. The resistance of the pass device is adjusted until the voltage returned on the feedback pin matches an internal reference voltage (in the case of a fixed-output regulator, the feedback could be all internal to the chip).
The output capacitor produces a pole in the open-loop response of the circuit, which comes with 90 degrees of phase lag above the pole frequency. The ESR then produces a zero, and a tendency toward phase lead, that (in certain designs) is necessary to ensure the total phase lag doesn't reach 180 degrees at the point of unity gain, thus avoiding oscillation according to the Barkhausen criterion.
I'm asking why a manufacture puts what I feel is contradicting information in their datasheet
It's not contradictory. You can use this regulator with a ceramic output capacitor. But you must be careful to select one with high enough ESR.
For example, here's a part with 1 uF and about 9 mOhms ESR. With a bit of narrow trace between that part and the regulator output, you'd be likely to get stable operation.
ESR also tends to increase below (and above) the capacitor's resonant frequency, so depending where the regulator control loop's crossover frequency is, it might be much easier to locate a ceramic part with adequate ESR at that frequency. Unfortunately they never tell you what frequency you ought to be looking at.
Best Answer
All voltage regulators want a peaceful life; if there is an output capacitor then any fast cyclic changes in load current are largely dealt with by that capacitor and the lazy output from the regulator is only expected to deal with topping up the charge to the capacitor so that the capacitor can handle the next cycle of load current change. There will be a small transient change of output voltage because the lazy regulator can't be expected to keep up with fast load changes. That is due to: -
$$I = C\cdot\dfrac{dv}{dt}$$
In other words, there will be a ramp down in voltage depending on how high the current is and how big or small the capacitor value is. And the lazy old regulator will try and deal with sorting out the average output voltage in its own time.
In other words, voltage regulators are not as quick to deal with load changes as you might think and therefore, the output capacitor does the main job of keeping dv/dt as slight as possible. Some voltage regulators are better than others of course but they are, after-all, a control loop and won't be as effective in the short term as an output capacitor. Having said that, output capacitors with relatively high ESR are going to cause problems too so, use the capacitor as recommended in the data sheet.
So, instead of the voltage regulator having to deal with step changes in current (without a capacitor) it has to deal with dv/dt (a much slower change).
As for the input capacitor, if there is a load change that causes the input voltage to fall suddenly, the lazy voltage regulator is not well equipped (speed wise) to adjust its series impedance to compensate for that input voltage fall in order to keep the output regulated. Having a capacitor at the input holds-up that voltage to a higher degree than if it were not present. This allows the lazy voltage regulator time to catch up with the situation that is occurring.
The same story if the input voltage changes due to external factors - having an input capacitor can slow down that change and make life easier for the lazy voltage regulator.
So, both capacitors are there to make life easier for the voltage regulator and deliver the performance expectations stated in the data sheet.