I wouldn't go so far as to call PID outdated. But there certainly is room for improvement. One way in which I have auto-tuned PID control loops is to use the Nelder-Mead method which is a form of hill climbing simplex algorithm. It has the benefit of being able to converge and reconverge on a target parameter that moves over time.
From this paper:
For example in our case of PID parameters tuning {KP,
KI, KD} a simplex is tetrahedron. Nelder–Mead
generates a new test position of simplex by extrapolating the behavior
of the objective function measured at each test point arranged as the
simplex. The algorithm then chooses to replace one of these test
points with the new test point and so the technique progresses.
My particular application was for motor control. We had two loops, a PID current control loop and a PI velocity control loop. We set our vertices to P, I, and D respectively and ran statistics on the output of the loop. We then ran the reflection, expansion, contraction, and reduction over and over again until the current or velocity control targets generated were within a few standard deviations.
With our product, the VP was very concerned with how the motor "sounded". And as it turned out, it "sounded" better when the current target bounced a bit more than was mathematically optimal. So, our tuning was done "live" in that we let the algorithm seek while the motor was running so that user's perception of the motor sound was also taken into account. After we found parameters that we liked, they were hard-coded and not changed.
This probably would not be ideal for you since you state, "putting the system in oscillation even as a part of auto-tuning is not acceptable to the users". Our system would most certainly oscillate and do other horrible things while it was auto-tuning.
However, you could run two copies of the PID controller. One that was "live" and actually controlling the process. And a second that was constantly being auto-tuned while being fed the same inputs as the "live" controller. When the output of the auto-tuned controller became "better" or more stable, you could swap the coefficients into the "live" controller. The controller would then perform corrections to the process until the desired performance was achieved. This would prevent oscillations that can be perceived by the user during auto-tuning. But if the inputs change drastically and the PID controller is no longer optimal, the auto-tuning can swap in new coefficients as they become available.
This is talking about a simple proportional controller. The control output is merely the system output minus the control input times some constant:
C = K(I - S)
Where I is the control input to the system, S the actual system response, C the control output that drives the system, and K the proportionality constant.
The tank example above is a little squirrely. Let's consider speed control of a car as a example. In that case, I is the speed you want to go, S is the speed you are actually going, and C is how hard you step on the gas. Let's say you want to go 50 MPH but the car starts at stop, which mean S is 0. Let's say K is 2, and the resulting C is then in percent of full throttle. Initially K(I - S) is 100, so you step on the gas all the way (100%). As the car speeds up, I-S gets lower, so you let up on the gas. That makes sense because you know intuitively you don't need to floor it to maintain 50 MPH on level ground with no wind.
However, the problem now is that you won't ever get to your desired speed. If you were at 50 MPH, then I-S would be 0, and you'd let off the gas completely, which clearly won't maintain 50 MPH. This simple proportional-only controller always requires some error to maintain a non-zero output. For example, let's say to maintain 50 MPH would require a 25% control output (step on the gas 1/4 of the way), and that steady state speed is linear with throttle setting. In that case, the system would assymptotically approach 40 MPH and stay there. K(I - S) is 20, which is the control output required to maintain 40 MPH.
One way to address this problem is to make K larger. However, that will make things unstable or jerky. Suppose you take this to the extreme and make K infinite. That means you either fully floor it or fully let up on the gas, depending on whether you are below or above 50 MPH. You can probably see intuitively that it would be a very jerky ride. While your speed would probably average reasonably close to 50 MPH, the car would constantly go a little slower, which would floor the gas, which would make it go faster after a little while, which would let off the gas, which would make it slower after a little while, etc.
Obviously this is bad for the car, uncomfortable for the passengers, and a inefficient way to run a gasoline engine. However, some control systems work quite nicely like this, especially when the system can only be on or off in the first place. The heating system in your house works like this, for example. The thermostat turns the furnace either fully on or fully off. The temperature does oscillate a little, but not enough for you to care. In this case, the heater only works full on or full off, so this is the efficient way to run it. To keep from turning the heater on and off too often, the thermostat has a little hysteresis. It might, for example, turn on the heater when the temperature gets down to 69.5° and off when it gets up to 70.5°. This keeps the heater on and off for long enough at a time to not stress it too much and for it to stay efficient.
In other control systems, this offset is dealt with by adding other terms to the equation above than just one that is proportional to the offset. In a PID (Proportional, Integral, Derivative) system, there are additional terms proportional to the time integral of the offset and the time derivative of the offset. Each of these terms has its own gain factor (K value in the equation above). It is the I term that nulls out any long term offset.
Best Answer
PID implicitly expects to be controlling a linear system. It seems that your system is not linear. If you ignore that, you will end up tuning the system for sluggish operation over one part of the range to ensure some other part of the range remains stable.
The first thing to do then is to linearize your plant. Measure a bunch of valve settings and resulting temperature, with whatever is producing the heat held constant. Then invert that table and put it between the PID controller and the valve setting. Now the PID controller thinks it is driving a linear system.
As for tuning the PID, there are whole books on that. There are equations you can use to get the PID coefficients if you know some things about your system. However, in practice that just gets you a starting point. Real PID tuning always includes some experimentation.
Since your system might be rather slow, it would help to get a lot of the PID tuning done on a simulation. If you can linearize your plant, then you can use the method I describe in How to use measured step response to tune control system.