The acceptable range of pot value substitution would depend on the rest of your design:
In audio applications, volume control pots could, for instance, affect the input impedance (or not) or other performance parameters. If the pot is being used as a simple voltage divider, then you would not want to decrease values too far i.e. increase the current draw across the pot too much compared to design.
At the same time, increasing the resistance too much would introduce an additional noise source, and may decrease the designed linearity of the divider - such as if the input impedance of whatever is reading that divided voltage, is of comparable magnitude to the pot's resistance. That's like throwing in a third resistance in parallel to one of the arms of the voltage divider, it's no longer a simple voltage divider.
Another parameter especially significant in audio designs is whether the pot is expected to have a linear, log or other more exotic taper (rate of change of resistance with wiper rotation / movement). Substituting one type for another will almost definitely introduce odd behavior.
As a general rule, one would substitute for a quick experiment, but not for a production run, without revalidating the design for the new value. Such substitutions should retain the taper of the original pot. On value range, it is a personal preference: Some would say do not go over twice or below half the design values, perhaps that too is pushing things to the edge.
Finally, you won't know till you try - and sometimes letting the grey smoke out of your circuit is a good indicator that the substitution was too far beyond acceptable range.
To answer your 500 Ohm for 100 K substitution query: Don't. That's a good candidate for smoke effects.
The circuit in the Commodore Paddle is connected as an R/C (resistor-capacitor) circuit. It uses the combination of P1 and C1 to form an "RC time-constant" where the computer discharges the capacitor, and it times how long it takes for R1 to charge-up the capacitor. The lower the resistance of P1, the shorter time it takes to charge C1.
And the other section P2 and C2 work exactly the same way. Alas, the variable resistor is NOT used as a true potentiometer. If it were, then you could substitute a 100K digital pot without any problem.
If you want to substitute a resistance of 100K instead of the original 500K, then you would need a corresponding capacitor that was 5x as big to maintain an equivalent RC time-constant, at least in theory.
It seems like a rather awkward way of controlling the paddle inputs. You didn't mention what is controlling your digital pot(s). It may be more expedient to replace the pot with a field-effect transistor (FET). Or simply monitor WHEN the Commodore discharges the capacitor and then wait an appropriate length of time (proportional to the "position" to apply a voltage to SIMULATE the RC time constant charging.
Best Answer
Perhaps you're visualizing that the pots merely affect the control voltages fed to voltage controlled attenuators, one for each channel.
However, it is most likely that the pots actually form two separate voltage dividers feed by the analog signals for each channel.
If you combine the L and R signals in a single pot, you will no longer have separate L and R signals; each channel will carry a L + R signal.
As Kaz points out, it is also possible that you might damage the stages feeding the volume control.