Electronic – Finding cable impedance, to determine RS485 line termination

What's the data rate of your RS-485? Since your cable isn't twisted pair you may have to limit the speed to 100kbps and use short cables.

But IMO the mismatched impedance isn't the real problem.

"The characteristic impedance of an RS-485 cable is 120 ohm according to the standard, but cables designed for high frequency operation usually has an impedance between 135 and 165 ohm, which limits the capacitance to approximately 30 pF/m." (source).

You don't mention voltage or current for the motor's power, but at 200W both may be too high to avoid trouble. A high current will inductively couple with the data wires, a high voltage will do the same capacitively. I expect you may see serious disturbances.

The cable just doesn't seem to be fit for the application. It should have been a double twisted pair at least, preferably with different twist rates.

edit
All will depend on how the motors behave and what your data look like. Have you had problems yet with the deployed cables? Eight noisy amperes coupled over a long stretch doesn't sound good. Bad looking waveforms in itself are not necessarily a problem if the receiver can properly detect edges. What are you sending, UART format? If you have continuously glitches sooner or later they might be seen as a start bit, and you'll have a ghost byte in between your actual data. Not only the data format is important, also what happens at the higher protocol levels. Is there a checksum or CRC? Acknowledge? Can faulty received data be retransmitted? If you would encounter communication problems I think this is the level where you can attack them.
It's also possible that for the installed cables no problems are reported at all. The 57kbps may help here. And maybe there is an error detection/correction mechanism in the protocol.

You must provide a common potential reference, so you need at least one pair for simplex data and one wire for common/GND. Just because it's differential transmission doesn't mean you only need two wires.

The power supplies to the nodes should be floating, and the 0V of the floating supply should be connected to the 0V on the cable. That way the node gets referenced to the master's 0V potential. For electromagnetic compatibility reasons, you should put a 1Meg resistor and a 1kV 10nF capacitor between 0V and chassis on each node, unless your nodes are truly isolated and have no exposed metal.

There's nothing wasteful about using Cat5e cable. It is the most ubiquitous type of high-performance balanced data cable you can find. You can buy it in grocery stores, after all.

Sidebar: If your RS-485 receiver chip is isolated or hooked up to a power/data isolator directly, you could do tricks with just using one pair and locally deriving the reference voltage, but that's making things way too complicated when you have the wires available.