Make a brushed DC motor move “free”

It looks like you're missing a few core principles about switching electronics and motor control. I'll go through your schematic and help you focus on what you need to learn.

1. The MOSFET in your schematic is an N-channel type. To turn on an N-channel FET, the gate pin must be a couple of volts* higher than the source pin. In your circuit, if M1 is on, than the source pin will be approximately 24V. So the gate pin would need to be a couple of voltage above 24V. I doubt your PWM signal goes that high, so M1 will never turn on.

(*)I'm intentionally using inexact language to keep the explanation simple.

2. As with all things electronic, there are several options for solving #1. Here's a couple of options listed in order of complexity: You could (a) use an N-channel FET on the low-side of the motor, (b) use a P-channel FET instead, or (c) use the N-channel FET on the high-side with a boost capacitor PWM drive. However, none of these options will give you directional control of the motor.

^{simulate this circuit – Schematic created using CircuitLab}

3. In order to get directional control of the motor, you will need to be able to reverse the current. The simplest way to do that using solid state components is the H-bridge. The most complex part of an H-drive at 24V is the drive circuitry for the high-side FETs. But that's out of scope of this question. D1 in your schematic is good for one-directional control, but will cause failure if you leave it in for bi-directional control.

^{simulate this circuit}

Alternatively, if it doesn't need to be solid state, you can use mechanic switches and/or relays:

^{simulate this circuit}

4. You don't need opto-isolators to drive a motor. However, if the logic side of your device is particularly sensitive to noise or you're doing very high precision analog readings, any effort to isolate an electric motor from the rest of your circuit can only help.

5. A fuse isn't a bad idea, of course. There are other ways to protect the circuit as well. If you do want to use a fuse, choose a slow blow fuse. Surge currents in DC motors can be very high for short periods of time. The fuse can be anywhere in series with the 24V source.

I can't address your question about wigwag potentiometer because I'm not familiar with what that is.

It would be more illuminating if you trigger your oscilloscope off of FET gate -- that should be a cleaner signal.

What's happening is that the circuit will push current into the motor, but it won't pull it out. So when the FET is on, if the motor is spinning any slower than it's no-load speed with 5V applied, the motor will speed up. When the FET is off, the motor will coast along, generating a voltage proportional to its speed. Because the system is lossy, the motor isn't spinning quite fast enough to generate 5V -- but it's pretty close to it. You're reading the average of 5V and whatever the motor generates as it's coasting.