I would say that these are indeed non-polarized caps. You would want to replace them with similar capacitors. Do NOT replace them with ceramic caps.
Ceramic caps are great for many purposes but not in the audio path. Among other problems, they suffer from piezo electric sensitivity and their value can vary greatly depending on the magnitude of the DC voltage across them.
Digikey and Mouser have suitable replacements available.
Just a single switch is not enough. Actually, two switches is not enough. You need a memory element, such as a relay, plus a switch at each end of your weight travel.
The following will do what you need. It consists of a relay (double pole, double throw), two limit switches (one normally open, the other normally closed), and a diode.
simulate this circuit – Schematic created using CircuitLab
The vertical travel of the weight is shown on the left, along with a protruding arm which will contact SW2 at the bottom of its travel, and SW1 at the top. The motor is wired so that with 6 volts on its + lead, the weight rises.
Start by assuming the relay is off. The two sets of contacts are active on the right, so +6 is applied to the - motor lead and the weight is descending. Because the motor + lead is at ground, there is no voltage applied to the diode.
Now let the arm on the weight activate SW2. +6 volts flows through SW2 and SW1 and applies power to the relay. Now the contacts to the left are active and two things happen. First, +6 is applied to the motor + lead, and the weight starts rising. Two, +6 volts flows through the diode, and also tries to drive the relay. So when the weight starts to rise and the arm no longer closes SW2, the relay is still driven and the motor continues to drive the motor to raise the weight.
This continues until the arm contacts SW1 and opens the relay circuit. With no power to the relay, the contacts shift to the right and the motor reverses direction. The weight starts down. Since the motor + lead is now connected to ground, there is no voltage on the diode, so when the weight drops and SW1 closes again, there is still no drive for the relay coil.
The cycle will continue as long as the battery holds out.
Speaking of which, I really recommend either much bigger batteries or a power supply you can plug into a wall. I believe you'll find that you drain AA cells much faster than you think.
Best Answer
You connect the positive terminal of the battery or power supply to the positive terminal of the circuit.
For components, the positive terminal is generally connected to the "more positive" point in the circuit, which may not be directly connected to the positive power supply.