R4 is needed to form the reference voltage with D2 by providing the bias current. The intersection of the V_in, R4 with the I-V curve of the Zener diode determines the bias point of that zener. If R4 was not present, D2 is only connected to the base of the BJT through R5, which would require negative base current.
Regarding your observation of a higher voltage at the output than at the input. From a theoretical standpoint, it should not be higher. Troubleshooting is all about breaking the problem down into smaller blocks and checking your measurements against your expectations.
Here's what I would do to troubleshoot the system.
Disconnect everything from the input, check V_in, does it match your power supply setting?
Double check the orientation, connections, and component values of all components just as a sanity check.
Measure the net between R4 and D2 while disconnected from the R5, Q2, Q1. What this is doing is checking where the reference level is at. This should be near 2.5 V assuming your input is 2.5.
Measure the base of Q1 and R6. This should be at 0 V as Q2 is off and R6 is pulling to ground.
If any of these voltages are off, you can look deeper into that area and try to determine what is going wrong.
Regarding charging supercapacitors. You can charge supercapacitors in many ways, constant current, constant power, etc. The design of the charging circuit should protect against overcurrent and reverse voltage.
A simple Shottkey diode and current limiting resistor is will work. This isn't the fastest way, but it gets the job done. Reference [1] shows an example of this circuit in Figure 2a. Size the current limiting resistor such that V_in / R is less than the maximum current rating of your supercapacitor.
If you want the overvoltage protection, you can put the Shottkey diode in front of your overvoltage circuit and the resistor on the output. The Shottkey will protect the overvoltage circuit as well. Before you do this, solve whatever problems you're having with the overvoltage circuit first. Don't add more complexity to troubleshoot.
[1] http://www.ti.com/lit/an/slva920/slva920.pdf
The MM3Z3V0T1G is designed to "regulate" to about 3.3v when 5mA is driven through it.
3.3v * 0.005A = 16.5mW typical quiescent dissipation. The package can dissipate a maximum of 300mW of power. 300mW/3.5v = ~86mA maximum sustained current.
To get 5mA to flow through it, R45 will need to be dropped from 10kΩ to ~0Ω because the supply voltage is the same as the regulation voltage... but then it will be impossible to drive the input pin low.
Try R45 = 220Ω and R10 = 0Ω. This should give >3v to the input when the switch is not active, and 3.3v/220Ω = 15mA through the switch when closed.
Reducing R45 and R10 will drastically negate the effects of C36 to "debounce" the switch. Either increase C36 or create a separate R-C filter after the zener.
Best Answer
Suppose the relay interrupts the motor current, 20 amperes, in 20 nanoseconds.
Suppose there is a wire running from the relay to the motor controller, and that wire couples into the RPi regulated power; the wiring between regulator and RPI forms a loop with VDD and RTN. <- EDIT
simulate this circuit – Schematic created using CircuitLab
Suppose the distance from wire to loop is 1cm. Suppose the loop area is 1cm by 10cm (4"). What voltage is induced, on top of the regulated RPi VDD?
$$Vinduce = [MU0 * MUr * Area/(2 * pi * Distance)] / dI/dT$$
What is the induced voltage? anything higher than 1 volt transient is bad news. Now insert value for MU0 of 4 * pi * 1e-7
Vinduce = 2e-7 * area/distance * dI/dT
Vinduce = 2e-7 * 1cm*10cm/1cm * 1amp/nanosecond
Vinduce = 2e-7 * 10cm * 10^+9 amp/second = 2e-7 * 0.1 * 1e+9
Vinduce = 2e+1 = 20 volts induced
To avoid this (e.g. reduce the induced voltage from 20 volts to 0.2 volts),
(1) keep the MCU PCB a meter away from the high-current fast-off motor wires (hot and return wires).
(2) place a steel plate adjacent to the MCU PCB, insulated to prevent shorts; have the plate be larger than the MCU PCB
(3) insert a PI filter (C-L-C) at the MCU PCB, as shown
simulate this circuit
The one Ohm across inductor is for dampening. Rdampen = sqrt(L/C) =sqrt(1mh/0.1mF) = sqrt(100 = 3.16 ohms ideal. The 1 ohm is over dampening.