Note that the LMC6772 chip you are using is not an opamp; it's a comparator that has an open-drain output. Since it can't actively drive its output high, the Schmitt trigger circuit won't work as designed.
The datasheet says, "Refer to the LMC6762 datasheet for a push-pull
output stage version of this device."
However, look at how you have the optocoupler wired up: the anode of the LED goes to the output of the comparator, and the cathode goes to ground. This means that when the comparator output is "high", the LED is driven by the current coming through your resistor network, and when it is "low", the diode is shorted out. This also means that the "high" voltage at the output of the comparator is the forward voltage of the LED. (According to the datasheet, this is at most 1.8V)
Do NOT switch to the push-pull version of the chip unless you also add a current-limiting resistor in series with the LED.
Note also that the website you used to calculate your resistor values is assuming that the opamp in question has bipolar supplies, and that the output swings to -Vcc when the output is "low". Your opamp is being powered by a singled-ended supply, which makes the lower threshold 7.56V when the output is low.
So, the actual threshold voltages of your circuit are 7.92V when the output is at 1.8V and 7.56V when the output is at 0V.
The general equation for the the voltage at the junction of three resistors, each fed by a voltage source is:
$$V_{junction} = \frac{\frac{V_A}{R_A}+\frac{V_B}{R_B}+\frac{V_C}{R_C}}{\frac{1}{R_A}+\frac{1}{R_B}+\frac{1}{R_C}}$$
In your circuit, one of the three resistors is always connected to ground, so you can ignore that term in the numerator. If we plug in the voltages and resistors you used:
$$V_{junction} = \frac{\frac{12 V}{150 \Omega}+\frac{1.8 V}{470 \Omega}}{\frac{1}{150 \Omega}+\frac{1}{470 \Omega}+\frac{1}{560 \Omega}} = 7.92339 V$$
$$V_{junction} = \frac{\frac{12 V}{150 \Omega}+\frac{0 V}{470 \Omega}}{\frac{1}{150 \Omega}+\frac{1}{470 \Omega}+\frac{1}{560 \Omega}} = 7.56141 V$$
But you're right, these thresholds should work fine in your application, and they don't explain the problems you're seeing. I can only surmise that you're seeing some secondary effects arising from the fact that you're trying to operate a "micropower" comparator at some fairly high voltage and current levels.
This might not be efficient, but works.
You can go for a voltage reference IC like this one.
This can be used as supply voltage for your Schmitt trigger IC alone. You can scale down the threshold of 13.8 V and 13.0 V to 4.6 V and 4.3 V respectively (or any other values), using high value resistor divider at the input side of the Scmitt trigger(So that, it doesn't waste current there). The UTP and LTP are now referenced to an independent constant voltage reference from the IC. The inputs however will vary depending on battery voltage.
Best Answer
The circuit acts like a comparator. If you want to show VL at the output you have to push the input voltage over the threshold, because the "-" input shall be greater than the "+" input. The threshold to exceed is the voltage on the "+" input of the op-amp, that is the application of the Kirchhoff's voltage law. Suppose that the output is in the high state: $$V_{out} = V_H$$ then the voltage on the "-" input shall exceed the voltage on the "+" input, that is: $$V_{UT} = {R_1\over{R_1+R_2}}\bullet V_H$$ This is why the upper voltage is used