Issue charging a 12V battery using solar charger

It sounds like your batteries aren't properly buffering the MPPT output. With the MPPT off, do your accessories still work off the batteries (even momentarily)? If not, you may have a bad cell or open battery fuse.

With healthy batteries connected in parallel to your MPPT, the voltage shouldn't vary more than a few tenths of a volt with a constant load.

You don't want to regulate the MPPT output to 12V, as it would not be able to float charge your batteries at that level. It is generally safe to float charge (constant voltage) a 12V monoblock at 13.5V indefinitely. Specs on your specific battery should identify the ideal float charge voltage as as function of its specific gravity.

If your batteries turn out to be failing, make sure that you're using deep-discharged rated batteries and not batteries intended for engine starting applications. So-called starter batteries are designed for very high instantaneous current, but relatively low discharge depth (you start your car, then they recharge immediately). Your application will likely pull your batteries down to a much higher depth of discharge on a regular basis, so you'll need a battery specifically designed for this use case if you expect a decent working life from the batteries.

EDIT:
Most MPPTs have a relatively low current output at 13.5V - usually less than a couple of amps. The reason that batteries are used in parallel is to allow loads that exceed the capacity of the MPPT. For this to work continuously, the total energy supplied by the MPPT per day/week must exceed the total energy removed from the system per day/week, as the battery just stores/buffers energy; it doesn't create it.

Consider this example (done in amps rather than watts for clarity, even though watts would be more accurate) - Assume an MPPT with a ~1A output is connected to a fully charged battery. Now assume a 10A load is connected to the system. Roughly 1A of that 10A will be supplied by the MPPT and the remaining 9A will be supplied by the battery. The battery's terminal voltage under discharge will define the voltage at the load and the MPPT will adjust its output to match (being a constant power device). This load can't be sustained, as the battery will eventually be depleted. When the load is removed, the MPPT's output will float charge the battery until it is returned to 100% SOC or until a load is re-applied.

Rapid load voltage fluctuations between 12.0 and 13.5 strongly suggest that the MPPT (or the load, if electronic) is being forced into a fold-back self protection mode as a result of a load that exceeds the MPPT output without adequate battery buffering.

This thread may also be helpful.

Both do the same, the LiPo Rider Pro just has everything on board. The 1n4001 diode used in the second instructable is used to prevent battery discharging though the solar panel at night, prevents reverse current through the solar panel when the usb charging (powering the lipo charger through usb) is used, and prevents the solar panel from affecting the charger when both light and usb charging is used. The LiPo rider has the same type of protection.