I'm confused -- if you have two 4V solar panels in series, your open-circuit solar cell voltage is going to be 8V, and you want to have a buck converter to get 5V for charging a phone. Hooking up 8V to that Amazon module will likely damage it.
If you have two 4V solar panels in parallel, now a boost converter would make sense to get 5V, and that module could potentially work for you.
Regardless, think about your problem in terms of power. A solar panel with 4V open-circuit voltage, and let's say your 50mA number is short-circuit current, we can back-of-the-envelope as a 200mW device. If you put these both in parallel, you have about 400mW (theoretical) you can deliver to your device, assuming perfect power transfer (which of course we cannot do).
Now, assuming that you are not doing any peak-power tracking and are applying the output of the cells straight into a boost converter to get 5V, assuming perfect everything (which again, is not true), your best case delivery to a phone is 5V @ 80mA...which is not very much. I doubt that's enough to even trickle charge a modern smartphone. If we assume that your energy conversion process is 80% efficient, that's even less transfer -- only 64mA delivered ((0.8 * 400mW) / 5V).
In the scenario where you have placed them in series, you would have the same amount of power available, but you would use a buck converter instead of a boost converter.
So:
- Don't hook up the panels in series to a boost converter.
- You don't have enough power in either scheme to do what you want and charge a phone sufficiently (best case, you charge a phone that is off very, very slowly).
What you refer to as a "battery" is actually a USB power bank, and you must not attempt to charge it with 12V!
The only safe method of charging that USB power bank, is a power source electrically equivalent to a USB port i.e. well-regulated 5V (I will see if I can find the minimum current requirement - check the USB power bank user guide for that information).
Edited to add: Based on the user manual for that USB power bank, it takes up to 5hrs to recharge, and has a 2200mAh capacity internal battery. Therefore it is unlikely to attempt to draw more than the expected 500mA from the external power source (which could be a standard USB port) when recharging its internal battery.
You mentioned "4200mAh" in your question - a typo of 2200mAh perhaps, as it doesn't agree with the webpage you linked?
Also note that not all (in fact I suspect a minority of) USB power banks can be recharged at the same time as also "charging" (i.e. powering) their output device. I did not find any mention of that feature on the web page which you linked - if simultaneously being charged and powering the Raspberry Pi Zero, was actually your intention.
Best Answer
Replace the solar lipo charger board by a USB power bank. The power bank will feed the raspberry zero. It will use it's battery or part of battery power when there too little or no solar power and if there is enough coming for the solar panel it will feed the raspberry zero and charge the power bank until it's full.