This is the universal conundrum of parallel vs. serial connection of cells/batteries, for both discharge, & re-charge. There are pros & cons for each scenario - efficiency/losses (P=I^2.R), safety (Lithium is an unforgiving mistress), convenience / accessibility, longevity, equalisation.
RC hobbyists trade convenience with a willingness to disconnect packs from their gear, and then connect them to a charger, which gives them flexibility to arrange cells in whatever series or parallel arrangement makes best sense for each discharge & recharge phase of their play-flow (work-flow - geddit? ;).
As you say, your application doesn't facilitate this physical flexibility - the cells aren't easily accessible, nor is changing harnesses. You could engineer a complicated switching scheme with either relays or mosfets, to allow you to reconfigure the array for discharge & recharge, but that's a lot of work, a lot of cost, and extra copper/mosfet losses.
If it were me, I'd suck it up & go for a Li-Po recharge solution that supports the number of series strings with cell/pack-balancing capability (which of course is critical in series configurations). That may mean you need to take your power source & use a switching boost voltage regulator to get the voltage you need (i.e. more than the number of series cells * 4.2V) to achieve this, and then a charger to match.
If you really want to stick with hobbyist-level chargers like the one in your photo that can handle a handful of cells in series, then one approach I can think of is to power each of those rechargers via isolated DC-to-DC converters (in turn powered from your solar/MPPT supply, or bulk mains-powered PSU) - that lets each charger handle a handful of cells in series, and provides the isolation needed between each charger as the string voltage escalates along the string. Here's some examples of the kind of DCDC modules I mean. One challenge of this approach is how to control each of those rechargers electronically (instead of humanely) whilst maintaining the isolation needed between them, but that's a separate question :).
Best Answer
Your circuit doesn't do what you want it to - both batteries are permanently wired in parallel, as is the charger. The diode does nothing.
You probably don't want to charge them in series from USB anyway, because you would need at least 5.6V (1.4V/cell x 4 cells) but you only have 5.0V available. Charging in parallel can be done efficiently with a switch-mode current regulator, or just a resistor to limit current (and perhaps a blocking diode to prevent discharging into the USB port when not powered) if you don't mind a lower charge rate.