The answer is to use a charging chip that has the ability to back off on charging current when the input voltage from the solar panel begins to collapse.
While using a regular Li-Ion charging IC will technically "work" with a solar panel, it will take forever to charge in practice, and, as mkeith pointed out: "it will never get a chance to terminate charging".
The discussion about this on the Adafruit website is the best explanation of this phenomenon that I can recommend to others with this question:
Most people try to plug a solar panel directly into a lipo charger and while it sort of works, the battery takes forever to charge because the efficiency is terrible! That's because most lipo chargers are meant to plug into a USB port or wall, and are very simple in their design. USB ports supply 5V at up to 500mA and they're pretty solid - the voltage doesn't change much even at the max current draw. So when you plug a charger into a computer with a USB port, they just draw 500mA or so and happily chug away. Same goes for wall adapters. The voltage and current limits are kept steady.
Solar panels are a little different, the voltage and current vary constantly depending on sunlight available. They are unstable! That instability confuses battery chargers, which causes them to do one of two things: rapidly turn on and off as they try to draw more current from the panel than possible and/or draw much less current than they can, to keep the voltage from collapsing
Yes you can connect several Li-Ion cells in parallel but before you do so, check that they have (almost) the same voltage.
If you buy several at the same time from the same supplier, changes are this will be the case.
If the voltages are more than 0.2 V (I just sucked this value out of my thumb !) different, you have to balance the cells. Either charge them fully with the same charge, after charging they will have the same voltage. OR you connect them in parallel but with a small value resistor between both + poles. A 100 ohm resistor will do. This resistor will limit the current flowing from one cell to the other while they balance themselves. When there's 0 V across the resistor left, the cells are balanced and you can connect the + poles also without the resistor.
If the 2.5W solar panel will be enough, depends on your patience ! Using 2 cells will double the charging time. Using a solar panel with double the power will halve that charging time again. But the charging circuit you're using can only supply up to 1 A so it makes no sense to use more than 2 2.5W, 5V (so 0.5A) solar panels.
Panasonic are excellent batteries ! Also Samsung and LG make excellent batteries. I would not recommend most cheap UltraFire. Either get cells with "solder tabs" to connect them in parallel and to connect wires to them OR get cells without "solder tabs" but then get a battery holder. You should avoid to solder directly on the battery.
I would recommend getting "protected cells" (these have a small battery protection circuit) without "solder tabs" (most protected cells do not have these anyway). And to use a battery holder, if you ever need to replace the batteries, it will be easy.
Best Answer
Parallelling panels: This can be done to increase the maximum producible current, but generally good practise to put a diode in series with each panel's output to deal with the situation where the panels are not equally illuminated. One panel that is in bright sunlight, and the other that is in shade, the panel in sunlight will be driving current back down into the panel in shade and potentially could damage it, so put some diodes in to ensure current can't flow back into the panels. Some panels come with built in diodes for this, and some don't. Chose a diode with a low forward voltage drop (you don't want to waste too much energy as the panels are naturally inefficient anyway).
DC-DC converters are not necessarily the device you want to use. You need to understand the difference between DC-DC converters, voltage regulators.
Often what you really want is a low drop out (LDO) voltage regulator to keep the output at a nice steady level, regardless of what voltage is coming out of the PV panel. Of course, if the sunlight intensity drops too much, then the output voltage of the panel will drop so low that the voltage falls below the minimum input voltage for the regulator and the regulator will not be able to maintain its output at the specified regulated output voltage.
It's difficult to maximise the power output from a solar panel because the input voltage can vary so much, you might want a 5 volt steady output but find the panel voltage can vary from say 3V to 12V. When the sun is bright enough to produce 6 or 7 volts or more from the panel, then you'll get a nice steady 5V on the output of the regulator, but when the sunlight intensity drops and the panel only produces 5.0V volts or less, then the regulator can't function and the output drops, but the panel is still producing power but you can't use it. This is where you need to start thinking about Maximum Power Point Tracking (MPPT) solar charge controllers. (I haven't yet seen a decent accurate explanation of how they work on the internet).
To answer your specific question, why can't you remove the DC-DC converters? As I indicated earlier, I'm not convinced they are really the type of device you want to use. If you remove them (or remove a voltage regulator), you may find your panel output voltage goes too high and causes damage to the device it is powering.