The ethernet shield is SPI. A quick look at the schematic for the LCD shows that the pins it uses don't conflict with the SPI ethernet pins, except that pin 4 is used as the SS pin for the SD card on the ethernet shield. You have to deal with this. You might be able to cut the track on the LCD shield to pin 4 and wire it to another pin. You'd also have to change the LCD software.
The LCD shield uses analog pin 1 for the buttons, so pick other analog pins for the accelerometer.
Adafruit.com have an LCD 'backback' that allows you to drive those 16x2 displays from I2C or SPI, which reduces pin conflicts.
This is good shield resource http://shieldlist.org/ and it lists the pins for each shield.
It's not the battery that's upside down, it's the entire circuit! Typically you want ground potentials on the bottom (only.)
That being said, I think your circuit will mostly work, except you need to move the resistor (R1) to be between the + output of the solar panel and the Zener(D1)/input(U2) pin, and probably also lower its resistance significantly. Dropping from 8V to 5V at 300 mA happens at (8-5)/0.3 == 10 Ohms.
Btw: To learn more about electronics, analyze the circuit until you see why the current position of R1 and D1 makes the U2 always see the full output of SOLAR, and all R1/D1 does in the schematic is wasting current.
Finally, the MCP1700 is just a linear voltage regulator, with a maximum input voltage of 6V. That's a pretty inefficient way of taking advantage of the voltage that comes from your panel. When the weather is overcast, the panel will provide less voltage than you can use, and you may not charge at all. When the sun is bright, the panel will provide a lot more energy than you can use, and you'll burn it off in all of R1, D1, and U1. I would highly recommend using a micropower harvesting circuit, or at least a buck/boost or SEPIC switching controller that can turn a variety of input voltages to useful output voltage (if not current.)
Finally, LiPo batteries do not like being trickle charged while full. Best case, you metalize the Lithium and the battery dies; worst case you overheat it and start a fire. If you're going to "float" the battery, make sure you "float" it below the "nominal recovery" voltage of 4.05V -- somewhere around 3.85-3.90V would probably be safer. Check the data sheet for your particular manufacturer/battery to get a better indication of where to set the limit.
Finally, if there is sufficient back-voltage prevention in the MCP1700, then it should be safe to leave it as indicated even when the panel is not providing (much) energy. However, many linear regulators, especially ULDO ones, do not like back-power, and thus you may need to add a low-drop diode or a P-channel MOSFET or other switch between the regulator and the battery, with the gate controlled by the presence/absence of power into the regulator.
Best Answer
Sure: Get this and this from Digikey (or their equivalents from your favorite distributor), and mount it to some perfboard! This is another option for an integrated clip/holder. Do you really need the rechargeable Lithium battery you linked? $70 buys a lot of 9-volts.
If you want to go for it, powering your device with a solar panel the size of the Arduino (.004m^2, or 40cm^2) will not be trivial. In the continental US, the sun puts out about 1kW/m^2 at noon on the brightest day of the year, but a solar panel will only be about 20% efficient, and you'll average about 20% of that peak power, so you'll get about 160mW from your shield, or about 3.2mA at 5V. Unfortunately, this power is not available at 5V - A solar cell puts out less than 0.5V. Putting the cells in series is possible, but you can't just draw power from them like a PSU. You'll need a switching regulator, and you'll probably want to bypass the 7805 on the Arduino for this (or tap into the USB Vbus line). Linear Technology makes a number of switching regulators for energy harvesting devices. I doubt that the components required will be availabe in through-hole configurations. You'll also need a battery and/or supercap to accumulate the charge from the panel.
Other mods you'll want to do are to remove indicator LEDs from your board, put the FTDI chip to sleep (pull pin 12 up with a resistor to 5V when you're not programming it), and to make sure any other shields you're using draw as little power as possible. In software, make sure you have no busy waits or polling loops (this may be a big change in programming style), and put the chip to sleep if you're not in an ISR. See this page running an Arduino on low power for years, and this one for some code.
It can be done, but it would be tricky! This would not be a shield for beginners.