You're going to be hard-pressed to compress all of this functionality into any smaller of a package. Let's go through what you're trying to accomplish:
Regulate down from wall wart power (most likely 9V-12V) to clean, regulated 5V
This is easy and could be accomplished a zillion different ways. Current draw and input voltage is what really influences your choice here. You can easily get a small package linear regulator, but if the input voltage is too high, you start needing bigger and bigger packages to handle the heat, and you can get up to D2PAK and still be throwing too much heat. Linear regulators handling high input voltages is usually sucky for any moderate output current.
In this case, you need to step up to a switcher so you can avoid these heat issues. As far as the smallest package/simplicity, I have used the TPS84250 from TI in a design. About 14mm x 14mm of board space and 7V-50V input with 2A of output current and adjustable output voltage. They are very expensive compared to the raw components (switching controller, inductor, diode, etc) at $10 - $13 per piece in low quantities, but we're talking about simplicity here, right?
There are similar switcher designs in the TI Webench design center (output current / board size wise) that can be built for much cheaper, but then you're using more components and spending more time on layout. It's going to be a trade-off.
Select between regulated 5V and USB VCC for input to our 3.3V regulator
There are also a few good ways to do this... mostly either using discretes (diodes) or MOSFETs. There are even some power controller ICs with the MOSFETs built in. Can't beat that for integrated/small. Again, a favorite part of mine is the LTC4415 from Linear Technology. This IC will OR two power sources for you with its integrated MOSFETs, and prioritizes one of the inputs automatically for you. It also lets your set independent output current limits for each output so you can configure the USB input to match your 500mA limit, and the wall wart current limit to match your switcher's output current limit, etc etc. Board space consumption here is pretty small.
Again, a little pricy... these badboys are like $3 - $5 in low quantities but they do make the prioritized power source requirement pretty straight forward.
Regulate down from 5V to 3.3V
This portion is pretty obvious. Find the smallest package size with enough output current. Optimize in pricing, etc. Done.
Other Considerations
I know you mentioned wanting to eliminate components, but don't forget that you still want system-level protection against component failures... i.e. a fuse in front of the switcher in case the switcher goes haywire somehow and short circuits. Same thing goes for USB power. Your device should be doing its best to play nicely with all systems and signals it integrates with.
So to confirm - you want a LiPo charger + a buck-boost regulator to 3.3V (primarily cause you want to be power efficient).
So you have a ton of options. If you want to go with the Microchip battery charger that comes with the Sparkfun breakouts (https://www.sparkfun.com/products/10217) or Adafruit breakouts (), there are a ton of buck-boost regulators you can put in the next stage. I personally use the TPS63031DSKT in one of my designs, and I'm using the exact same schematic as in the first page of the datasheet (http://www.ti.com/lit/ds/symlink/tps63031.pdf). The efficiency curve is great for the Iout range you are expecting.
However, if you are starting from scratch, I'd look for a PMIC that combines the two like the LTC3558 from Linear (http://www.linear.com/product/LTC3558). That will really help cut down on board space, and cost. Really great chip - all its missing is a battery gauge. I know that MAXIM makes similar PMICs but I can't find it right now.
Best Answer
A device like the Power Integrations LinkSwitch-TN allows you to implement a simple buck converter that converts rectified mains voltage to a reasonably-well-regulated non-isolated low voltage with reasonably good efficiency.
If you need low ripple after the fact, you can put a linear regulator on the stepped-down output of the buck and minimize your resistive losses.
I've used these as primary-referenced control supplies and have had good success with them.