Electronic – AT90USBKEY as USB host

There are enough ways available to provide host mode with a microcontroller that trying to do it with something not intended for the role is "doing it the hard way".

Look for devices with USB OTG" = "On the Go" capability.
These can provide either host of slave capability as required.

If "... something like Atmel ATmega32U2 ... " includes an Atmel AT90USB64 , then yes.

ie using one of the available parts that DO have this capability is the "easy way" to go. There are enough ways available to provide host mode with a microcontroller that trying to do it with something not intended for the role is "doing it the hard way". Look for devices with USB OTG" = "On the Go" capability. These can provide either host of slave capability as required.

ICs with OTG capability of some sort

Cheapest in 1's in stock at Digikey:

AVR microcontroller with USB OTG capability $8.43/1 Digikey in stock.

PIC OTG & Host capable microcontroller $4.22/1 in stock Digikey

The PIC device is also the cheapest host / OTG microcontroller of any sort available in stock in 1's from Digikey.

What exactly are the differences between a USB host and device?

The host initiates all communication on the bus, the device only responds when asked by the host. For Details see the specs on usb.org.

When two devices connect [...]

One must behave as the host and the other as the device. Details can be found in the USB OTG Specs.

Can one device acts as a host as well as a device?

For example, many Android phones and tablets can (requires Android 3+). The "magic" is in the On-the-Go cable/adapter, which switches the phone into host mode by pulling the ID pin to GND.