Short Answer:
Inductor: at t=0 is like an open circuit
at 't=infinite' is like an closed circuit (act as a
conductor)
Capacitor: at t=0 is like a closed circuit (short circuit)
at 't=infinite' is like open circuit (no current through the
capacitor)
Long Answer:
A capacitors charge is given by \$Vt=V(1-e^{(-t/RC)})\$ where V is the applied voltage to the circuit, R is the series resistance and C is the parallel capacitance.
At the exact instant power is applied, the capacitor has 0v of stored voltage and so consumes a theoretically infinite current limited by the series resistance. (A short circuit) As time continues and the charge accumulates, the capacitors voltage rises and it's current consumption drops until the capacitor voltage and the applied voltage are equal and no current flows into the capacitor (open circuit). This effect may not be immediately recognizable with smaller capacitors.
A nice page with graphs and some math explaining this is http://webphysics.davidson.edu/physlet_resources/bu_semester2/c11_rc.html
For an inductor, the opposite is true, at the moment of power-on, when voltage is first applied, it has a very high resistance to the changed voltage and carries little current (open circuit), as time continues, it will have a low resistance to the steady voltage and carry lots of current (short circuit).
Physics (among other things)!
If you wish to shrink a capacitor in physical size, while keeping the capacitance the same, some other property has to go up, as in every capacitance the actual dimension does matter for the size of the capacitance. In this case you would need to increase the relative permittivity of the dielectric material. As this is a material constant you cannot increase it arbitrarily.
Another noteworthy effect is that you will have a reduced breakdown voltage, so reducing the size will also lead to a reduced voltage handling capability of the capacitor, which is probably unwanted.
On the side of inductors you have similar effects. Shrinking the wire will lead to increased ohmic resistance of the inductor which is generally unwanted (less optimal inductor). You can see that already if you look up an inductor with same inductance but different sizes.
Small wires will also have a problem with the current they are supposed to carry as there is a limit on the current density a material can handle.
Another major factor is the magnetic saturation of the core, which will also limit how small you can make the core for a given inductance.
There are of course practical things like production and handling to keep in mind and you can probably find some more things which will cause trouble. These came to my mind first. Also note that I saw this from an engineering perspective, if you would choose a very minuscule value for C or L, their sizes could become very very small I guess. If you are looking for limits in those regions, I guess Physics SE would be a better place to ask.
Best Answer
An ideal resistor dissipates (converts into heat) electrical power. They are not capable of delivering power. Capacitors and inductors both are capable of absorbing and delivering (positive) power. When power is absorbed by an ideal capacitor, all of it is stored in the form of an electric field. Likewise, all of the power absorbed by an ideal inductor is stored in the form of a magnetic field. These devices can deliver this stored energy, but cannot produce energy.
Real capacitors and inductors, however, are not ideal, and will dissipate some power due to imperfections within the device (leakage within a capacitor, for example). This is why in simulations, capacitors and inductors will sometimes have very complex models to attempt to simulate real-world behavior (such as a leakage within a capacitor, which can be modeled simply with a high-resistance resistor in parallel with the capacitor).