I have a .75 Watt/8 ohm speaker. Do I need to use an amplifier, what specs should I look for

Don't be so shy.

0.3W is pretty loud! And don't forget that audio equipment potentiometers are logarithmical ones, so when you turn it halfway, it is roughly one tenth of the output power. So a 20W amp, driven with its rated RMS input signal would output 2W RMS when turned up halfway. A quarter setting would result in 0.2W RMS accordingly. And you said you'd want to test with music - rated music power is around 2x the rated RMS power for most of the speakers.

Don't fret, speakers are quite difficult to destroy in so low power situations. I did try with a PC speaker and a ~20W amp when I was young and feeble, to the point when it got quite warm on the outside - but no pops (it was horrible, by the way)... Ok, PC speakers are not too flimsy. I did build a tube amp from scrap parts, and on that I regularly used a way too -electrically- underrated speaker, and to make the situation even worse, I used it as a guitar amplifier - driven into the maximal power tube saturation I could achieve...

If however you'd want a professional approach, use an LM386 power op-amp to play with, a few external components and you are ready to go. I think that is far easier to obtain than an impedance matcher transformator - not to mention, it degrades and colours the sound a lot less.

LM386 datasheet

None of the options is good. The speaker produces a sound pressure level for a certain wattage inputted and this is the criteria you need to look at. Some speakers are more effective than others and this effectiveness is largely determined by their enclosure. Luckily, for any given size and shape of speaker the enclosure is roughly constant.

Decide what SPL you need and look at speaker specs to shortlist a handful then design your enclosure to suit the shortlisted items then do comparative studies. You are obviously looking for bandwidth, loudness and distortion as the key ingredients of the selected speaker. Choose also, the speaker that is capable of withstanding the power output that can be thrown at it. Choose an amplifier that won't melt down ie has protection built in.