If you intend to use the mic input mic input to your sound card, it will have a current source or pullup resistor to supply a bias to the microphone to which it was intended.
It might not be good for the speaker to have DC flowing through it.
If your phone is connected to the telco, be very careful. The speaker may run at quite a CM voltage and the DC offset might be -48V (depending which way round it is wired).
Telephones wires are very good at attracting lightning. During a storm high common mode voltages may be on the earpiece.
Stricly speaking you should use an audio transformer to couple the earpiece signal into the sound card, this provides DC isolation and some protection against lightning. Adding a TVS across the output of that transformer would be a good idea too.
The ac signal on the earpiece might be quite small. Only a few hundred mV. For a mic input, this will be fine (and can be potted down at the output of your transformer) but for line levels, which are usually 1Vrms it might be very quiet.
Not all phones work the same way, so measure the voltage first using a scope and judge the divsor as required.
The audio transformer should also be protected against DC usign a DC blocking (aka coupling) capacitor.
This circuit protects against discharge and DC currents, though use with caution as a mistake will cost you a PC!

Is it possible to use the power from the headphone jack line out to
amplify the signal in order to get the speaker to be louder?
Here's the thing: The power in the headphone jack is in the audio itself. The power in a normal wall outlet is 50/60Hz AC. The power in a headphone jack is also AC and in the frequencies that you want for your audio! There is no additional power contained in a headphone jack other than the audio itself.
So, if you could somehow convert the power in the headphone jack into something useful and convert it back into a form for driving a speaker, and could do it with 100% efficiency then you would end up with exactly what came out of the headphone jack in the first place.
In short: You can't amplify the power without another source of power. But all is not lost!
What you really want is louder audio, and a way to do that is to make your speaker more efficient. Speaker efficiency is usually expressed as "sensitivity" and is specified as XX dB at 1 meter at 1 watt. A normal speaker is 75 to 90 dB (@ 1 meter @ 1 watt). It might not look like it, but 75-90 dB is a huge range! You can simply get or build a speaker with better efficiency.
Another thing that effects efficiency is the speaker enclosure itself, and the type of speaker. Look into horn speakers. There are horn speakers that have sensitivities up to 105 dB (@1m@1w). Although these will likely be too large for your application, it might be interesting anyway.
Driving an 8 ohm speaker with a headphone jack is questionable. Some devices will do this just fine. Some devices might distort but otherwise be fine. And other devices might be permanently damaged. Unfortunately there is no good way to find out without just trying it.
In theory you could make an impedance matching transformer that would show a proper load to the headphone amplifier, but who's output is good enough to drive an 8 ohm speaker. Just keep in mind that a transformer will not increase the power, only convert the form of that power so different devices can play nice together. I have not seen such a transformer for sale, but I also have not looked for one. But in theory you could make one.
Given all the different issues, if it were me I would figure out a way to make a small battery pack with an amplifier in it. Yes, it's bigger than what you want. But it will probably be the best compromise between volume, size, cost, and robustness.
Best Answer
You should be able to utilize the phones internal button used to activate the hang up action to disconnect the "sense" function of the USB audio device. ie; tell the computer that you have disconnected the headphones and it should send the audio through the original default device, the desktop speakers.
You would want to like you said adjust the volume on the phone so that the levels would stay even and comfortable, I would suggest installing a 10k logarithmic potentiometer as a voltage divider to simply cut down the volume going to the handset speaker.
If you are going to be using a button for push to talk you should not have a problem with putting the phone down, it will already be muted when you release the PTT.
There should not be any need for an Arduino here, maybe a couple op-amps if your mic level is to low.
Sorry I dont have more time to point you to some components.