I suspect there are lots of ways to do this. One that comes to mind is callback.
You register a dialin ISP account for the modem on the server; that's simple enough, and non-contentious. You will need an ISP that will allocate you a static address; if this costs extra, pay it.
You then configure something like diald to bring the link up and down on demand. Then write a little script, perhaps using swatch, to detect "RING" messages appearing in the modem logfiles, and when it sees them, to generate traffic that diald considers "interesting" - ie, traffic that causes it to bring up the line.
When you want to connect to the server over the regular internet, you call the phone line attached to the server, from any landline or GSM phone. The script on the server detects the phone RINGing, and generates interesting traffic for diald. Diald brings up the PPP connection, at which point the server is available on the known, static IP address.
You, using regular internet tools, connect in via SSH (or any other IP protocol or combination thereof which pleases you), until you're done. When you're done, you drop all these connections, and after a short while diald realises the line is idle and hangs up the modem.
Rinse. Repeat.
Possible enhancements: if you can't get an ISP service that does static addresses, you could configure diald so that the first thing that happens after the internet link comes up is that it emails it's current IP address to a pre-arranged address, or in some other way communicates it to the internet. That way you can find out what dynamic address it's got each time the line comes up, so you can talk to it. But a static address would be easier.
Alternative suggestion: consider that most GSM phones will happily make a PPP connection to any old POTS number, to establish the terminal session. I wrote this technote some years ago on the nuts and bolts of getting a GSM phone with a modem that honoured the AT command set (which is most of them) to dial in to an ISP.
It's completely legal to have the GSM phone dial a standard landline instead of the magic "bring up GPRS" number, just replace
atd*99***1#
with
atd0039nnnnnnnn
in /etc/ppp/chat-gprs, where nnnnnnnn is your server's landline number. Note this will only work with GPRS, and maybe EDGE. 3G is a whole different protocol accessed in different ways, so you'll only get a slow (20kb/s or so) connection this way. But for a straight terminal session, that may be enough.
Best Answer
To save your sanity, don't.
Modems (and fax) over VoIP are extremely sensitive to timing, and usually break over VoIP, it might even seem to work at first, but it's certain to break.
Adding virtual machines just makes the problem worse.
There are "short haul modems" around which can emulate a phone line.
Another option is just to attach the two modems back to back:
http://www.jagshouse.com/modem.html