You mentioned that you cannot access your server "from a location outside the house". Does it mean that you cannot access your home server from any location, or just from work?
If you cannot establish an RDP session to your server from any location, and assuming that everything is properly configured, I'd suggest switching RDP default port (3389) to another one.
For example, try using port 33890 on your terminal server. I've seen this same problem a couple of times, and it happened because the customer's ISP was blocking traffic on port 3389. If the ISP blocks traffic by port number, then changing the default port to another one will let you solve this issue. However, if the ISP is filtering traffic at the application level (i.e., Layer 7), this tip will not help.
If the problem you're having only happens when you try to connect from work, that's probably because you're behind an HTTP proxy, which usually will only forward HTTP/HTTPS traffic, and then you won't be able to use other protocols (like RDP or VNC).
You can check that by using the telnet command, as pnti suggested. If telnetting fails, then it's quite likely that you won't be able to establish an RDP or VNC session from work to your home server.
Best Answer
It is possible to have VNC in "read-only" mode. Have you checked that? When "read-only", no input events (keyboard or mouse) are transmitted to the remote end.