IPSec with PSK under Win Server 2k8 R2

ipsecwindows-server-2008-r2

I'm trying to connect a Win Server 2k8 R2 system to a remote machine via IPSec. For this, I got the following settings from the other party:

Phase 1: PSK, AES256, SHA-1 Group 2
Phase 2: AES256, SHA-1

Encryption Domain: xxx.yyy.zzz.0/24
<-> aaa.bbb.ccc.ddd

To test the IPSec connection use
xxx.yyy.zzz.104:22 (telnet to that
Port).

Where xxx.yyy.zzz are the first three bytes of the remote IP address and aaa.bbb.ccc.ddd represent the IP address of our 2k8 R2 server.

I tried to set the IPSec connection up in Windows Firewall with Advanced Security, but choosing PSK for the first authentication method disables the use of the second authentication method. I also tried to create a VPN of type Layer 2 Tunneling Protocol with IPSec (L2TP/IPSec) the "conventional" way for which I can provide the PSK. But then I cannot figure out how to set up all the other settings like AES, SHA, IP addresses, second authentication method, etc. at all.

I've never set up an IPSec connection before, so I'm totally new to this field. Detailled instructions or maybe some hints whether I'm searching in the right direction would very much help me.

I know that a similar question can be found here: Can I use Win 2k8 R2 as an IPSEC client?. But I couldn't figure out how to solve the problem based on the answer provided there.

Best Answer

It's probably worth noting that by the looks of it Win2k8 only allows the combination of IPSec over L2TP, but that is not the usually used standard in other VPN devices. More often than not IPSec is put on top of IKE. From the details provided by the other side, I would suspect that that is what they are suggesting.
Further to that I remember dimly (although this may have changed over the years) that the L2TP implementation by Microsoft did not conform 100% to the standard, and therefore would only really talk to other Windows hosts.

I would personally suggest to contact the other party to get more detail from them, and then probably look into using OpenVPN, which is a lot more configurable than the VPN client in Win2k8.

Related Topic