You've asked a great question. The question seems very simple, but in fact the answer is somewhat more complex. I'll do my best to answer it in a succinct manner. Also, since you mentioned ISAKMP, I am going to assume you are interested in IKEv1. Things change a little for IKEv2 (well, a lot), but I did want to mention the answer below only correlates to IKEv1.
Phase 1 can be accomplished in two different mods: Main Mode and Aggressive Mode. In either mode, the first message is sent by the Initiator, and the second message is sent by the Responder. Both of these messages include what is known in the cryptography world as a Nonce. A Nonce is simply a randomly generated number to use in key generation. (the term Nonce comes from _N_umber used _Once_). So, after message 1 and message 2, both sides know each other's Nonces.
The Nonce's are combined with the Pre-Shared-Key to create a Seed value for generating secret keys. The relative part of the IKE RFC is here:
For pre-shared keys: SKEYID = prf(pre-shared-key, Ni_b | Nr_b)
SKEYID is the Seed value that will later be used to generate additional secret keys. The Pre-Shared-Key and both Nonce values (Ni_b is the Initiator's Nonce, and Nr_B is the Responder's Nonce) is combined by using a PRF, or Psuedo Random Function. A PRF is like a hashing algorithm, except that the result can be as many bits as you need.
Now, if you were initially doing Main Mode, messages 3 and 4 share the Initiator's and Responder's (respectively) Diffie-Hellman public keys. They both use the these values to generate the Diffie-Hellman shared secret. If you were doing Aggressive mode, these DH Public values are also included in Message 1 and Message 2 (along with the Nonces).
The Seed value is then combined with the DH Shared Secret (and a few other values) to create three Session Keys: a Derevative key, an Authentication key, and an Encryption key. The RFC states it as such:
The result of either Main Mode or Aggressive Mode is three groups of
authenticated keying material:
SKEYID_d = prf(SKEYID, g^xy | CKY-I | CKY-R | 0)
SKEYID_a = prf(SKEYID, SKEYID_d | g^xy | CKY-I | CKY-R | 1)
SKEYID_e = prf(SKEYID, SKEYID_a | g^xy | CKY-I | CKY-R | 2)
SKEYID_d, _a, and _e are the three Session keys mentioned above. SKEYID
is the Seed value. g^xy
is the DH Shared Secret. CKY-I
and CKI-R
are the Initiator and Responder Cookies, these are just additional randomly generated values that are used later to identify this particular ISAKMP exchange and security association. 0 1 2
are the literal numbers 0, 1, and 2. The Pipe symbol |
represents concatenation.
In any case, all these values are combined together using a PRF which creates three Session keys:
- Derivative Key -- this key is not used by ISAKMP, and is instead handed to IPsec so that IPsec can create its own Secret Keys
- Authentication Key -- this key is used by ISAKMP in its HMAC (aka, Hashing algorithm secured with a Secret key)
- Encryption Key -- this key is used by ISAKMP to symmetrically encrypt anything ISAKMP wants to securely to the other peer. So if the chosen Encryption algorithm for Phase1 is AES, AES will use this key to symmetrically encrypt data -- AES will not generate its own keying material.
The Authentication Key and Encryption Key are used to secure/encrypt the ensuing Phase2 negotiation. In Main Mode, messages 5 and 6 of Phase1 are also protected by these keys. Furthermore, any future ISAKMP informational exchanges (DPD, Rekey events, Delete messages, etc) are also protected by these two keys.
The Derivative Key is handed to IPsec, and IPsec generates its own Keying material from this Key. If you recall, IPsec does not innately include a Key Exchange mechanism, so the only way for it to acquire secret keys is to either set them manually (which is archaic, and never really done anymore), OR to depend on an external service to provide the keying material, like ISAKMP.
The RFC says it like so:
SKEYID_e is the keying material used by the ISAKMP SA to protect
the confidentiality of its messages.
SKEYID_a is the keying material used by the ISAKMP SA to
authenticate its messages.
SKEYID_d is the keying material used to derive keys for non-ISAKMP
security associations.
With all that said, we can refer back to your question: What key is used to secure the Pre-Shared-Key?
In Main Mode, the Pre-Shared-Key (PSK) is verified in Messages 5 and 6. Message 5 and 6 are Protected by the Session keys ISAKMP generates, described above.
In Aggressive Mode, none of the messages in the negotiation are encrypted. And the PSK is verified in Messages 2 and 3. Notice, I said in both cases the PSK is verified, and I never said the PSK is exchanged. Obviously, if nothing in Aggressive mode is Encrypted, and you simply sent the Pre-Shared-Key across the wire unencrypted, there would be a huge gaping security vulnerability.
Lucky for us, the writers of ISAKMP already thought of that. And as a result, they created a special method for verifying that each party has the correct PSK, without actually sharing it across the wire. There are two items that are use to validate to each Peer that they both have the same PSK: the Identity Method and the Identity Hash.
VPN Peers can choose to identify themselves by various methods; most commonly, peers will simply use their source IP address. But they have the option to use a FQDN or Hostname. Each of these, along with the correlating value for the chosen method, are what make up the Identity Method. So for example, if I had the IP 5.5.5.5, and I wanted to use my IP address to identify myself, my ID Method would effectively be [IP Address, 5.5.5.5]. (Note: BOTH values make up the entire ID Method)
The ID Method is then combined (using a PRF) with the Seed value we discussed earlier (SKEYID), and a few other values, to create the Identity Hash. Recall, that what went into creating SKEYID in the first place was the Pre-Shared-Key.
The ID Method and ID Hash are then sent across the wire, and the other party attempts to re-create the ID Hash using the same formula. If the receiver is able to re-create the same ID Hash, it proves to the receiver that the sender must have had the correct pre-shared-key.
This is described in the RFC here:
To authenticate either exchange the initiator of the protocol
generates HASH_I and the responder generates HASH_R where:
HASH_I = prf(SKEYID, g^xi | g^xr | CKY-I | CKY-R | SAi_b | IDii_b )
HASH_R = prf(SKEYID, g^xr | g^xi | CKY-R | CKY-I | SAi_b | IDir_b )
IDii and IDir are the ID Method. And HASH_I and HASH_R are the Initiator and Responder hash's. Both of these are shared in Phase1. From the RFC:
When doing a pre-shared key authentication, Main Mode is defined as
follows:
Initiator Responder
---------- -----------
HDR, SA -->
<-- HDR, SA
HDR, KE, Ni -->
<-- HDR, KE, Nr
HDR*, IDii, HASH_I -->
<-- HDR*, IDir, HASH_R
Aggressive mode with a pre-shared key is described as follows:
Initiator Responder
----------- -----------
HDR, SA, KE, Ni, IDii -->
<-- HDR, SA, KE, Nr, IDir, HASH_R
HDR, HASH_I -->
And now, we can finally answer your question fully:
The Pre-Shared-Key is combined using a PRF with Nonces, and a bunch of other values known to anyone else in the negotiation. The result is a value that can only be mutually attained by two parties if both parties started with the same values -- aka, the same pre-shared-key. The resulting value is what is shared on the wire, with allows two parties to verify they have matching Pre-Shared-Keys without actually exposing any information about the Pre-Shared-Key itself.
Main Mode goes a step more secure by also encrypting the exchange of the "resulting value" described above, making it even more difficult to extract any useful information about what the Pre-Shared-Key was.
(it seems I failed miserably at answering this succinctly)
Best Answer
Good questions.
The cookie's are arbitrarily generated (often randomly). They are 64 bits from the Initiator and 64 bits from the Responder, and collectively (the 128 bits) serve as the identification of a particular ISAKMP "tunnel" (or ISAKMP "SA") -- i.e., a particular set of the 3 ISAKMP keys (encryption, authentication, derivative).
(much like SPI's will identify a particular IPsec SA / tunnel)
The values are also used within the calculation of the 3 ISAKMP keys, but that doesn't happen until after message 3 and 4, so the Initiator and Responder will have already agreed upon the necessary hashing algorithms.
The PSK is "exchanged" between the peers before the tunnel has ever tried to build. Theoretically via phone call or manually configuring the PSK on both sides.
Both parties are verified to have the proper PSK in Messages 5 and 6. What goes into creating the E/A/D ISAKMP keys is the PSK, so if both parties didn't start with the correct PSK, they could never create the same E/A/D keys, and would not be able to "understand" each other's message 5 and 6.
Yes. After Messages 1-4 of Main Mode, three ISAKMP Keys exist: Encryption Key, Authentication Key, Derivative Key.
The Encryption Key and Authentication Key will secure Messages 5 and 6 of Main Mode, and any ensuing Quick Mode negotiation messages.
The Derivative Key is not used by ISAKMP, it is instead given to IPsec so IPsec can generate its own Encryption and Authentication Keys.
What is exchanged is somewhat arbitrary, the why is what is important. The purpose of Messages 5/6 are to validate the other party is who they claim to be (aka, the other party has the correct pre-shared-key).
Here is (some of) what goes into calculating the value that is exchanged in message 5 and 6, and more importantly the purpose it serves:
There are two parts to the Arbitrary Data, one set is from the Initiator and the other set is from the Responder. So in the end, you have two collective values exchanged in Messages 5 and 6:
Both the Initiator and Responder have the information they need to put together both values above. Therefore, both Initiator and Responder have a value they can use to prove they are who they say they are, and a value they can use to verify the other is who they say they are.
The specific details are outlined on Page 10 of RFC 2409. I also explain some of the details of the process in this answer.
Yes and Yes.
And in Main Mode they protect the Exchange of the identity values described above in Message 5 and 6.