How to extract private key from pfx file using openssl

Firstly, you have a few terminology problems:

• the X509 standard defines certificates, and RSA and DSA are two of the public key algorithms that can be used in those certificates;
• certificates are used to hold public keys, and never private keys.
• PKCS#12 is a standard for a container which can hold an X509 client certificates and the corresponding private keys, as well as (optionally) the X509 certificates of the CAs that signed the X509 client certificate(s).

So, if you're examining a PKCS#12 file (typically .p12 extension), then you already know:

• It contains at least one X509 client certificate, which contains a public key; and
• It contains the corresponding private keys.

All you don't know is whether those certificate & private key are RSA or DSA. You can check this by extracting the certificate(s), and then examine them:

openssl pkcs12 -in mycert.p12 -clcerts -nokeys -out mycert.crt
openssl x509 -in mycert.crt -text

The text output of the openssl x509 command should include a Subject Public Key section, which will include fields that let you see if it's an RSA or DSA key (along with the key size).

On Windows you can also set the environment property OPENSSL_CONF. For example from the commandline you can type:

set OPENSSL_CONF=c:/libs/openssl-0.9.8k/openssl.cnf

to validate it you can type:

echo %OPENSSL_CONF%

You can also set it as part of the computer's environmental variables so all users and services have it available by default. See, for example, Environment variables in Windows NT and How To Manage Environment Variables in Windows XP.

Now you can run openssl commands without having to pass the config location parameter.