I have a rather strange problem with my SSH configuration. I set up my server with the help of a Remote Access Card and configured everything with a KVM viewer.
So while being logged into the server via the KVM Viewer I configured SSH with only pubkey and tried to login from my local laptop. It worked fine.
If I quit the KVM Session (or logout with the user in the KVM session) I cannot login via ssh anymore (pubkey denied). SSH login only works as long as the user is somewhere still logged in.
Any hints what the problem might be?
Console output for a failed login (all personal data exchanged):
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/mylocaluser/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 103: Applying options for *
debug1: Connecting to 100.100.100.100 [100.100.100.100] port 12345.
debug1: Connection established.
debug1: identity file /Users/mylocaluser/.ssh/id_rsa type 1
debug1: identity file /Users/mylocaluser/.ssh/id_rsa-cert type -1
debug1: identity file /Users/mylocaluser/.ssh/id_dsa type -1
debug1: identity file /Users/mylocaluser/.ssh/id_dsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.2
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1p1 Ubuntu-2ubuntu2
debug1: match: OpenSSH_6.6.1p1 Ubuntu-2ubuntu2 pat OpenSSH*
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5-etm@openssh.com none
debug1: kex: client->server aes128-ctr hmac-md5-etm@openssh.com none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: RSA ab:12:23:34:45:56:67:78:89:90:12:23:34:45:56:67
debug1: Host '[100.100.100.100]:12345' is known and matches the RSA host key.
debug1: Found key in /Users/mylocaluser/.ssh/known_hosts:36
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /Users/mylocaluser/.ssh/id_rsa
debug1: Authentications that can continue: publickey
debug1: Offering RSA public key: /Users/mylocaluser/.ssh/id_rsa2
debug1: Authentications that can continue: publickey
debug1: Trying private key: /Users/mylocaluser/.ssh/id_dsa
debug1: No more authentication methods to try.
Permission denied (publickey).
Console output for a successfull login (only possible while "active session" exists):
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/mylocaluser/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 103: Applying options for *
debug1: Connecting to 100.100.100.100 [100.100.100.100] port 12345.
debug1: Connection established.
debug1: identity file /Users/mylocaluser/.ssh/id_rsa type 1
debug1: identity file /Users/mylocaluser/.ssh/id_rsa-cert type -1
debug1: identity file /Users/mylocaluser/.ssh/id_dsa type -1
debug1: identity file /Users/mylocaluser/.ssh/id_dsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.2
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1p1 Ubuntu-2ubuntu2
debug1: match: OpenSSH_6.6.1p1 Ubuntu-2ubuntu2 pat OpenSSH*
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5-etm@openssh.com none
debug1: kex: client->server aes128-ctr hmac-md5-etm@openssh.com none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: RSA ab:12:23:34:45:56:67:78:89:90:12:23:34:45:56:67
debug1: Host '[100.100.100.100]:12345' is known and matches the RSA host key.
debug1: Found key in /Users/mylocaluser/.ssh/known_hosts:36
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /Users/mylocaluser/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 279
debug1: Authentication succeeded (publickey).
Authenticated to 100.100.100.100 ([100.100.100.100]:12345).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env LANG = de_DE.UTF-8
Welcome to Ubuntu 14.04.1 LTS
Best Answer
When the user's home directory is encrypted with
ecryptfs
sshd
cannot read theauthorized_keys
file from the user's home directory before the home directory has been mounted.During login
sshd
will usepam
to authenticate the user, andpam
will use the password entered by the user to mount the encrypted home directory.This is problematic if you want to restrict
sshd
to only permit public key authentication.However you can place an unencrypted
authorized_keys
file on the server as well. This will permit the user to login using a key, but since this does not invokepam
, the home directory will not be mounted, and mounting the home directory without knowing the password won't work either.Since the unencrypted home directory gets hidden by the encrypted home directory, placing the unencrypted
authorized_keys
file in the first place can be a bit tricky. A bind mount of the underlying file system can help with that.If for example
/home
is just a directory on the root file system, you can do as follows:And then you can create
/mnt/rootfs/home/$USER/.ssh/authorized_keys
There is more you can do. Since the encrypted and unencrypted version of
authorized_keys
are two different files, you can put different contents in them. For example the unencrypted version can invoke a script in order to mount the encrypted home directory:The important part is the
command
specified before the key. This get invoked instead of the shell. But that happens only when this particular public key is used, and only if the user's home directory is not mounted.If the user's home directory is already mounted, this
authorized_keys
file is hidden and the encrypted version is used instead. The encrypted version ofauthorized_keys
does not have thecommand
, so the script to mount the home directory is not run.So, what goes in the script. Here is my version:
In the example above, the
authorized_keys
file is invoked without arguments, so the firstif
block is skipped. Theecryptfs-mount-private
command will thus ask for the user's password. But this does not requiresshd
to have password authentication enabled, and thus will work onsshd
with public key authentication only.The next command will change to the user's encrypted home directory (until then the script would be running inside the unencrypted home directory).
The last part of the script will run the command given as argument to the
ssh
command if any, or the users login shell if no command was given.One caveat is that this does not work with X11 forwarding, because the home directory is not available yet, when the cookie would be stored. But any other session opened while the home directory is already mounted, will be able to handle X11 forwarding.
Using
~/.ssh/rc
instead could possibly solve the X11 forwarding issue. This is something I have not looked into yet.The first
if
block is a bit of a hack, which I came up with to allow the user's home directory to be mounted without needing a password. Instead it uses a forwardedssh-agent
to mount the user's home directory. That part comes with disclaimers about not having had any peer review, so trusting the cryptography in thessh-agent-ecryptfs-decryption.py
is entirely at your own risk.The python script looks like this:
So how does the decryption work? First of all the argument to the script as provided by
authorized_keys
is any random value. A uuid generated withuuidgen
could work. The shell script uses grep to find the relevant line in theauthorized_keys
file to extract the public key.The base64 encoded public key as well as the uuid are given to the python script. The public key used is exactly the one, which the user authenticated with. The python script asks the forwarded agent for a signature on a specific message using the public key in question (because signing messages is exactly what
ssh-agent
can do). Part of the signature is then encoded with base64 to produce a password.This password is used to decrypt an
ecryptfs
wrapped password file, but the primary file is encrypted using the user's login password. This one is encrypted with a password generated from the ssh key.