Amazon EC2 – Unable to SSH After Changing Permissions to Home Directory

amazon ec2amazon-web-services

AS the title mentions. I was dumb enough to change the permissions of home/ec2-user recursively to 777 after which I could not log in. I then detached the volume and then attached it to another instance and changed the permission to 700 recursively of that folder. Now when I try to connect to it using the following command I get the error

     ssh -v -i  ~/Amazon-fooDev/fooDev.pem ec2-user@ec2-34-212-108-144.us-west-2.compute.amazonaws.com
OpenSSH_7.8p1, LibreSSL 2.7.3
debug1: Reading configuration data /Users/admin/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: Connecting to ec2-34-212-108-144.us-west-2.compute.amazonaws.com port 22.
debug1: Connection established.
debug1: identity file /Users/admin/Amazon-fooDev/fooDev.pem type -1
debug1: identity file /Users/admin/Amazon-fooDev/fooDev.pem-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.8
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
debug1: Authenticating to ec2-34-212-108-144.us-west-2.compute.amazonaws.com:22 as 'ec2-user'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:XJWnn7TLa5feJVY7kAOUJQGmWm8J3UNQ/MM+uQZa+Sk
debug1: Host 'ec2-34-212-108-144.us-west-2.compute.amazonaws.com' is known and matches the ECDSA host key.
debug1: Found key in /Users/admin/.ssh/known_hosts:1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/admin/Amazon-fooDev/fooDev.pem
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
ec2-user@ec2-34-212-108-144.us-west-2.compute.amazonaws.com: Permission denied (publickey).

Any suggestions on what I could do to fix this problem. I wasted an entire day trying to connect to that instance and get back to the state it was originally in. Any suggestions would be appreciated. Is 700 permission for home/ec2-user correct ? or should i change it to something else ? Do u think I might be looking in the wrong direction ?

Here are a list of some of the latest permissions

[root@ip-172-31-42-113 vol1]# chmod 600 home/ec2-user/.ssh/authorized_keys
[root@ip-172-31-42-113 vol1]# ls -l
total 120
dr-xr-xr-x  2 root root  4096 Nov 17 09:05 bin
dr-xr-xr-x  4 root root  4096 Nov 17 09:05 boot
drwxr-xr-x  2 root root  4096 Feb 28  2014 cgroup
drwxr-xr-x  2 root root  4096 Aug 11 01:27 dev
drwxr-xr-x 83 root root  4096 Nov 27 01:51 etc
drwx------  3 root root  4096 Nov 17 05:10 home
dr-xr-xr-x  7 root root  4096 Nov 17 08:48 lib
dr-xr-xr-x 10 root root 12288 Nov 17 09:05 lib64
drwxr-xr-x  2 root root  4096 Aug 11 01:26 local
drwx------  2 root root 16384 Aug 11 01:26 lost+found
drwxr-xr-x  2 root root  4096 Jan  6  2012 media
drwxr-xr-x  2 root root  4096 Jan  6  2012 mnt
drwxr-xr-x  3 root root  4096 Aug 11 01:26 opt
drwxr-xr-x  2 root root  4096 Aug 11 01:26 proc
dr-xr-x---  5 root root  4096 Nov 26 12:00 root
drwxr-xr-x  3 root root  4096 Nov 17 05:10 run
dr-xr-xr-x  2 root root 12288 Nov 17 09:05 sbin
drwxr-xr-x  2 root root  4096 Jan  6  2012 selinux
drwxr-xr-x  2 root root  4096 Jan  6  2012 srv
drwxr-xr-x  2 root root  4096 Aug 11 01:26 sys
drwxrwxrwt  3 root root  4096 Nov 27 03:15 tmp
drwxr-xr-x 16 root root  4096 Nov 22 23:12 usr
drwxr-xr-x 20 root root  4096 Nov 17 08:31 var
[root@ip-172-31-42-113 vol1]# cd  home/ec2-user/.ssh/
[root@ip-172-31-42-113 .ssh]# ls -l
total 4
-rw------- 1 ec2-user ec2-user 391 Nov 27 01:51 authorized_keys

Best Answer

SSH parameter -i expects that the next parameter is the key name.

In your case -iv tells ssh to read the key from file v - not quite what you wanted :) The flow-on effect is that the next parameter (the key name) is deemed to be the host name, which it obviously can't resolve.

This will work:

ssh -v -i ~/Amazon-Permission/FooDev.pem ec2-user@ec2-...

Update after your update :)

If you did recursive chmod 700 it changed the authorized_keys mode too. Mount the volume back to your little "helper" instance and do:

chmod 600 home/ec2-user/.ssh/authorized_keys

Update 2

Your home/ must be mode 755 and not 700. Otherwise ssh can't check open the file as ec2-user.

chmod 755 home
chmod 700 home/ec2-user home/ec2-user/.ssh
chmod 600 home/ec2-user/.ssh/authorized_keys

Hope that helps :)

Related Solutions

Ssh – EC2 after restart can not access via ssh

The line

debug1: Authentications that can continue: publickey

Indicates that you disabled password authentication. You can only authenticate using a public key (while having the corresponding private key).

The two consecutive lines

debug1: read PEM private key done: type RSA
debug1: Authentications that can continue: publickey

indicates that you are using an RSA private key that could be read fine, but that its public counterpart could not be authenticated by the server.

Some options to try:

Double check that you are using the correct key.
Double check the username. Maybe you made a typo when creating the account or did you connect using your own username earlier when copying over the key (ssh-copy-id).
In case you have access to the server in another way, check the permissions (and ownership!) on the complete ~ubuntu/.ssh directory recursively to be strict enough (OpenSSH will refuse access if too permissive).
- ~ubuntu/.ssh/ directory should be owned by the user and have mode 0700 (drwx------)
- ~ubuntu/.ssh/authorized_keys file should be owned by the user and have mode 0600 (-rw-------)
If you did not disable PasswordAuthentication in your SSH daemon, this may indicate that the password has been reset to be empty or that the user account is disabled (usermod -L) for login or expired. Try logging in with a different user (maybe root a regular user account).
If you did disable PasswordAuthentication in the configuration file of your SSH daemon but didn't restart it right after that, then the change may only become active after a reboot. See the option below for how to fix this.
Do you have direct access to the text console using a control panel in EC2? I'm not familiar with Amazon AWS, but I think you should have this option. Then you should be able to login using the console to reset your password and gain access via SSH.
Try to shut down your EC2 instance and attach the storage device to another instance. Mount it and do a manual password recovery. Re-attach it back to the original instance and it should be fixed.
After you gain access again, check that your server hasn't compromised. Changes in configuration triggered only after a reboot may indicate that it has. I have seen this happening on several servers.

EC2 instance keeps respawning, even after termination

It seems likely that this instance was started within an Auto Scaling Group, possibly as part of an Elastic Beanstalk deployment. If this is the case, either delete the Elastic Beanstalk deployment or set the min and max sizes for the ASG to 0 and this instance will go away.