File created by ansible seems to have mangled permissions

ansiblepermissionsubuntu-12.04

Issue with files created by Ansible

I’m looking at using to setup some new servers at a new ISP and I’ve been doing some basic steps to test first. I am new but have done some testing on diff servers at another ISP which seemed to go OK (I had created files there no problem). Both are Unbuntu 12 but the original testing had been on a server that had already its kernel updated to 3.8.0-44-generic and other maint added. This new server is also Ubuntu 12 but comes from the ISP template, and is using Kernel 3.5 (Ubuntu 12.04.2 LTS (GNU/Linux 3.5.0-23-generic x86_64))

My first test was going to be creating new users and uploading their already existing SSH keys for future login. The user was created with no issues, and the file creation gave no indication of any errors during the creation process, but did when I tried to use them. I tried to login(new user)_ using the key but the system acted as if the SSH key file wasn’t there. Since this was the first new user I had been logged in as root and did my first looking and creation as root

As root everything looks good. The file “authorized_keys” in the .ssh directory exists and appears to have the correct permissions and does have the correct content. But trying to login via SSH acts as if it is not there. I logged out of root and logged back in as the new user (using the password since the key wasn’t being recognized ). I do an “ls -al” and see the .ssh directory fine, but doing “ls -al .ssh” to see the “authorized_keys” file in the directory gives me a very strange result. ‘ls’ put out a “Permission denied” error message for each item inside the directory, and then displays what should be command results, but while the file name is visible everything else (permissions, file size, user/group, date) are replaced by question marks.

First as user – myusername

myusername@my-server:~$ ls -al .ssh
ls: cannot access .ssh/authorized_keys: Permission denied
ls: cannot access .ssh/..: Permission denied
ls: cannot access .ssh/.: Permission denied
total 0
d????????? ? ? ? ?            ? .
d????????? ? ? ? ?            ? ..
-????????? ? ? ? ?            ? authorized_keys
myusername@my-server:~$

Now as root

sudo bash
[sudo] password for myusername:
root@my-server:~# sudo ls -al .ssh
total 12
drw-r--r-- 2 myusername myusername 4096 Sep 17 19:54 .
drwxr-xr-x 4 myusername myusername 4096 Sep 17 22:51 ..
-rw-r--r-- 1 myusername myusername  406 Sep 17 19:54 authorized_keys
root@my-server:~#

As far as the documentation goes, my server meets the minimum requirements (linux, SSH and Python)

The specs are – Ubuntu 12.04.2 LTS (GNU/Linux 3.5.0-23-generic x86_64)

The ansible version is 1.7.1

I am running the ansible session from my OSX (10.9.4) Python Python 2.7.5 laptop

Below is the Ansible Playbook I ran to do this

---
# This playbook create my user {{userid}} and loads the public ssh-key

- name: create my user {{userid}} and loads the public ssh-key  
  hosts: myservername-public 
#  gather_facts: no
#  remote_user: myusername
  vars:
#    security_groups: "sudo,adm"
    security_groups: ""
    userid: testjunk01
  tasks:
  - name: test connection
    ping:
    remote_user: myusername

  - name: Create user {{userid}} groups={{security_groups}} 
    user: name={{userid}} shell=/bin/bash groups={{security_groups}} append=yes 
      password=$hashed_password_was_here_and_it_worked

  - name: Verify that needed directories are in place before file copy 
    file: dest="/home/{{userid}}/.ssh"
          mode=0644
          owner={{userid}} group={{userid}} 
          state=directory

  - name: Copy file into user {{userid}}'s directory 
    copy: src="/Users/osx_user/Documents/Projects/Projects Internal/Security/ssh-key-public/myusername"
          dest="/home/{{userid}}/.ssh/authorized_keys"
          mode=0644
          owner={{userid}} group={{userid}} 
          backup=yes

  - name: Reset permissions for file after file copy 
    file: dest="/home/{{userid}}/.ssh/authorized_keys"
          mode=0644
          owner={{userid}} group={{userid}} 
          state=file

===== Just upgraded to ansible 1.7.2 and tried again. Same results. Execution below

server name and IP address masked

$ ansible-playbook playbooks/test01/Test02/create-user -i  ansible-hosts --ask-pass -vv
SSH password:

PLAY [create my user testjunk01 and loads the public ssh-key] *****************

GATHERING FACTS ***************************************************************
<192..168.1.1> REMOTE_MODULE setup
ok: [myserver-public]

TASK: [test connection] *******************************************************
<192..168.1.1> REMOTE_MODULE ping
ok: [myserver-public] => {"changed": false, "ping": "pong"}

TASK: [Create user testjunk01 groups=] ****************************************
<192..168.1.1> REMOTE_MODULE user name=testjunk01 shell=/bin/bash groups= append=yes password=VALUE_HIDDEN
ok: [myserver-public] => {"append": true, "changed": false, "comment": "", "group": 1003, "groups": "", "home": "/home/testjunk01", "move_home": false, "name": "testjunk01", "password": "NOT_LOGGING_PASSWORD", "shell": "/bin/bash", "state": "present", "uid": 1003}

TASK: [Verify that needed directories are in place before file copy] **********
<192..168.1.1> REMOTE_MODULE file dest="/home/testjunk01/.ssh" mode=0644 owner=testjunk01 group=testjunk01 state=directory
changed: [myserver-public] => {"changed": true, "gid": 1003, "group": "testjunk01", "mode": "0644", "owner": "testjunk01", "path": "/home/testjunk01/.ssh", "size": 4096, "state": "directory", "uid": 1003}

TASK: [Copy file into user testjunk01's directory] ****************************
changed: [myserver-public] => {"changed": true, "dest": "/home/testjunk01/.ssh/authorized_keys", "gid": 1003, "group": "testjunk01", "md5sum": "fd6c6017993e847026a010bf67a96c1a", "mode": "0644", "owner": "testjunk01", "size": 406, "src": "/root/.ansible/tmp/ansible-tmp-1412114658.34-25330110994991/source", "state": "file", "uid": 1003}

TASK: [Reset permissions for file after file copy] ****************************
<192..168.1.1> REMOTE_MODULE file dest="/home/testjunk01/.ssh/authorized_keys" mode=0644 owner=testjunk01 group=testjunk01 state=file
ok: [myserver-public] => {"changed": false, "gid": 1003, "group": "testjunk01", "mode": "0644", "owner": "testjunk01", "path": "/home/testjunk01/.ssh/authorized_keys", "size": 406, "state": "file", "uid": 1003}

PLAY RECAP ********************************************************************
myserver-public         : ok=6    changed=2    unreachable=0    failed=0

Best Answer

The first ls you quoted is what happens when you try to ls -l a directory on which you have r permission but no x permission. The r bit on a directory gives you the ability to call readdir on it, so you can get the filenames inside it, but without x you can't stat them to find out all the other information that ls -l normally prints.

ls told you about the inability to get the information with its Permission denied error messages and fields full of question marks.

The second ls confirms the diagnosis with this line:

drw-r--r-- 2 myusername myusername 4096 Sep 17 19:54 .

The user myusername has read permission but no execute permission. You probably want to u+x or a+x that directory.

One interesting thing I learned from your post is that r-without-x now gives slightly more information than it did in olden times... you can see in the original listing that authorized_keys is a regular file and . and .. are directories (first column d vs. -). On an older Unix/Linux system that first column would be a question mark too! Nowadays readdir returns file type information as a bonus so you can get it without stating.

...I kind of skimmed past the second half of your question because I've never seen that stuff before, but this looks like the culprit

file: dest="/home/{{userid}}/.ssh"
      mode=0644

I'd expect a .ssh directory to be 0700 normally. And the files inside to be 0600. It's an area where you really don't want other users wandering in. Anyway it needs to be at least 0500 for ls -l to function normally for the owner.

Related Solutions

Linux – Easier Way to Add Public-Key Authentication for New Users

You need to change the ownership of the file and set permissions appropriately.

$ chown -R newuser /path/to/home/.ssh
$ chmod 700 /path/to/home/.ssh
$ chmod 600 /path/to/home/.ssh/authorized_keys

To make this process more simple, repeatable, and audit-able, use a configuration management system for user management. All widely-used config management systems (puppet, chef, ansible, etc.) ship with the ability to create users and deploy keys. A huge added benefit of using a config management system is that you can also keep your configuration in source control.

Security – How to implement ansible with per-host passwords, securely

You've certainly done your research...

From all of my experience with ansible what you're looking to accomplish, isn't supported. As you mentioned, ansible states that it does not require passwordless sudo, and you are correct, it does not. But I have yet to see any method of using multiple sudo passwords within ansible, without of course running multiple configs.

So, I can't offer the exact solution you are looking for, but you did ask...

"So... how are people using Ansible in situations like these? Setting NOPASSWD in /etc/sudoers, reusing password across hosts or enabling root SSH login all seem rather drastic reductions in security."

I can give you one view on that. My use case is 1k nodes in multiple data centers supporting a global SaaS firm in which I have to design/implement some insanely tight security controls due to the nature of our business. Security is always balancing act, more usability less security, this process is no different if you are running 10 servers or 1,000 or 100,000.

You are absolutely correct not to use root logins either via password or ssh keys. In fact, root login should be disabled entirely if the servers have a network cable plugged into them.

Lets talk about password reuse, in a large enterprise, is it reasonable to ask sysadmins to have different passwords on each node? for a couple nodes, perhaps, but my admins/engineers would mutiny if they had to have different passwords on 1000 nodes. Implementing that would be near impossible as well, each user would have to store there own passwords somewhere, hopefully a keypass, not a spreadsheet. And every time you put a password in a location where it can be pulled out in plain text, you have greatly decreased your security. I would much rather them know, by heart, one or two really strong passwords than have to consult a keypass file every time they needed to log into or invoke sudo on a machine.

So password resuse and standardization is something that is completely acceptable and standard even in a secure environment. Otherwise ldap, keystone, and other directory services wouldn't need to exist.

When we move to automated users, ssh keys work great to get you in, but you still need to get through sudo. Your choices are a standardized password for the automated user (which is acceptable in many cases) or to enable NOPASSWD as you've pointed out. Most automated users only execute a few commands, so it's quite possible and certainly desirable to enable NOPASSWD, but only for pre-approved commands. I'd suggest using your configuration management (ansible in this case) to manage your sudoers file so that you can easily update the password-less commands list.

Now, there are some steps you can take once you start scaling to further isolate risk. While we have 1000 or so nodes, not all of them are 'production' servers, some are test environments, etc. Not all admins can access production servers, those than can though use their same SSO user/pass|key as they would elsewhere. But automated users are a bit more secure, for instance an automated tool that non-production admins can access has a user & credentials that cannot be used in production. If you want to launch ansible on all nodes, you'd have to do it in two batches, once for non-production and once for production.

We also use puppet though, since it's an enforcing configuration management tool, so most changes to all environments would get pushed out through it.

Obviously, if that feature request you cited gets reopened/completed, what you're looking to do would be entirely supported. Even then though, security is a process of risk assessment and compromise. If you only have a few nodes that you can remember the passwords for without resorting to a post-it note, separate passwords would be slightly more secure. But for most of us, it's not a feasible option.

Best Answer

Related Solutions

Linux – Easier Way to Add Public-Key Authentication for New Users

Security – How to implement ansible with per-host passwords, securely

Related Topic