Linux – Ansible module_stdout: command not found when using sudo on Centos 7

ansiblecentoslinuxsudo

I am connecting to centos 7 virtual and physical machines using ansible with a user called ansible that has passwordless sudo permissions on all the boxes. On most machines it works, but on one box I get this error:

FAILED! => {
"changed": false,
"module_stderr": "Shared connection to ... closed.\r\n",
"module_stdout": "/var/tmp/sclpbsoCZ: line 8: -H: command not found\r\n",
"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
"rc": 127
}

I set up passwordless sudo using advice from this link:
https://code-maven.com/enable-ansible-passwordless-sudo
(basically, edit /etc/sudoers to let user ansible sudo without a password)

I thought these machines were identical, but obviously something is amiss.

Adding -vvv to the ansible-playbook command lets me see that the command it is trying to run is:

'/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '

How is it that this one machine isn't working with ansible?

Best Answer

I found this, but at first blush it seemed unrelated to ansible:

https://unix.stackexchange.com/questions/192809/sudo-i-returns-an-error

It turns out that centos has scl (software collections) that can be enabled.

I had devtoolset-7 enabled at boot:

https://www.softwarecollections.org/en/scls/rhscl/devtoolset-7/

This happens to wrap sudo with a script that doesn't seem to actually handle all of the options of the original sudo. It seems like this could be considered a bug (or at least bad behavior) in devtoolset-7.

For now the fix was to remove the entry in /etc/profile.d/ that was enabling the devtoolset-7.

It was running this code:

source /opt/rh/devtoolset-7/enable

If I ssh'd into the machine and ran the offending command:

sudo -H -S -n -u root /bin/sh

It would give me the same error:

# sudo -H -S -n -u root /bin/sh /var/tmp/sclKpdWFR: line 8: -H: command not found

And pointed me to the problem when I queried which sudo was being used...

# which sudo /opt/rh/devtoolset-7/root/usr/bin/sudo

What a rabbit hole!