Centos – SELinux corrupted? Now unable to boot CentOS 7 with SELinux enabled

centosselinux

We recently experienced a power failure and simultaneous backup generator failure, severe enough to require safely shutting down all servers as their UPSs were draining.

Upon bringing one CentOS 7.4.1708 server back up (its first "reboot" in months, but it is up-to-date in terms of CentOS updates) I hit a brick wall preventing me from booting it with SELinux enabled. I have researched extensively but can't seem to find evidence that anyone else has experienced this, nor do I know what to try next. I'm hoping someone here can offer up some ideas.

Here's the timeline:

  1. Booted

  2. Boot failed due to several services not starting up:

    FAILED Failed to start Login Service.
See 'systemctl status systemd-logind.service' for details.
FAILED Failed to start Authorization Manager.
See 'systemctl status polkit.service' for details.
DEPEND Dependency failed for Dynamic System Tuning Daemon.

  3. Prompted by this I rebooted with selinux=0 in grub

  4. This works and gets the system running, but with SELinux disabled which is not viable for us as anything other than a temporary workaround

  5. Followed advice found online:

    sudo yum reinstall selinux-policy-targeted

  6. Rebooted

  7. Boot now failed due to:

    Failed to load SELinux policy, freezing

  8. Rebooted with selinux=0 in grub again

  9. Found more advice so performed:

    sudo yum reinstall selinux-policy-targeted
sudo touch /.autorelabel

    and set permissive in /etc/selinux/config

  10. Rebooted

  11. Could see the following banner:

    Warning -- SELinux targeted policy relabel is required.
Relabeling could take a very long time, depending on file
system size and speed of hard drives.

    but instead of actually performing the relabelling, the system immediately rebooted itself — too fast to see any other output

  12. Boot again failed with the original error.

    So ugh we're back here again. And I can see that /.autorelabel still exists, suggesting the relabel didn't happen. It's surprising to me that we're back to square one with the errors though.

    Also recall that SELinux is still in permissive mode, not enforcing.

The end result is that I'm stuck without being able to enable SELinux in either permissive or enforcing mode, which is not okay.

How should I proceed?

Best Answer

tl;dr: Everything came down to an invalid value for SELINUXTYPE.

Make sure SELINUXTYPE has a valid value, then perform relabelling if needed (e.g. if you've booted with SELinux off during your diagnostics), reboot and crack open the champagne.

For some reason, and at some point in time, /etc/selinux/config had acquired the setting SELINUXTYPE=permissive.

This is not a valid option for that parameter, and it appears to make the value fall back on the value "default", based upon the reason listed for why Dbus, Login Service and Authorization Manager failed:

Failed to open "/etc/selinux/default/contexts/dbus_contexts": No such file or directory

This is troublesome because there is no selinux-policy-default package in CentOS 7 (on Debian, for example, it was deliberately removed in Jessie so I'd imagine the same is true here).

I suspect this is also why attempts to relabel the file system using restorecon (from single-user mode, and from a shell reached by init=/sbin/sh) resulted in baffling "No such file or directory" outputs, and getenforce would still show "Disabled" for no evident reason.

To switch to the selinux-policy-targeted stuff which is installed, I fixed the configuration to be SELINUXTYPE=targeted (as I believe it should have been all along), then again rebooted with enforcing=0 autorelabel=1.

The relabelling then took place. Following that, the system booted up as normal.

Related Topic