Supermicro Firmware Upgrade – Is It Safe to Upgrade IPMI/BMC Firmware on a Running Supermicro Server?

Using /conf/crontab, as dlawson pointed out, sounds like an excellent idea to me. This allows me to run a script once a minute that ensures everything but http and ssh is shut down:

/etc/init.d/cdserver stop
/etc/init.d/fdserver stop
/etc/init.d/cim_sfcb stop
/etc/init.d/webgo stop

That still leaves me with a web server with password-based access control (I can see no way to have it validate client certificates) and who knows what remote vulnerabilities. Turning it off when I'm not using it (which is most of the time) seems like a reasonable solution; adding a crontab entry to shut it down every five or ten minutes would catch those cases where someone forgets to shut it down when he's done.

The ssh daemon is a version of dropbear that appears to be fairly heavily modified. It reads usernames and plaintext passwords from /conf/PMConfig.dat (which is also used by the web server), logs in any valid name and password as the root user, and ignores the ~/.ssh/authorized_keys file. This last problem is annoying; it forces you to allow password logins and opens the possibility of backdoors depending on from where-all it gets its names and passwords.

So that's the dilemma you face: how much do you really trust this modified ssh daemon installed on a system that was fairly obviously designed by security-naïve developers? Not much at all, given the number of broken bits of cruft I've seen in their shell scripts. There are unusual naming conventions (/etc/rc?.d/sshd is a symlink to /etc/init.d/ssh), huge amount of code that appear to be unused, and features in just the ssh startup script, such as the /conf/portcfg_ssh file and even the restart command are entirely broken. (Don't try using these; sshd won't restart and you'll be screwed unless you have an existing login. We rebooted the BMC and ended up having to reflash it.)

The best option I can think of, if one's going to use the thing at all, is to start ssh on an alternate port using a cron job, so at least it's less likely to appear in a portscan.

The final component is the IPMI network management ports; I can't see how to turn these off.

Welcome to Supermicro, where the docs are inconsistent and often wrong. What I typically end up doing is digging around on ftp://ftp.supermicro.com until I find something that seems useful.

In your case, I'd suggest you check for upgrades to the actual BIOS before you try anything else. The IPMI BIOS and the real BIOS appear to be closely linked, and upgrading the real one may help.

From there, I'd suggest you take a look at the tools in ftp://ftp.supermicro.com/utility/ , one of them will probably work for you, even though none explicitly claim to support your motherboard. The 'IPMI FW flash tools' directory is probably where I'd start.

I should note that it's possible to brick the motherboard or IPMI controller by using the wrong firmware flash tool. The only people that can tell you which tool is actually correct is Supermicro themselves.