Linux – Port 22 has been closed during upgrade (not accessible through SSH/SFTP)

debian-wheezylinux

After upgrading somehow some of the ports have been closed!? Now port 22 is no longer open and the server is not accessible from neither SSH nor SFTP

All other services like apache, mysql, webmin etc. is running as expected

Have tried to reboot the system

Open ports before upgrade

# nmap -PN 127.0.0.1
Host is up (0.0000090s latency).
Not shown: 996 closed ports
PORT      STATE SERVICE
22/tcp    open  ssh
80/tcp    open  http
3306/tcp  open  mysql
10000/tcp open  snet-sensor-mgmt

Open ports after upgrade

# nmap -PN 127.0.0.1
Host is up (0.0011s latency).
Not shown: 998 closed ports
PORT      STATE SERVICE
80/tcp    open  http
10000/tcp open  snet-sensor-mgmt

Below are the exact steps during upgrade. The upgrade has been performed on multiple servers and same issue in all cases with closed ports after the upgrade was completed

1 step: Upgrading mysql 5.6.15 to 5.6.20

service mysql stop
dpkg -r mysql
dpkg -r mysql-client
dpkg -r mysql-client-5.5
dpkg -r mysql-common
wget http://cdn.mysql.com/Downloads/MySQL-5.6/mysql-5.6.20-debian6.0-x86_64.deb && dpkg -i mysql-5.6.20-debian6.0-x86_64.deb
cd /usr/local && ln -s /opt/mysql/server-5.6 mysql && cd mysql && scripts/mysql_install_db --user=mysql --datadir=/var/lib/mysql
rm /opt/mysql/server-5.6/my.cnf && ln -s /var/ini/my.cnf /opt/mysql/server-5.6/my.cnf
cp support-files/mysql.server /etc/init.d/mysql
update-rc.d mysql defaults
service mysql start

2 step: Further upgrading

apt-get update && apt-get upgrade

What could have gone wrong during the upgrade since some of the ports is no longer open? How to re-open the closed ports?

The following ports are no longer open:

22 ssh
3306 mysql

Mysql is running (the webpage is accessible) But I can't connect to the server through SSH (putty) because port 22 has somehow been closed during upgrade

putty error

Server unexpected closed network connection

enter image description here

Best Answer

IIRC the default MySQL config is not to listen via TCP, but via socket file only, which explains why port 3309 isn't bound after the upgrade. It's likely that somehow you ended up with the default config.

Why sshd isn't running isn't discernable from the commands you shared, but likely some config error that makes sshd bail out on start. Without access to the system for you to give us more information, there's not enough to go on for thinking of anything else.

Update

According to the output you shared, sshd complains that openssl doesn't match its version. Get access to your box and run

apt-get update
apt-get install openssl openssh-server libssl1.0.0

If you get errors on updates, this should fix APT. Be careful though, you need to review your config after this:

apt-get install -f

That should take care of it.

Related Solutions

Linux – nmap shows opened port but netstat doesn’t

Are you on the same segment as the server in question? Portscanning via routers can give bogus results.

Linux port not accessible from remote machine

My guess is it's a firewall issue. To diagnose, either switch to root or use sudo to execute these commands described below:

Firstly, service iptables status will show you your current rules. Depending on how esoteric your firewall rules are, you may need to add them into the main question to debug but essentially you're looking for a line that accepts TCP port 8800 prior to a line that rejects all connections.

e.g.

Table: filter
Chain INPUT (policy ACCEPT)
num  target     prot opt source       destination
...
14   ACCEPT     tcp  --  0.0.0.0/0    0.0.0.0/0      state NEW tcp dpt:8800
15   REJECT     all  --  0.0.0.0/0    0.0.0.0/0      reject-with icmp-host-prohibited

If you don't have a REJECT line on your input chain, then the firewall is probably already turned off.

One way to get around this is obviously to turn off the firewall (service iptables stop) although this defeats the entire purpose of having a firewall.

If you need to add a new firewall rule, you have a couple of options:

Use the excellent Firewall Configuration tool (system-config-firewall) that ships with Fedora and can be found at the System->Administration->Firewall menu item in Gnome.
On the command line, insert a new rule before the REJECT rule by executing the following (uses numbers from my example above): iptables -I INPUT 15 -m state --state NEW -m tcp -p tcp --dport 8800 -j ACCEPT. NOTE that these changes are not permanent and will not persist beyond the next reboot.
Manually edit the /etc/sysconfig/iptables file and add a rule (-A INPUT -m state --state NEW -m tcp -p tcp --dport 8800 -j ACCEPT) to the firewall prior to the REJECT rule. Cycle the firewall service to pick up the changes via a service iptables restart. These changes will be applied every time the firewall service is started.