Mysql – Cannot connect to MySQL over TCP locally – Connection Timeout – Ubuntu 9.04

connectionMySQLtimeoutubuntu-9.04

I am running Ubuntu and am ultimately trying to connect Tomcat to my MySQL database using JDBC.

It has worked previously but after a reboot the instance now fails to connect.

  • Both Tomcat 6 and MySQL 5.0.75 are on the same machine
  • Connection string: jdbc:mysql:///localhost:3306
  • I can connect to MySQL on the command line using the mysql command
  • The my.cnf file is pretty standard (Available on request) has bind address: 127.0.0.1
  • I cannot Telnet to the MySQL port despite netstat saying MySQL is listening
  • I have one IpTables rule to forward 80 -> 8080 and no firewall I'm aware of.

I'm pretty new to this and I'm not sure what else to test. I don't know whether I should be looking in etc/interfaces and if I did what to look for. It's weird because it used to work but after a reboot it's down so I must have changed something…. :).

I realise a timeout indicates the server is not responding and I assume it's because the request isn't actually getting through. I installed MySQL via apt-get and Tomcat manually.

MySqld processes

root@88:/var/log/mysql#  ps -ef | grep mysqld
root     21753     1  0 May27 ?        00:00:00 /bin/sh /usr/bin/mysqld_safe
mysql    21792 21753  0 May27 ?        00:00:00 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --port=3306 --socket=/var/run/mysqld/mysqld.sock
root     21793 21753  0 May27 ?        00:00:00 logger -p daemon.err -t mysqld_safe -i -t mysqld
root     21888 13676  0 11:23 pts/1    00:00:00 grep mysqld

Netstat

root@88:/var/log/mysql# netstat -lnp | grep mysql
tcp        0      0 0.0.0.0:3306            0.0.0.0:*               LISTEN      21792/mysqld
unix  2      [ ACC ]     STREAM     LISTENING     1926205077 21792/mysqld        /var/run/mysqld/mysqld.sock

Toy Connection Class

root@88:~# cat TestConnect/TestConnection.java

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;

public class TestConnection {

  public static void main(String args[]) throws Exception {
    Connection con = null;

    try {
      Class.forName("com.mysql.jdbc.Driver").newInstance();
      System.out.println("Got driver");
      con = DriverManager.getConnection(
                "jdbc:mysql:///localhost:3306",
                "uname", "pass");
      System.out.println("Got connection");

      if(!con.isClosed())
        System.out.println("Successfully connected to " +
          "MySQL server using TCP/IP...");

    } finally {
        if(con != null)
          con.close();
    }
  }
}

Toy Connection Class Output

Note: This is the same error I get from Tomcat.

root@88:~/TestConnect# java -cp mysql-connector-java-5.1.12-bin.jar:. TestConnection
Got driver
                Exception in thread "main" com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 1 milliseconds ago. The driver has not received any packets from the server.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1122)            
        at TestConnection.main(TestConnection.java:14)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1122)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:344)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2181)
        ... 12 more
Caused by: java.net.ConnectException: Connection timed out
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        ... 13 more

Telnet Output

root@88:~/TestConnect# telnet localhost 3306
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection timed out

Iptables

NOTE: I did have a rule setup in for NAT but I've removed it and the problem persists.

root@88:~# iptables -nL
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

root@88:~# iptables -t nat -nL
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Command used to setup iptables

iptables -t nat -I PREROUTING --src 0/0 --dst 88.198.31.14 -p tcp --dport 80 -j REDIRECT --to-ports 8080

I have since removed the rule this setup for the NAT so unbless there's a side effect to this command it's not relevant.

UPDATE -I can't ping localhost

ip a

root@88:~/TestConnect# ip a
1: lo: <LOOPBACK> mtu 16436 qdisc noop state DOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3: venet0: <BROADCAST,POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/void
    inet 127.0.0.1/32 scope host venet0
    inet 88.198.31.14/32 scope global venet0:0

ip r

root@88:~/TestConnect# ip r
    192.0.2.1 dev venet0  scope link
    default via 192.0.2.1 dev venet0

ip rule

root@88:~/TestConnect# ip rule
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default

ping -c 1 localhost

root@88:~/TestConnect# ping -c 1 localhost
PING localhost.localdomain (127.0.0.1) 56(84) bytes of data.

--- localhost.localdomain ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

cat /etc/hosts

root@88:~/TestConnect# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
# Auto-generated hostname. Please do not remove this comment.
88.198.31.14 88.198.31.14  88 88

ifconfig

root@88:/var/log/mysql# ifconfig
venet0    Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet addr:127.0.0.1  P-t-P:127.0.0.1  Bcast:0.0.0.0  Mask:255.255.255.255
          UP BROADCAST POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1
          RX packets:144432 errors:0 dropped:0 overruns:0 frame:0
          TX packets:153825 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:37896766 (37.8 MB)  TX bytes:28722595 (28.7 MB)

venet0:0  Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet addr:88.198.31.14  P-t-P:88.198.31.14  Bcast:0.0.0.0  Mask:255.255.255.255
          UP BROADCAST POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1

Thanks to pQd I think we've narrowed down the error, I need to go read up about what this all means and why localhost isn't ping-able.

Best Answer

are you absolutely sure the is no iptables rule preventing tcp/3306 traffic? try this as temporary workaround:

iptables -I INPUT -p tcp --dport 3306 -i lo -j ACCEPT
iptables -A OUTPUT -p tcp --sport 3306 -o lo -j ACCEPT

or maybe your NAT rules to a little bit more than you expect them to do?

ok - your loopback is not only down, but does not have 127.0.0.1 bound to it. add to /etc/network/interfaces:

auto lo
iface lo inet loopback

and.. why on earth did you bound it to vnet0? remove it from there.

quick one-time cmd line fix:

ip a d 127.0.0.1/32 dev venet0
ip a a 127.0.0.1/32 dev lo
ip link set dev lo up