Java FTPS fails to retrieve file list (FileZilla Client works fine)

I am using Apache Commons Net (v3.5) with a Java 8 to connect to a remote FTPS site (i.e. out on the internet). I am able to easily connect with a FileZilla client on my Windows 10 machine, but my Java program is unable to complete the same steps. I've googled high and low, but cannot find the root cause. Here are things that I have confirmed:

I ensured the Java FTP commands are in the exact same order as the FileZilla client.
I disabled Windows Firewall and Anti-Virus on the PC
I re-enabled Windows Firewall and enabled logging. When using FileZilla, the Windows Firewall Log lists the TCP connection when the passive mode connection is established. I see no such entry with the Java program.
I installed a FileZilla server on my PC. The java program worked after I un-checked "Require TLS session resumption on data connection when using PROT P." The Java exception was different, so I do not believe this is a smoking gun.
I successfully ran this same code against test.rebex.com server.

Below is the code and any thoughts are greatly appreciated:

import java.io.IOException;
import java.io.PrintWriter;
import org.apache.commons.net.PrintCommandListener;
import org.apache.commons.net.ftp.FTP;
import org.apache.commons.net.ftp.FTPFile;
import org.apache.commons.net.ftp.FTPReply;
import org.apache.commons.net.ftp.FTPSClient;

public class testProgram {

  public static void main(String[] args) {

    String ftpServer = "ftp.domain.com";
    String ftpUsername = "user@domain.com";
    String ftpPassword = "********";

    FTPSClient ftp = null;

    // CONNECT TO THE SERVER
    try {
        // I have tried "SSL" as the argument, but same result
        ftp = new FTPSClient(); 
        ftp.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out)));

        ftp.connect(ftpServer,21);

        int reply = ftp.getReplyCode();

        if (!FTPReply.isPositiveCompletion(reply)) {
            ftp.disconnect();
            System.err.println("---------->FTP server refused connection.\n");

        } 

    } catch (Exception e) {
        System.out.println(e.getMessage());
        e.printStackTrace();

    }

    // LOGIN INTO SERVER
    try {
        if (!ftp.login(ftpUsername, ftpPassword)) {
            ftp.logout();

        } else {

            ftp.sendCommand("OPTS UTF8 ON");            
            ftp.execPBSZ(0);            
            ftp.execPROT("P");
            ftp.pwd();
            ftp.setFileType(FTP.BINARY_FILE_TYPE);      
            ftp.enterLocalPassiveMode();

            /* The next command always fails.

               The FTP Server responds with "150 Accepted data connection" then:

                org.apache.commons.net.ftp.FTPConnectionClosedException: Connection closed without indication.
                at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:316)
                at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:292)
                at org.apache.commons.net.ftp.FTP.getReply(FTP.java:712)
                at org.apache.commons.net.ftp.FTPClient.completePendingCommand(FTPClient.java:1857)
                at org.apache.commons.net.ftp.FTPClient.listNames(FTPClient.java:2919)
                at org.apache.commons.net.ftp.FTPClient.listNames(FTPClient.java:2952)
                at myPackage.testProgram.main(testProgram.java:78)

                I have tried other commands, but it disconnects here...
             */

            FTPFile[] ftpFiles = ftp.listFiles();
            System.out.println("---------->Number of Files = " + ftpFiles.length);
            ftp.logout();

        }
    } catch (Exception e) {

        e.printStackTrace();
    } 

    //Ensure Disconnected at the end.
    if (ftp.isConnected()) {
        try {
            ftp.disconnect();
        } catch (IOException f) {
            // do nothing
        }

    }
  }
}

Here is the FileZilla Client log from my PC:

2016-09-06 09:09:50 4756 1 Status: Resolving address of ftp.domain.com
2016-09-06 09:09:51 4756 1 Status: Connecting to h1.h2.h3.h4:21...
2016-09-06 09:09:51 4756 1 Status: Connection established, waiting for welcome message...
2016-09-06 09:09:51 4756 1 Response: 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
2016-09-06 09:09:51 4756 1 Response: 220-You are user number 2 of 50 allowed.
2016-09-06 09:09:51 4756 1 Response: 220-Local time is now 13:09. Server port: 21.
2016-09-06 09:09:51 4756 1 Response: 220-This is a private system - No anonymous login
2016-09-06 09:09:51 4756 1 Response: 220-IPv6 connections are also welcome on this server.
2016-09-06 09:09:51 4756 1 Response: 220 You will be disconnected after 15 minutes of inactivity.
2016-09-06 09:09:51 4756 1 Command: AUTH TLS
2016-09-06 09:09:51 4756 1 Response: 234 AUTH TLS OK.
2016-09-06 09:09:51 4756 1 Status: Initializing TLS...
2016-09-06 09:09:51 4756 1 Status: Verifying certificate...
2016-09-06 09:09:51 4756 1 Status: TLS connection established.
2016-09-06 09:09:51 4756 1 Command: USER user@domain.com
2016-09-06 09:09:51 4756 1 Response: 331 User user@domain.com OK. Password required
2016-09-06 09:09:51 4756 1 Command: PASS *************
2016-09-06 09:09:51 4756 1 Response: 230 OK. Current restricted directory is /
2016-09-06 09:09:51 4756 1 Command: SYST
2016-09-06 09:09:51 4756 1 Response: 215 UNIX Type: L8
2016-09-06 09:09:51 4756 1 Command: FEAT
2016-09-06 09:09:51 4756 1 Response: 211-Extensions supported:
2016-09-06 09:09:51 4756 1 Response:  EPRT
2016-09-06 09:09:51 4756 1 Response:  IDLE
2016-09-06 09:09:51 4756 1 Response:  MDTM
2016-09-06 09:09:51 4756 1 Response:  SIZE
2016-09-06 09:09:51 4756 1 Response:  MFMT
2016-09-06 09:09:51 4756 1 Response:  REST STREAM
2016-09-06 09:09:51 4756 1 Response:  MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;
2016-09-06 09:09:51 4756 1 Response:  MLSD
2016-09-06 09:09:51 4756 1 Response:  AUTH TLS
2016-09-06 09:09:51 4756 1 Response:  PBSZ
2016-09-06 09:09:51 4756 1 Response:  PROT
2016-09-06 09:09:51 4756 1 Response:  UTF8
2016-09-06 09:09:51 4756 1 Response:  TVFS
2016-09-06 09:09:51 4756 1 Response:  ESTA
2016-09-06 09:09:51 4756 1 Response:  PASV
2016-09-06 09:09:51 4756 1 Response:  EPSV
2016-09-06 09:09:51 4756 1 Response:  SPSV
2016-09-06 09:09:51 4756 1 Response:  ESTP
2016-09-06 09:09:51 4756 1 Response: 211 End.
2016-09-06 09:09:51 4756 1 Command: OPTS UTF8 ON
2016-09-06 09:09:51 4756 1 Response: 200 OK, UTF-8 enabled
2016-09-06 09:09:51 4756 1 Command: PBSZ 0
2016-09-06 09:09:51 4756 1 Response: 200 PBSZ=0
2016-09-06 09:09:51 4756 1 Command: PROT P
2016-09-06 09:09:52 4756 1 Response: 200 Data protection level set to "private"
2016-09-06 09:09:52 4756 1 Status: Logged in
2016-09-06 09:09:52 4756 1 Status: Retrieving directory listing...
2016-09-06 09:09:52 4756 1 Command: PWD
2016-09-06 09:09:52 4756 1 Response: 257 "/" is your current location
2016-09-06 09:09:52 4756 1 Command: TYPE I
2016-09-06 09:09:52 4756 1 Response: 200 TYPE is now 8-bit binary
2016-09-06 09:09:52 4756 1 Command: PASV
2016-09-06 09:09:52 4756 1 Response: 227 Entering Passive Mode (h1,h2,h3,h4,133,150)
2016-09-06 09:09:52 4756 1 Command: MLSD
2016-09-06 09:09:52 4756 1 Response: 150 Accepted data connection
2016-09-06 09:09:52 4756 1 Response: 226-Options: -a -l 
2016-09-06 09:09:52 4756 1 Response: 226 6 matches total

Using Mike's suggesting, I turned on the TLS debugging. It appears the program goes through the TLS handshake again. The output is very long, but after issuing the list command, I see "*** ClientHello, TLSv1.2" and what looks like the same commands as initiating the FTP connection.

The difference appears to come at the end:

%% Cached client session: [Session-2, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256]
main, received EOFException: ignored
main, called closeInternal(false)
main, SEND TLSv1.2 ALERT:  warning, description = close_notify
main, WRITE: TLSv1.2 Alert, length = 26
main, called closeSocket(false)
main, called close()
main, called closeInternal(true)
main, called close()
main, called closeInternal(true)
main, received EOFException: ignored
main, called closeInternal(false)
main, SEND TLSv1.2 ALERT:  warning, description = close_notify
main, WRITE: TLSv1.2 Alert, length = 26
main, called closeSocket(false)
org.apache.commons.net.ftp.FTPConnectionClosedException: Connection closed without indication.

Best Answer

Although this looks like an old post, I was faced with a similar problem today and could not figure out (initially) the solution. I could connect via FileZilla but not with FTPSClient and after running ftpClient.enterLocalPassiveMode(), I used to get 425 cannot open data connection

My solution was to change the ftpClient.enterLocalPassiveMode() before logging in but after connecting to the FTPServer and it worked. Usually all code examples I saw uses the enterlocalpassivemode before sending/receiving the data but after login. See below code for an example which worked for me.

FTPSClient ftpClient = new FTPSClient(false);
ftpClient.connect("remote.ftp.server", port);
ftpClient.enterLocalPassiveMode();// Run the passive mode command now  instead of after loggin in.
ftpClient.login("username", "password");
ftpClient.execPBSZ(0);
ftpClient.execPROT("P");
ftpClient.type(FTP.BINARY_FILE_TYPE);
//ftpClient.enterLocalPassiveMode(); Previously it was here.
FTPFile[] files = ftpClient.listDirectories("/");

Hope this helps. Please also note that all other code and good practices are omitted to keep the answer short.

Best Answer

Related Solutions

Java – Creating FTP Client Login Issue In java

Java – How to get files from FTPS server with java

Related Topic