FTPS – Explicit FTP over TLS – can’t get directory listing

ftps

UPDATE –
Summary of what resolved it: After fixing a mis-typed port on the router, I got the error "450 TLS session of data connection has not resumed or the session does not match the control connection" – which led me to unticking the server setting "Require TLS session resumption on data connection when using PROT P" which fixed the problem. Not sure what disabling this "feature" means for security.
——

Original Post –
My goal is to get Explicit FTP over TLS working with Firezilla server. I've set up Firezilla and the Windows Firewall to let in the same range of ports.

Firewall Ports: 21, 50000-51000
Firezilla Server: Passive Ports: 50000-51000

My router also has this range of ports forwarded on TCP to the correct LAN IP.

My server is behind a NAT and has a static IP which is also listed on the passive settings tab of Firezilla Server. The only way I've had any success is when the client is on the same side of the NAT and only when straight up plain FTP and only in active mode.

I tested with ftptest.net and this is the log

Explicit FTP over TLS

Status: Resolving address of xxx.x.xxx.xxx
Status: Connecting to xxx.x.xxx.xxx
Warning: The entered address does not resolve to an IPv6 address. 
Status: Connected, waiting for welcome message... 
Reply: 220-FileZilla Server 0.9.60 beta 
Reply: 220 Welcome 
Command: CLNT https://ftptest.net on behalf of xxx.x.xxx.xxx 
Reply: 200 Don't care 
Command: AUTH TLS 
Reply: 234 Using authentication type TLS 
Status: Performing TLS handshake... 
Status: TLS handshake successful, verifying certificate... 
Status: Received 1 certificates from server. 
Status: cert[0]: subject='CN=xxx.x.xxx.xxx,C=US,ST=My state,L=My city' issuer='CN=xxx.x.xxx.xxx,C=US,ST=My state,L=My city' 
Command: USER username 
Reply: 331 Password required for username 
Command: PASS ************* 
Reply: 230 Logged on 
Command: SYST 
Reply: 215 UNIX emulated by FileZilla 
Command: FEAT 
Reply: 211-Features: 
Reply: MDTM 
Reply: REST STREAM 
Reply: SIZE 
Reply: MLST type*;size*;modify*; 
Reply: MLSD 
Reply: AUTH SSL 
Reply: AUTH TLS 
Reply: PROT 
Reply: PBSZ 
Reply: UTF8 
Reply: CLNT 
Reply: MFMT 
Reply: EPSV 
Reply: EPRT 
Reply: 211 End 
Command: PBSZ 0 
Reply: 200 PBSZ=0 
Command: PROT P 
Reply: 200 Protection level set to P 
Command: PWD 
Reply: 257 "/" is current directory. 
Status: Current path is / 
Command: TYPE I 
Reply: 200 Type set to I 
Command: PASV 
Reply: 227 Entering Passive Mode (xxx,x,xxx,xxx,196,44) 
Command: MLSD 
Reply: 425 Can't open data connection for transfer of "/" 
Error: Listing failed

Allow fallback to plain FTP

Warning: Allowing fallback to plaintext FTP is insecure. You should use explicit FTP over TLS.
Status: Resolving address of xxx.x.xxx.xxx 
Status: Connecting to xxx.x.xxx.xxx 
Warning: The entered address does not resolve to an IPv6 address. 
Status: Connected, waiting for welcome message... 
Reply: 220-FileZilla Server 0.9.60 beta 
Reply: 220 Welcome 
Command: CLNT https://ftptest.net on behalf of xxx.x.xxx.xxx 
Reply: 200 Don't care 
Command: AUTH TLS 
Reply: 234 Using authentication type TLS 
Status: Performing TLS handshake... 
Status: TLS handshake successful, verifying certificate... 
Status: Received 1 certificates from server. 
Status: cert[0]: subject='CN=xxx.x.xxx.xxx,C=US,ST=My state,L=My city' issuer='CN=xxx.x.xxx.xxx,C=US,ST=My state,L=My city' 
Command: USER username 
Reply: 331 Password required for username 
Command: PASS ************* 
Reply: 230 Logged on 
Command: SYST 
Reply: 215 UNIX emulated by FileZilla 
Command: FEAT 
Reply: 211-Features: 
Reply: MDTM 
Reply: REST STREAM 
Reply: SIZE 
Reply: MLST type*;size*;modify*; 
Reply: MLSD 
Reply: AUTH SSL 
Reply: AUTH TLS 
Reply: PROT 
Reply: PBSZ 
Reply: UTF8 
Reply: CLNT 
Reply: MFMT 
Reply: EPSV 
Reply: EPRT 
Reply: 211 End 
Command: PBSZ 0 
Reply: 200 PBSZ=0 
Command: PROT P 
Reply: 200 Protection level set to P 
Command: PWD 
Reply: 257 "/" is current directory. 
Status: Current path is / 
Command: TYPE I 
Reply: 200 Type set to I 
Command: PASV 
Reply: 227 Entering Passive Mode (xxx,x,xxx,xxx,196,107) 
Command: MLSD 
Reply: 425 Can't open data connection for transfer of "/" 
Error: Listing failed

I'd appreciate any help in getting past this. Thanks in advance.

Best Answer

The problem with FTP over TLS with both firewalls and NAT appliances is two-fold.

  1. FTP is a crappy protocol that requires two connections:

    1. in passive FTP that is initially one control connection to a default port, TCP 21
    2. to actually transfer data (and getting a directory listing is a data transfer) the client needs to make a second TCP connection, the data connection.
      After an FTP client requests a passive ftp connection with the PASV control word the FTP server selects a "random available ephemeral", non-default port and announces that port with the PORT response.
      The PORT response typically contains 6 octets, eg PORT h1,h2,h3,h4,p1,p2 where h1.h2.h3.h4 is the server's IPv4 IP-address and the TCP port number is (p1*256) + p2 i.e. xxx,x,xxx,xxx,196,107 represents port (196 * 256) + 107 = 50283

  2. In FTP over TLS the connection between client and FTP server is encrypted. That results in problems with servers that are protected by firewalls and which are behind NAT devices.

Firewall issues
Normally in an intelligent firewall when you allow FTP you need to open the port for the control connection, TCP 21 and then, in the clear text FTP protocol, the firewall can scan for and detect the PORT response. An FTP helper module will then automatically open up the port number that gets assigned by the FTP server to that specific client, allowing for quite granular access control.

When the connection is encrypted with TLS the firewall can't detect the PORT response anymore and therefor not automatically open the assigned port.
The solution for that is to :

  • fix the range of ports the FTP server will use for passive connections to a small range
  • in your firewall open both port 21 and that fixed range of ports for data connections

NAT issues
When the FTP server is behind a device that does network address translation and port forwarding traffic on the control connection on port 21 will be forwarded from the (external) ip-address of the NAT device to port 21 on the FTP server.
In addition it will run a similar NAT FTP helper module that in the clear text FTP protocol, can scan for and detect the PORT response.
Upon seeing a PORT response it will typically make two changes:

  • In the response PORT h1,h2,h3,h4,p1,p2 the ip-address h1,h2,h3,h4 of the FTP server will be replaced with e1,e2,e3,e4 representing the (external) ip-address e1.e2.e3.e4 of the NAT server that the client is connected to.
  • A dynamic port forwarding rule is made for that client forwarding the port defined by p1,p2 from the NAT device to the FTP server.

When the FTP connection is encrypted with TLS the NAT device can't detect the PORT response and therefor the NAT device can neither rewrite the ip-address nor automatically create the required port forwarding rules anymore.
The typical solution for such NAT issues is then to :

  • reconfigure the FTP server so that rather than with it's own ip-address h1.h2.h3.h4 it will use the external ip-address e1.e2.e3.e4 of the NAT device in PORT responses.
  • fix the range of ports the FTP server will use for passive connections to a small range
  • In addition to TCP port 21 also create explicit port forwarding (NAT) rules for every port in the passive port range you configured in the FTP server to the corresponding port on the FTP server, I.e. 50000 --> 50000 , 50001 --> 50001 , 50002 --> 50002 etc. .

Don't overlook that in addition to the host based firewall on the FTP server there may be additional firewalls or security groupings involved. For instance in addition to the port forwarding rules you may also need to create firewall rules in the NAT device.

Related Topic