Unable to access vsftpd from public ip

I am trying to setup a vsftpd on ubuntu

I have installed successfully installed vsftpd

My networks works like this

Lan network on host(localhost,127.0.0.1, 192.168.1.105) -> connects to router(192.168.1.1(LAN) -> 10.255.1.204(WAN)) -> connects to ISP (10.255.1.1 -> DNS -> 106.34.26.78(public ip)). (Note: The ip addresses used is just to illustrate and not real ones)

I have enabled port forwarding on my router, which redirects all requests to port 21 to my local machine on which vsftp deamon is running.

I am able to access ftp from my internal LAN addresses and the interface that connects to ISP (i.e., 10.255.1.204). But I am unable to access it from my public ip address(i.e., 106.34.26.78), I get the following message:

Status: Connecting to 106.34.26.78:21...
Status: Connection established, waiting for welcome message...
Response:   220 (vsFTPd 3.0.2)
Command:    AUTH TLS
Error:  Connection timed out after 20 seconds of inactivity
Error:  Could not connect to server

My vsftpd.conf (/etc/vsftpd.conf)

listen=YES
anonymous_enable=NO
local_enable=YES
write_enable=YES
local_umask=022
local_root=/var/www
chroot_local_user=YES
allow_writeable_chroot=YES
hide_ids=YES

#virutal user settings
user_config_dir=/etc/vsftpd_user_conf
guest_enable=YES
virtual_use_local_privs=YES
pam_service_name=vsftpd
nopriv_user=vsftpd
guest_username=vsftpd

My vsftpd (/etc/pam.d/vsftpd)

auth required pam_pwdfile.so pwdfile /etc/vsftpd/ftpd.passwd
account required pam_permit.so

I am breaking my head over this since a month now, I found few articles, But they were not related to access using public IP.

EDIT

I have also enabled port forwarding on port 20.