Linux – SSH Lockout after failed login attempts

Don't use a password. Generate a passphrase-less SSH key and push it to your VM.

If you already have an SSH key, you can skip this step… Just hit Enter for the key and both passphrases:

$ ssh-keygen -t rsa -b 2048
Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/username/.ssh/id_rsa.
Your public key has been saved in /home/username/.ssh/id_rsa.pub.

Copy your keys to the target server:

$ ssh-copy-id id@server
id@server's password:

Now try logging into the machine, with ssh 'id@server', and check-in:

.ssh/authorized_keys

Note: If you don't have .ssh dir and authorized_keys file, you need to create it first

to make sure we haven’t added extra keys that you weren’t expecting.

Finally, check to log in…

$ ssh id@server

id@server:~$

You may also want to look into using ssh-agent if you want to try keeping your keys protected with a passphrase.

You also need ChallengeResponseAuthentication yes in /etc/ssh/sshd_config.

To display the error, pam needs a conversation function.

This option tells ssh to provide a more complete PAM conversation function, which covers amongst other things providing output and asking for arbitrary input (instead of just being handed over a password by sshd).

Edit: You'll want PasswordAuthentication no to make sure the password input always goes through this PAM conversation.