Ssh – Fabric used to work with ssh-keys, but not anymore

I have my ssh key set up correctly and I can ssh into the remote machine without password.

However, my fabric deploy script which could authenticate automatically until a couple of hours ago, started to ask me for my private key passphrase whenever I tried to deploy to the remote machine.

Even if I enter my passphrase, it doesn't seem to care whether I give the correct ones and will prompt the systems login, as if I do not have ssh keys at all.

Please keep in mind that I am a developer and not a system engineer, who are not familiar with how Fabric works. The possibility that I have missed something obvious is quite high. Any solution or even a pointer of where to look will be greatly appreciated.

This is a proofing script which I derived from the real deploy script:

#!/usr/bin/env python2

import sys
from fabric.api import *
from fabric.network import ssh

ssh.util.log_to_file("paramiko.log", 10)

@hosts(['host-ip-address'])

def boot():
    env.use_ssh_config = True
    env.user = 'userlogin'
    env.key_filename = ['~/.ssh/some-public-key.pem']
    env.gateway = 'user@gateway.com'
    execute(a)

def a():
    run('ls')

boot()

And here is the log message from paramiko:

DEB [20141007-19:22:50.804] thr=1   paramiko.transport: starting thread (client mode): 0xfe2790L
INF [20141007-19:22:50.847] thr=1   paramiko.transport: Connected (version 2.0, client OpenSSH_6.2)
DEB [20141007-19:22:50.856] thr=1   paramiko.transport: kex algos:[u'ecdh-sha2-nistp256', u'ecdh-sha2-nistp384', u'ecdh-sha2-nistp521', u'diffie-hellman-group-exchange-sha256', u'diffie-hellman-group-exchange-sha1', u'diffie-hellman-group14-sha1', u'diffie-hellman-group1-sha1'] server key:[u'ssh-rsa', u'ssh-dss', u'ecdsa-sha2-nistp256'] client encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'arcfour256', u'arcfour128', u'aes128-gcm@openssh.com', u'aes256-gcm@openssh.com', u'aes128-cbc', u'3des-cbc', u'blowfish-cbc', u'cast128-cbc', u'aes192-cbc', u'aes256-cbc', u'arcfour', u'rijndael-cbc@lysator.liu.se'] server encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'arcfour256', u'arcfour128', u'aes128-gcm@openssh.com', u'aes256-gcm@openssh.com', u'aes128-cbc', u'3des-cbc', u'blowfish-cbc', u'cast128-cbc', u'aes192-cbc', u'aes256-cbc', u'arcfour', u'rijndael-cbc@lysator.liu.se'] client mac:[u'hmac-md5-etm@openssh.com', u'hmac-sha1-etm@openssh.com', u'umac-64-etm@openssh.com', u'umac-128-etm@openssh.com', u'hmac-sha2-256-etm@openssh.com', u'hmac-sha2-512-etm@openssh.com', u'hmac-ripemd160-etm@openssh.com', u'hmac-sha1-96-etm@openssh.com', u'hmac-md5-96-etm@openssh.com', u'hmac-md5', u'hmac-sha1', u'umac-64@openssh.com', u'umac-128@openssh.com', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-ripemd160', u'hmac-ripemd160@openssh.com', u'hmac-sha1-96', u'hmac-md5-96'] server mac:[u'hmac-md5-etm@openssh.com', u'hmac-sha1-etm@openssh.com', u'umac-64-etm@openssh.com', u'umac-128-etm@openssh.com', u'hmac-sha2-256-etm@openssh.com', u'hmac-sha2-512-etm@openssh.com', u'hmac-ripemd160-etm@openssh.com', u'hmac-sha1-96-etm@openssh.com', u'hmac-md5-96-etm@openssh.com', u'hmac-md5', u'hmac-sha1', u'umac-64@openssh.com', u'umac-128@openssh.com', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-ripemd160', u'hmac-ripemd160@openssh.com', u'hmac-sha1-96', u'hmac-md5-96'] client compress:[u'none', u'zlib@openssh.com'] server compress:[u'none', u'zlib@openssh.com'] client lang:[u''] server lang:[u''] kex follows?False
DEB [20141007-19:22:50.856] thr=1   paramiko.transport: Ciphers agreed: local=aes128-ctr, remote=aes128-ctr
DEB [20141007-19:22:50.856] thr=1   paramiko.transport: using kex diffie-hellman-group14-sha1; server key type ssh-rsa; cipher: local aes128-ctr, remote aes128-ctr; mac: local hmac-sha1, remote hmac-sha1; compression: local none, remote none
DEB [20141007-19:22:50.903] thr=1   paramiko.transport: Switch to new keys ...
DEB [20141007-19:22:50.907] thr=2   paramiko.transport: Trying key 3433922e6c7bef8672043fbfe07c22f3 from /Users/ganeshwara/.dme/sg-master.pem
DEB [20141007-19:22:50.954] thr=1   paramiko.transport: userauth is OK
INF [20141007-19:22:51.033] thr=1   paramiko.transport: Authentication (publickey) failed.
DEB [20141007-19:22:51.040] thr=2   paramiko.transport: Trying SSH agent key f8f4822cc3d40df79b9f7c79f219c42c
DEB [20141007-19:22:51.045] thr=1   paramiko.transport: userauth is OK
INF [20141007-19:22:51.061] thr=1   paramiko.transport: Authentication (publickey) failed.
DEB [20141007-19:22:51.704] thr=3   paramiko.transport: starting thread (client mode): 0xff0a10L
INF [20141007-19:22:51.723] thr=3   paramiko.transport: Connected (version 2.0, client OpenSSH_6.2)
DEB [20141007-19:22:51.728] thr=3   paramiko.transport: kex algos:[u'ecdh-sha2-nistp256', u'ecdh-sha2-nistp384', u'ecdh-sha2-nistp521', u'diffie-hellman-group-exchange-sha256', u'diffie-hellman-group-exchange-sha1', u'diffie-hellman-group14-sha1', u'diffie-hellman-group1-sha1'] server key:[u'ssh-rsa', u'ssh-dss', u'ecdsa-sha2-nistp256'] client encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'arcfour256', u'arcfour128', u'aes128-gcm@openssh.com', u'aes256-gcm@openssh.com', u'aes128-cbc', u'3des-cbc', u'blowfish-cbc', u'cast128-cbc', u'aes192-cbc', u'aes256-cbc', u'arcfour', u'rijndael-cbc@lysator.liu.se'] server encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'arcfour256', u'arcfour128', u'aes128-gcm@openssh.com', u'aes256-gcm@openssh.com', u'aes128-cbc', u'3des-cbc', u'blowfish-cbc', u'cast128-cbc', u'aes192-cbc', u'aes256-cbc', u'arcfour', u'rijndael-cbc@lysator.liu.se'] client mac:[u'hmac-md5-etm@openssh.com', u'hmac-sha1-etm@openssh.com', u'umac-64-etm@openssh.com', u'umac-128-etm@openssh.com', u'hmac-sha2-256-etm@openssh.com', u'hmac-sha2-512-etm@openssh.com', u'hmac-ripemd160-etm@openssh.com', u'hmac-sha1-96-etm@openssh.com', u'hmac-md5-96-etm@openssh.com', u'hmac-md5', u'hmac-sha1', u'umac-64@openssh.com', u'umac-128@openssh.com', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-ripemd160', u'hmac-ripemd160@openssh.com', u'hmac-sha1-96', u'hmac-md5-96'] server mac:[u'hmac-md5-etm@openssh.com', u'hmac-sha1-etm@openssh.com', u'umac-64-etm@openssh.com', u'umac-128-etm@openssh.com', u'hmac-sha2-256-etm@openssh.com', u'hmac-sha2-512-etm@openssh.com', u'hmac-ripemd160-etm@openssh.com', u'hmac-sha1-96-etm@openssh.com', u'hmac-md5-96-etm@openssh.com', u'hmac-md5', u'hmac-sha1', u'umac-64@openssh.com', u'umac-128@openssh.com', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-ripemd160', u'hmac-ripemd160@openssh.com', u'hmac-sha1-96', u'hmac-md5-96'] client compress:[u'none', u'zlib@openssh.com'] server compress:[u'none', u'zlib@openssh.com'] client lang:[u''] server lang:[u''] kex follows?False
DEB [20141007-19:22:51.728] thr=3   paramiko.transport: Ciphers agreed: local=aes128-ctr, remote=aes128-ctr
DEB [20141007-19:22:51.728] thr=3   paramiko.transport: using kex diffie-hellman-group14-sha1; server key type ssh-rsa; cipher: local aes128-ctr, remote aes128-ctr; mac: local hmac-sha1, remote hmac-sha1; compression: local none, remote none
DEB [20141007-19:22:51.799] thr=3   paramiko.transport: Switch to new keys ...
DEB [20141007-19:22:51.806] thr=2   paramiko.transport: Trying key 3433922e6c7bef8672043fbfe07c22f3 from /Users/ganeshwara/.dme/sg-master.pem
DEB [20141007-19:22:51.848] thr=3   paramiko.transport: userauth is OK
INF [20141007-19:22:51.932] thr=3   paramiko.transport: Authentication (publickey) failed.
DEB [20141007-19:22:51.938] thr=2   paramiko.transport: Trying SSH agent key f8f4822cc3d40df79b9f7c79f219c42c
DEB [20141007-19:22:51.943] thr=3   paramiko.transport: userauth is OK
INF [20141007-19:22:51.955] thr=3   paramiko.transport: Authentication (publickey) failed.
DEB [20141007-19:22:51.979] thr=3   paramiko.transport: userauth is OK
DEB [20141007-19:22:51.984] thr=3   paramiko.transport: Authentication type (password) not permitted.
DEB [20141007-19:22:51.985] thr=3   paramiko.transport: Allowed methods: [u'publickey']
DEB [20141007-19:22:52.717] thr=1   paramiko.transport: EOF in transport thread
DEB [20141007-19:22:52.794] thr=3   paramiko.transport: EOF in transport thread

EDIT:

My ~/.ssh/config

Host gateway
 HostName gateway.com
 User user
 Port 22
 IdentityFile ~/.ssh/some-public-key.pem

Host remote-machine
 HostName host-ip-address
 User userlogin
 IdentityFile ~/.ssh/remote-machine.pem
 ProxyCommand ssh gateway -W %h:%p

I would like to clarify that on my local machine, I can indeed connect to remote-machine successfully, like this:

# ssh remote-machine