SSH Authentication – Public Key Authentication Over HTTP/HTTPS

Your question says you want to log in to the webserver from the CI server. But you then say you created the keys on the webserver. I'm not sure whether this is just a misunderstanding or a mistype.

Here's a brief overview of what you need to do to enable logging in to the webserver from the CI server with keys:

1. Create the public/private key pair on the CI server
2. Copy the public part of the key (id_rsa.pub) onto the webserver into the ~/.ssh/authroized_keys file for each user you want to log in as (root, user etc.). I usually just use scp to copy the file over to the server. If you have multiple keys that are going to be used for each user, append them to authorized_keys. Otherwise, if this is the only key for that user, you can just rename id_rsa.pub to authorized_keys.
3. Make sure the permissions on the .ssh folder are 644 with chmod 644 .ssh and the authorized key file has permissions of 700 with chmod 700 .ssh/authorized_keys.
4. Attempt to log into the webserver from the CI server with a username who you have copied the public key to on the webserver e.g. ssh user@webserver

It shouldn't ask for a password (this is assuming you have followed @khaled's advice regarding the sshd config file but the defaults are typically set to allow key authentication before password).

You can't (with only the public key). The user may change or remove the password for the private key without affecting the server.

There is a similar question: How to tell if a public SSH key has a passphrase

If you are implementing two-factor authentication, maybe you could consider PAM or a custom script. Making it work with some SFTP clients may be a pain though.