Ssh-copy-id specifying which key and without password

Use ssh-copy-id user@client and it will copy the correct data to the correct place. If it's the same user as you, though, it can be problematic since it may overwrite something important. Make a copy of your .ssh directory before trying.

EDIT: Strangely enough, I tried to add my id_rsa.pub to my .ssh/authorized_keys and it worked. Are you sure that your local sshd is configured to accept shared key authentication?

Don't use a password. Generate a passphrase-less SSH key and push it to your VM.

If you already have an SSH key, you can skip this step… Just hit Enter for the key and both passphrases:

$ ssh-keygen -t rsa -b 2048
Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/username/.ssh/id_rsa.
Your public key has been saved in /home/username/.ssh/id_rsa.pub.

Copy your keys to the target server:

$ ssh-copy-id id@server
id@server's password:

Now try logging into the machine, with ssh 'id@server', and check-in:

.ssh/authorized_keys

Note: If you don't have .ssh dir and authorized_keys file, you need to create it first

to make sure we haven’t added extra keys that you weren’t expecting.

Finally, check to log in…

$ ssh id@server

id@server:~$

You may also want to look into using ssh-agent if you want to try keeping your keys protected with a passphrase.