GitHub Account Password Change – Effect on Generated SSH Keys

GitHub user account matching for commits is done via the email address associated with the git commit on one side (i.e. your user.email setting) and those associated with the GitHub account on the other (see “About commit email addresses” and “Why are my commits linked to the wrong user?” in the GitHub User Documentation). To match your commits to your account, just add their associated email address(es) to your GitHub account.

You got it almost right, you'll have to add the other email to your profile settings. However, Github picks up the first listed email address for your gravatar.

So you'll have to remove the email ids such that the preferred gravatar email id bubbles up. Once done, you can re-add the removed email ids.