How to manage multiple SSH keys

Reduce your secure backup needs to a single key file.
192 readers like this.
Data stack in blue

In the first two articles in this series, I shared how you can use your GPG key to authenticate your SSH connections and import existing SSH keys. Once you're using more than two or three SSH keys, managing them can become more complicated. In this final article, I'll explain how to manage multiple SSH keys using the control file and how to make changes to manage them as GPG authentication subkeys. I'll also show how to identify your GPG authentication keys in case they've built up over time and you haven't kept notes.

Why have more than one SSH key?

If you think about SSH keys like you would physical keys, you'll start to see reasons to have more than one key. You wouldn't want to have your car and your house on the same key. If your key gets stolen, the thief has access to everything. Much like your car and house, you may want to have separate keys for each project, use case, or aspect of your life.

Having multiple keys can also be useful when you are doing work for clients. Some clients may give you a specific key to use and others may require you to hand over all accounts, passwords, and keys at the end of the project.

How does SSH manage multiple keys?

Once you have more than a couple of SSH keys, you discover that you need to add IdentifyFile lines to your ~/.ssh/config to get SSH to offer the right key to the right server. This is because most servers will disconnect after only a few SSH key attempts. I have seven keys, so I definitely need this.

For example, this entry specifies that when I use machines in the Fedora Project, my user name is "bex" and I want my Fedora SSH key to be used. The referenced file is the private key file.

Host *
    User bex
    IdentityFile ~/.ssh/fedora_id_rsa

Without the IdentifyFile keyword, SSH would just try each key serially.

Teach SSH to manage multiple GPG authentication keys

When you use GPG to authenticate SSH, you no longer have private key files. Therefore, the configuration above won't work anymore. In order to get this working, you have to do something different to identify the key that should be used. Thankfully, SSH provides a workaround: You can reference the public key on the IdentifyFile line instead.

Just modify your ~/.ssh/config to reference the proper public key. Typically, this is as easy as adding pub to the end of each IdentifyFile line. Don't forget to make sure that the public key file is set as user readable only.

This means you will need to have public key files available for your authentication keys. If you didn't save them (or have them before), you can get the fingerprints from the ssh-add -L command.

Identify specific authentication subkeys

Once you have created more than a few authentication subkeys, you'll want to be able to identify them so that you can give the right public keys to the right servers and make the proper IdentifyFile entries. Unfortunately, GPG doesn't support comments or names for subkeys. Therefore, you can only get a list of key hashes and grips from the gpg2 command and an unmatched set of private and public fingerprints from ssh-add.

There are two ways to identify your keys. You can start from the SSH public key fingerprint and find the GPG keygrip, or you can start from the GPG subkey hash and find the SSH public key fingerprint.

SSH public key fingerprint to GPG keygrip

Start by finding the public key fingerprint you want to identify. You can do this by running ssh-add -L. This example works with the first fingerprint. Save the public key fingerprint to a file, as the next step requires this.

$ ssh-add -L | head -1 > key

Now you need the MD5 format of the SSH fingerprint, which you can get from ssh-keygen.

$ ssh-keygen -l -E md5 -f key
2048 MD5:9e:98:82:87:5d:a4:fe:e3:8a:9c:db:aa:59:5b:30:ac (none) (RSA)

Match the MD5 SSH fingerprint with the keys, as displayed by the keyinfo command of the gpg-agent.

$ gpg-connect-agent 'KEYINFO --list --ssh-fpr' /bye | fgrep
S KEYINFO 9EEA76057C168686EAE8B807845326D7F60FB1C4 D - - - P
MD5:9e:98:82:87:5d:a4:fe:e3:8a:9c:db:aa:59:5b:30:ac - -

This returns multiple fields of data. The important one is the third element, which is the GPG keygrip. From here, it is easy to identify your specific authentication subkey using gpg2 -K --with-keygrip.

GPG subkey hash to SSH public key fingerprint

Start by finding the subkey hash for the key you want to identify. You can do this by running gpg2 -K --with-subkey-fingerprint.

$ gpg2 -K --with-subkey-fingerprint
ssb   rsa2048 2019-03-16 [A]

Converting this into a public key fingerprint is really easy; just use gpg2 again.

$ gpg2 --export-ssh-key 941BE28372F6759AB7073766E0A70B46BA68E808
ssh-rsa AAAAB3NzaC1yc...MQJ3FK3 openpgp:0xBA68E808

You'll notice that there is a comment, openpgp:0xBA68E808, in this output. This is not guaranteed in all cases, as it depends on exactly how you are using GPG. If your output includes it, it's an easy way to identify the GPG key and SSH fingerprint relationship; however you shouldn't rely on it always being available.

I encourage you to consider documenting the relationships in your .gnupg/sshcontrol file or as comments in your public key files.

Wrapping up

Now you're ready to get access to every server in the world using a GPG-authenticated SSH session. You can manage multiple keys and, when needed, identify them. You've reduced your secure backup needs to a single key file, which you can use to unlock or decrypt everything else.

What to read next
User profile image.
Brian "bex" Exelbierd is the RHEL Community Business Owner and works to inform the RHEL roadmap with community efforts and to support Operating System communities. At Red Hat, Brian has worked as a technical writer, software engineer, content strategist, community architect and now as a product manager.


You can have them anywhere you want, but their permission and the permission of the parent folders need to be strict:

no writable access for the parent folder (for others and all)
644 for a public key
600 for a private key.
You then:

declare those different keys in a ~/.ssh/config file (example here)
change the remote url in order to use the appropriate entry of the ~/.ssh/config file which described the right ssh key to use.

change the remote url in order to use the appropriate entry of the ~/.ssh/config file which described the right ssh key to use.

In reply to by Gabriella

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.