SSH MFA Guide: A Second Factor on Every Server Login

One stolen file shouldn't open your servers

Most servers let you in with one thing: an SSH key, or worse, a password. The key is a file. It gets copied to a new laptop, backed up to a cloud drive, left on the old laptop that went to a cousin, and it works from any computer in the world for as long as it sits in authorized_keys. Nobody gets an alert when it's used from somewhere new.

A second factor changes the arithmetic. The file alone stops being enough; whoever logs in also needs a physical key on the desk or a code from a phone in their pocket.

For the owner, in one line: after this, someone who copies an employee's laptop files still can't log in to your servers.

Pick a route

Pick one route per server. The hardware key is the stronger one, so it comes first.

Before you change anything: don't lock yourself out

This is the step that saves your afternoon.

1. Keep one SSH session open to the server the whole time, logged in as a user who can sudo. Changing the config doesn't drop sessions that are already connected, so this is your way back in.
2. Test from a second terminal, never from the session you're relying on.
3. Know where your hosting provider's web console is. It reaches the server without SSH, and it's the real safety net.
4. Check the config before every reload. This prints nothing when the file is valid and names the bad line when it isn't:

sudo sshd -t

Route A, step 1: Make a hardware-backed key

On your own computer, with the security key plugged in:

ssh -V
ssh-keygen -t ed25519-sk -O verify-required -C "alex-yubikey"

ssh -V should report OpenSSH 8.2 or later, the release that added these key types. The key asks you to touch it, and verify-required makes every login ask for the key's PIN as well. That's two factors in one gadget: something you have, something you know. If your key doesn't support Ed25519, use -t ecdsa-sk instead.

The file this creates in ~/.ssh is useless without the physical key, which is the whole point. Add -O resident to store the key handle on the security key itself, so you can load it on another computer with ssh-add -K. Resident keys usually need a PIN set on the security key first.

Register a second security key now. Run the same command with the backup key plugged in, and keep that key somewhere other than your laptop bag. A lost key with no backup is a trip to the provider's console.

Route A, step 2: Put the key on the server

ssh-copy-id -i ~/.ssh/id_ed25519_sk.pub alex@server

Do it for both security keys, then log in once with the new one to prove it works before you go any further.

Route A, step 3: Accept only hardware keys

On the server, create /etc/ssh/sshd_config.d/10-mfa.conf:

PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAcceptedAlgorithms sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com
PubkeyAuthOptions verify-required

The third line is the one people miss. PubkeyAuthOptions verify-required only applies to hardware-backed keys; the manual says it has no effect on any other key type. So an old ordinary key still sitting in authorized_keys would walk straight past it. Limiting the accepted algorithms to the two sk- types shuts that door.

The file name matters too. sshd keeps the first value it reads for each setting, and the files in sshd_config.d are read in order, so a low number like 10- beats a file from your hosting provider that sets the opposite.

sudo sshd -t && sudo systemctl try-reload-or-restart ssh

On distributions where the service is called sshd, use that name instead.

Route B: An authenticator app code

This follows Ubuntu's own server guide. Set up key login first: this route asks for a key and then a code, and users without a working key are locked out.

sudo apt update && sudo apt install libpam-google-authenticator

Each person who logs in then runs this as themselves, on the server:

google-authenticator

It shows a QR code to scan into an authenticator app and prints emergency scratch codes. Those codes are the recovery plan, so they go in your password manager, not in a text file on the server.

Create /etc/ssh/sshd_config.d/10-mfa.conf:

KbdInteractiveAuthentication yes
PasswordAuthentication no
AuthenticationMethods publickey,keyboard-interactive

On Ubuntu 20.04 and earlier, use ChallengeResponseAuthentication yes in place of the first line. AuthenticationMethods is what makes both factors required: the key, then the code.

In /etc/pam.d/sshd, replace the line @include common-auth with:

auth required pam_google_authenticator.so

Swapping the line, rather than adding beside it, is what keeps the server from asking for a password as well as the code. Then check and reload:

sudo sshd -t && sudo systemctl try-reload-or-restart ssh

Prove it worked

Ask sshd what it will actually enforce, after every drop-in file has had its say:

sudo sshd -T | grep -iE 'passwordauthentication|authenticationmethods|pubkeyacceptedalgorithms|pubkeyauthoptions'

The output line that matters is the one you set in 10-mfa.conf. If it shows something else, another file got there first. Then, from your second terminal, try to break in on purpose:

• Route A: log in with an old ordinary key, ssh -i ~/.ssh/id_ed25519 -o IdentitiesOnly=yes alex@server. It should be refused. Then log in with the security key and watch it ask for a touch and a PIN.
• Route B: log in and confirm you're asked for a verification code, and that a wrong code fails.

Only close the safety session once both checks pass.

Scripts and deploy tools

A deploy script can't touch a security key. Rather than switch the second factor off for everyone, give the automation its own user and its own rule at the bottom of sshd_config:

Match User deploy
    PubkeyAcceptedAlgorithms ssh-ed25519
    AuthenticationMethods publickey

That account still needs a key, just not a person. Keep what it can do as small as the job: an account that only deploys the website shouldn't have sudo.

When someone loses the key or the phone

• Lost security key: log in with the backup key, then delete the lost key's line from ~/.ssh/authorized_keys. The comment you set with -C tells you which line is which.
• Lost phone: log in with one of the emergency scratch codes, then run google-authenticator again to start over with the new phone.
• Everything lost: the provider's web console reaches the server without SSH. From there, remove 10-mfa.conf, reload, and set up again.
• Someone leaves: remove their keys from every server's authorized_keys the same day. A second factor makes a copied key harder to use; it doesn't remove the need to revoke it.

Sources

Checked on 2026-10-03:

• OpenSSH 8.2 release notes (FIDO key types)
• Ubuntu Server: two-factor authentication with TOTP or HOTP
• The OpenSSH 10.2 manual pages for sshd_config, sshd and ssh-keygen. Run man sshd_config on your server to check its version supports each setting.