SSH Keys and config: From Generation to Managing Many Hosts

What key-based login actually solves

Password login has two hard problems: it can be brute-forced and it must be typed every time. Key-based login proves identity with asymmetric keys, the private key stays local, the public key goes on the server. As long as the private key is safe, no amount of password guessing gets in.

Generate and deploy

# Prefer ed25519: short, fast, strong
ssh-keygen -t ed25519 -C "you@example.com"

# Copy the public key (or append it to ~/.ssh/authorized_keys)
ssh-copy-id -i ~/.ssh/id_ed25519.pub deploy@host

# If the private key has a passphrase, cache it with an agent
eval "$(ssh-agent -s)" && ssh-add ~/.ssh/id_ed25519

~/.ssh/config makes many hosts manageable

Host prod
  HostName 203.0.113.10
  User deploy
  Port 2222
  IdentityFile ~/.ssh/id_ed25519
  ServerAliveInterval 30

Host *.internal
  User ops
  ProxyJump bastion
  IdentityFile ~/.ssh/id_ed25519

After that, ssh prod is all you need; * plus ProxyJump also expresses "reach internal hosts through the bastion" in one block.

Hardening on the server side

• Disable password login (after confirming keys work): PasswordAuthentication no;
• No direct root login: PermitRootLogin no;
• A non-default port is not a substitute for keys, it only reduces log noise;
• Restrict who may log in with AllowUsers.

Four common pitfalls

1. Loose private key permissions: must be 600, or SSH refuses to use it;
2. known_hosts conflicts: after a rebuild or IP reuse the fingerprint changes, verify it, then remove the old line, never blindly clear the file;
3. Agent forwarding abuse: ForwardAgent lets root on the jump host borrow your key to move laterally, leave it off unless required;
4. One key everywhere: separate key pairs per purpose so a leak has a smaller blast radius.

Common questions

Why passphrase-protect the private key? It is one more barrier if the device is lost, and with an agent it costs you nothing day to day. Can keys be brute-forced? Not ed25519 or a long RSA key with today's compute, the real risk is the key file leaking. Connections keep dropping? Set ServerAliveInterval on the client and a sensible ClientAliveInterval on the server.

Choosing and generating keys

Use Ed25519 for all new keys, falling back to RSA 4096 only for old devices. Add a comment so keys can be cleaned up later:

• ssh-keygen -t ed25519 -C "you@example.com", always with a passphrase on the private key;
• Private keys need mode 600 and their directory 700, or OpenSSH refuses to load them;
• One key pair per device so it can be revoked individually, never share one private key across machines.

Practical ~/.ssh/config

Putting jump hosts, ports, users and key paths in config removes a lot of repeated flags:

• ServerAliveInterval 30 under Host * stops long idle sessions being dropped;
• ProxyJump bastion reaches internal hosts without a manual hop;
• Set IdentityFile and IdentitiesOnly yes explicitly so too many public keys are not offered before one is accepted.

Common misconceptions

• Agent forwarding: after hopping to an untrusted host it can borrow your agent to reach other machines, prefer ProxyJump and keep forwarding off;
• Copying private keys around: generate a new key pair and distribute the public key instead;
• Disabling password auth without testing keys: verify key login works before setting PasswordAuthentication no, or you may lock yourself out;
• Ignoring known_hosts changes: a changed host key may be a rebuild or a MITM, verify before clearing it.

Server hardening checklist

No direct root login, no password authentication, an allow-list of users, failure rate limiting, and sshd logs shipped to central storage so brute-force attempts are visible.

Key rotation and revocation

• One key pair per device: when a device is lost or someone leaves, remove that single line from authorized_keys without affecting anyone else;
• The comment is your inventory: put the owner and device in -C, or nobody will dare delete a line six months later;
• Rotate periodically: unchanged private keys accumulate risk, rotate yearly with a short overlap window;
• Certificates beat bare keys: at modest scale, SSH certificates issued by a CA carry expiry and therefore revoke themselves.

Common connection failures

• Permissions 0644 ... are too open: private key too permissive, set 600;
• Too many authentication failures: too many local keys offered, add IdentitiesOnly yes and an explicit IdentityFile;
• Host key verification failed: the remote host key changed, verify why before updating known_hosts.

Organising multi-environment config

Split ~/.ssh/config per environment and pull the pieces in with Include so one file does not grow indefinitely. Prefix production aliases clearly (for example prod-) and keep strict host key checking on, naming discipline reduces connecting to the wrong box.