SSH Keys and config: From Generation to Managing Many Hosts

What key-based login actually solves

Password login has two hard problems: it can be brute-forced and it must be typed every time. Key-based login proves identity with asymmetric keys — the private key stays local, the public key goes on the server. As long as the private key is safe, no amount of password guessing gets in.

Generate and deploy

# Prefer ed25519: short, fast, strong
ssh-keygen -t ed25519 -C "you@example.com"

# Copy the public key (or append it to ~/.ssh/authorized_keys)
ssh-copy-id -i ~/.ssh/id_ed25519.pub deploy@host

# If the private key has a passphrase, cache it with an agent
eval "$(ssh-agent -s)" && ssh-add ~/.ssh/id_ed25519

~/.ssh/config makes many hosts manageable

Host prod
  HostName 203.0.113.10
  User deploy
  Port 2222
  IdentityFile ~/.ssh/id_ed25519
  ServerAliveInterval 30

Host *.internal
  User ops
  ProxyJump bastion
  IdentityFile ~/.ssh/id_ed25519

After that, ssh prod is all you need; * plus ProxyJump also expresses "reach internal hosts through the bastion" in one block.

Hardening on the server side

• Disable password login (after confirming keys work): PasswordAuthentication no;
• No direct root login: PermitRootLogin no;
• A non-default port is not a substitute for keys — it only reduces log noise;
• Restrict who may log in with AllowUsers.

Four common pitfalls

1. Loose private key permissions: must be 600, or SSH refuses to use it;
2. known_hosts conflicts: after a rebuild or IP reuse the fingerprint changes — verify it, then remove the old line, never blindly clear the file;
3. Agent forwarding abuse: ForwardAgent lets root on the jump host borrow your key to move laterally — leave it off unless required;
4. One key everywhere: separate key pairs per purpose so a leak has a smaller blast radius.

Common questions

Why passphrase-protect the private key? It is one more barrier if the device is lost, and with an agent it costs you nothing day to day. Can keys be brute-forced? Not ed25519 or a long RSA key with today's compute — the real risk is the key file leaking. Connections keep dropping? Set ServerAliveInterval on the client and a sensible ClientAliveInterval on the server.

Choosing and generating keys

Use Ed25519 for all new keys, falling back to RSA 4096 only for old devices. Add a comment so keys can be cleaned up later:

• ssh-keygen -t ed25519 -C "you@example.com", always with a passphrase on the private key;
• Private keys need mode 600 and their directory 700, or OpenSSH refuses to load them;
• One key pair per device so it can be revoked individually — never share one private key across machines.

Practical ~/.ssh/config

Putting jump hosts, ports, users and key paths in config removes a lot of repeated flags:

• ServerAliveInterval 30 under Host * stops long idle sessions being dropped;
• ProxyJump bastion reaches internal hosts without a manual hop;
• Set IdentityFile and IdentitiesOnly yes explicitly so too many public keys are not offered before one is accepted.

Common misconceptions

• Agent forwarding: after hopping to an untrusted host it can borrow your agent to reach other machines — prefer ProxyJump and keep forwarding off;
• Copying private keys around: generate a new key pair and distribute the public key instead;
• Disabling password auth without testing keys: verify key login works before setting PasswordAuthentication no, or you may lock yourself out;
• Ignoring known_hosts changes: a changed host key may be a rebuild or a MITM — verify before clearing it.

Server hardening checklist

No direct root login, no password authentication, an allow-list of users, failure rate limiting, and sshd logs shipped to central storage so brute-force attempts are visible.

Key rotation and revocation

• One key pair per device: when a device is lost or someone leaves, remove that single line from authorized_keys without affecting anyone else;
• The comment is your inventory: put the owner and device in -C, or nobody will dare delete a line six months later;
• Rotate periodically: unchanged private keys accumulate risk — rotate yearly with a short overlap window;
• Certificates beat bare keys: at modest scale, SSH certificates issued by a CA carry expiry and therefore revoke themselves.

Common connection failures

• Permissions 0644 ... are too open: private key too permissive — set 600;
• Too many authentication failures: too many local keys offered — add IdentitiesOnly yes and an explicit IdentityFile;
• Host key verification failed: the remote host key changed — verify why before updating known_hosts.

Organising multi-environment config

Split ~/.ssh/config per environment and pull the pieces in with Include so one file does not grow indefinitely. Prefix production aliases clearly (for example prod-) and keep strict host key checking on — naming discipline reduces connecting to the wrong box.