SSH Server Hardening: Step-by-Step Guide Linux 2026

Skip the small talk. If your SSH server still lets root log in with a password on port 22, a bot will find you — it’s just a question of when, not if. Here’s the exact routine I run on every Linux box before it ever touches production: full ssh server hardening, from key-based auth to fail2ban. No fluff, just commands you can copy and verify.

I’ve cleaned up more compromised boxes than I care to count, and the root cause is almost always the same. It’s never the clever zero-day or the sophisticated malware. It’s SSH left wide open — default port, root login enabled, password auth on, no rate limiting. The bots find it within hours, and once a credential slips, it’s game over for everything behind that door.

The damage isn’t limited to the box itself. An attacker with SSH access can move laterally: read your database dumps, steal API keys, tamper with configs, plant backdoors, mine crypto, or use your server as a relay for bigger attacks. And on a production environment, a compromise like that means downtime, data leaks, and clients asking very uncomfortable questions. I’ve seen it turn into legal headaches more than once.

So how do you know if you’re already being scanned? Check your auth log. On most distros that’s /var/log/auth.log (Debian/Ubuntu) or /var/log/secure (RHEL). You’ll usually see a tell-tale pattern: a wall of “Failed password for invalid user” lines from a stream of different IPs. That’s bots working through their dictionary lists. If you see that, they’ve already found you — you just aren’t locked yet.

# Debian/Ubuntu
sudo tail -n 200 /var/log/auth.log

# RHEL/CentOS
sudo tail -n 200 /var/log/secure

# Or via journald
sudo journalctl -u sshd --since "1 hour ago" | grep -i failed | head -50

The root causes are remarkably consistent, which is honestly kind of sad. Default port 22 still open everywhere, PermitRootLogin still set to yes, password authentication still enabled when key-based auth would do, no firewall rules limiting who can reach the port, no fail2ban to eject repeat offenders, and a pile of stale users with old keys that were never cleaned up. Fix those six things and you’ve closed the door on 99% of the attacks I see daily. Here’s exactly how.

One rule before we start: make sure you have out-of-band access to the server — a KVM console from your provider, a hypervisor console if it’s a VM, anything. Because some of these steps can lock you out if done wrong. Never, ever run this sequence with SSH as your only way in.

Why SSH Server Hardening Matters Before You Go Live

SSH is the front door to your Linux server. Terminal access, file uploads, config management, deploys — it all flows through that single service. If ssh server hardening is skipped, you’re betting your whole stack on a login prompt being strong enough. And default settings aren’t strong enough; they’re designed to be convenient, not safe. This isn’t paranoia — it’s about making sure the door requires a real key and turns away people who don’t have one.

Think of it like a house: you don’t need to turn it into a bank vault, but you do need decent locks, and you don’t leave every window wide open and announce it on a public billboard. Port 22 with password auth is exactly that billboard.

Step 1: Back Up Your SSH Configuration

This is boring and I still do it every single time, because one bad edit on a remote box without console access is a very bad day.

sudo cp -a /etc/ssh /etc/ssh.bak.$(date +%F)
ls -la /etc/ssh.bak.*

Also copy the backup off the box — scp it to your workstation. A backup stored on the compromised box is not much of a backup.

Step 2: Set Up ed25519 Key-Based Authentication

Passwords are guessable, brute-forceable, phishable. Keys aren’t — as long as the private key stays with you and has a strong passphrase. I use ed25519 on every new setup: faster, shorter, and cryptographically stronger than RSA for the same level of security. Generate the keypair on your own machine, not on the server.

ssh-keygen -t ed25519 -a 100 -C "admin@my-workstation"

Then push the public key to the server. ssh-copy-id works while passwords are still enabled:

ssh-copy-id -i ~/.ssh/id_ed25519.pub admin@server-ip

No ssh-copy-id on your platform? Append the public key manually to ~/.ssh/authorized_keys on the server. Either way, test a fresh login with the key before moving on. This order matters — never disable password auth before you’ve confirmed the key actually works.

Step 3: Disable Root Login and Password Authentication

Edit the server’s sshd_config:

sudo nano /etc/ssh/sshd_config

Set these directives:

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
ChallengeResponseAuthentication no
KbdInteractiveAuthentication no
UsePAM yes

Validate the config first, then reload. This two-step habit saves you from locking yourself out:

sudo sshd -t
sudo systemctl reload ssh

On RHEL-family systems the service is sshd: sudo systemctl reload sshd. After the reload, open a second terminal and log in with your key. Only close your current session once the new one works. I cannot stress this enough — I’ve watched people lose the only session they had open mid-change.

Step 4: Move Off Port 22 and Lock It Down with a Firewall

Changing the port (say, to 22022) is not security by design — a determined attacker will scan and find it. What it does is kill the noise: nearly all automated bot traffic only ever targets port 22, so the signal-to-noise ratio of your logs improves dramatically.

Set the new port in sshd_config:

Port 22022

Reload, then fix the firewall. With UFW (Debian/Ubuntu):

sudo ufw allow 22022/tcp
sudo ufw delete allow 22/tcp
sudo ufw enable
sudo ufw status

On RHEL/CentOS with firewalld:

sudo firewall-cmd --permanent --add-port=22022/tcp
sudo firewall-cmd --permanent --remove-port=22/tcp
sudo firewall-cmd --reload
sudo firewall-cmd --list-ports

And don’t forget your client: ssh -p 22022 admin@server-ip, plus any port forwards or load balancer rules still pointing at the old port.

Step 5: Restrict Which Users Can SSH In

Don’t let every user on the box log in over SSH. Restrict it to a dedicated group. Create the group and add your admins:

sudo groupadd sshusers
sudo usermod -aG sshusers admin
sudo usermod -aG sshusers deploy

Then, in sshd_config:

AllowGroups sshusers
DenyUsers root

Anyone outside the group can’t authenticate over SSH, even with a valid password or key. It’s a cheap, effective second layer. If you’re managing several servers, this pairs well with a user management best practice workflow so stale accounts never pile up.

Step 6: Add fail2ban for Automatic Banning

fail2ban is the bouncer: it watches your auth logs, and when an IP fails too many times, it drops them at the firewall level automatically. I wrote a full fail2ban complete guide, but the essentials for SSH look like this:

sudo apt install fail2ban
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

In /etc/fail2ban/jail.local, make sure the sshd jail is enabled and points at your new port:

[sshd]
enabled = true
port = 22022
maxretry = 5
bantime = 3600

Start and check it:

sudo systemctl enable --now fail2ban
sudo fail2ban-client status sshd

You’ll watch the banned counter climb, and your auth logs turn quiet again. Pair it with a properly configured UFW firewall guide and port scanning becomes a waste of an attacker’s time.

Step 7: Harden Session Handling and Login Limits

Last stretch. These directives stop idle sessions from hanging around forever and cap how much guesswork a bot can do per connection:

MaxAuthTries 3
MaxStartups 10:30:60
LoginGraceTime 60
ClientAliveInterval 300
ClientAliveCountMax 0
X11Forwarding no
AllowTcpForwarding no

Quick rundown: MaxAuthTries 3 limits password or key attempts per connection; MaxStartups caps unauthenticated concurrent connections so attackers can’t tie up resources; LoginGraceTime 60 gives sixty seconds to complete login; ClientAliveInterval 300 plus ClientAliveCountMax 0 kills idle sessions after five minutes; X11Forwarding and AllowTcpForwarding off shut down rarely-used features that only add attack surface.

Troubleshooting: Common SSH Hardening Errors

Things will go wrong at least once — that’s normal. Here’s the table I keep pinned to my desk.

If you hit something outside that table, work the problem backwards: restore your last known-good config, run sudo sshd -t, and compare. When in doubt, roll back. And if you’re prepping a new box for a move, my KVM VPS migration guide covers locking down SSH as part of a full handover.

FAQ: SSH Hardening on Linux

While you’re at it, pair these changes with server log monitoring with Netdata and Grafana so you see auth spikes the moment they happen instead of finding out weeks later.

Before you close this tab, run the checklist: keys in place and verified, root login off, password auth off, port changed with firewall updated, users restricted, fail2ban active, session limits set. Tick each one and you’re done — that’s the whole game. Go lock your doors.