SSH is the gateway to your VPS. Get it wrong and you've left the front door unlocked. Yet even experienced sysadmins make preventable mistakes when hardening SSH—mistakes that range from inconvenient lockouts to complete server compromises. This guide walks through the most common SSH hardening errors and shows you the correct approach for each.
Mistake 1: Only Changing the SSH Port and Calling It Secure
Changing the default SSH port from 22 to something like 2222 or 22022 is often the first and only hardening step people take. While obscurity reduces automated scan noise, it provides no real security against a determined attacker.
Why This Is a Problem
Port scanning is trivial. Any attacker who targets your server specifically will find your SSH port in seconds. Relying solely on a non-standard port creates false confidence while leaving authentication and access controls weak.
The Correct Approach
- Change the port as one layer among many, not as your primary defense
- Focus first on disabling password authentication and enforcing key-based auth
- Implement fail2ban or similar intrusion prevention
- Use firewall rules to restrict SSH access by IP when possible
Edit /etc/ssh/sshd_config:
Port 2849
PasswordAuthentication no
PubkeyAuthentication yes
PermitRootLogin prohibit-password
Then restart SSH:
systemctl restart sshd
Port changes belong in a layered security strategy, not standing alone.
Mistake 2: Locking Yourself Out Before Testing
The classic mistake: you edit sshd_config, restart the SSH service, disconnect, and suddenly cannot reconnect. The changes contained a syntax error, disabled your authentication method, or blocked your IP.
Why This Is a Problem
Without an active session or console access, you're locked out of your own server. Recovery requires your hosting provider's emergency console, VNC access, or a support ticket—none of which are ideal.
The Correct Approach
Always maintain two SSH sessions when making changes:
- Open your first SSH session
- Make configuration changes in that session
- Test the configuration:
sshd -t - Reload (not restart) SSH:
systemctl reload sshd - Open a second SSH session to verify you can still connect
- Only after successful connection in the second session should you close the first
If the new connection fails, you still have your original session to revert changes.
For critical changes, use systemctl reload instead of restart when possible—reload preserves existing connections while applying new settings to future connections.
Mistake 3: Weak SSH Key Generation and Management
Many guides tell you to generate SSH keys, but few emphasize key strength and proper management. Common mistakes include using default RSA key sizes, storing keys insecurely, or reusing the same key across multiple servers.
Why This Is a Problem
Weak keys can be cracked. Reused keys mean one compromise affects multiple servers. Keys without passphrases offer no protection if your local machine is compromised.
The Correct Approach
Generate strong keys with proper algorithms:
# Ed25519 (preferred for modern systems)
ssh-keygen -t ed25519 -C "[email protected]" -f ~/.ssh/id_ed25519_production
# Or RSA with 4096 bits if Ed25519 isn't supported
ssh-keygen -t rsa -b 4096 -C "[email protected]" -f ~/.ssh/id_rsa_production
Always protect keys with strong passphrases. Use different keys for different servers or contexts (production, staging, personal). Store private keys with restrictive permissions:
chmod 600 ~/.ssh/id_ed25519_production
chmod 644 ~/.ssh/id_ed25519_production.pub
On the server side, ensure proper permissions:
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
Never commit private keys to version control or store them in cloud sync folders without encryption.
Mistake 4: Leaving Root Login Enabled
Permitting direct root SSH login is one of the most dangerous configurations you can leave in place. Many distributions still ship with PermitRootLogin yes by default.
Why This Is a Problem
Root is a known username on every Linux system. Attackers only need to guess or crack the password or key. If compromised, they have immediate full system access with no audit trail of which user account was responsible.
The Correct Approach
Disable root login entirely and use sudo for privilege escalation:
# In /etc/ssh/sshd_config
PermitRootLogin no
Create a non-root user with sudo privileges:
adduser youruser
usermod -aG sudo youruser
Copy your SSH key to the new user's authorized_keys, test login as that user, then disable root login. This creates an audit trail (you know which user account authenticated) and adds a second authentication step via sudo.
If you absolutely must allow root login during initial setup, use PermitRootLogin prohibit-password to require key-based auth, then disable it entirely once your user account is configured.
Mistake 5: Forgetting to Configure the Firewall
Hardening SSH configuration is pointless if your firewall allows SSH connections from anywhere. Many admins configure sshd_config carefully but leave ufw or firewalld wide open or disabled entirely.
Why This Is a Problem
Without firewall rules, your SSH port is exposed to the entire internet. Every bot, scanner, and attacker can attempt connections. Even with strong authentication, you're wasting resources and increasing attack surface.
The Correct Approach
Restrict SSH access at the firewall level. If you have a static IP or known IP ranges:
# UFW example
ufw default deny incoming
ufw allow from YOUR_IP_ADDRESS to any port 2849 proto tcp
ufw enable
# firewalld example
firewall-cmd --permanent --remove-service=ssh
firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="YOUR_IP_ADDRESS" port port="2849" protocol="tcp" accept'
firewall-cmd --reload
If you need SSH from dynamic IPs, implement fail2ban:
apt install fail2ban
Create /etc/fail2ban/jail.local:
[sshd]
enabled = true
port = 2849
maxretry = 3
bantime = 3600
findtime = 600
Restart fail2ban:
systemctl restart fail2ban
Combine IP whitelisting, fail2ban, and rate limiting for defense in depth.
Mistake 6: Enabling Password Authentication as a Backup
Some admins disable password auth for root but leave it enabled for other users "just in case" they lose access to their keys. This defeats the purpose of key-based authentication.
Why This Is a Problem
Password authentication is vulnerable to brute force attacks, credential stuffing, and password reuse. If any user account has a weak password, your entire server is at risk. The "just in case" scenario almost never happens with proper key backup procedures.
The Correct Approach
Disable password authentication completely:
# In /etc/ssh/sshd_config
PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no
Instead of keeping passwords as a backup:
- Store your private key in multiple secure locations (encrypted backups, password manager)
- Maintain access to your hosting provider's emergency console (VNC, Serial Console)
- Document your recovery procedure
- Use SSH key forwarding when connecting through bastion hosts
If you must maintain password access during a transition period, set a hard deadline to disable it and use strong, unique passwords generated by a password manager.
Mistake 7: Ignoring SSH Configuration File Permissions
SSH is strict about file permissions for good reason. Incorrect permissions on configuration files or key files can cause SSH to silently ignore them or refuse connections, leading to confusion and potential security issues.
Why This Is a Problem
If your authorized_keys file is world-writable, anyone with local access could add their own key. If your private key is readable by others, it's not private. SSH will often refuse to use improperly secured files.
The Correct Approach
Verify and correct permissions on all SSH-related files:
# Server side
chown -R youruser:youruser /home/youruser/.ssh
chmod 700 /home/youruser/.ssh
chmod 600 /home/youruser/.ssh/authorized_keys
chmod 644 /etc/ssh/sshd_config
chmod 600 /etc/ssh/ssh_host_*_key
chmod 644 /etc/ssh/ssh_host_*_key.pub
# Client side
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_*
chmod 644 ~/.ssh/id_*.pub
chmod 644 ~/.ssh/config
chmod 644 ~/.ssh/known_hosts
When troubleshooting SSH issues, check permissions first. The error messages SSH provides about permission problems can be cryptic.
Mistake 8: Not Limiting User Access with AllowUsers or AllowGroups
By default, SSH permits all system users to attempt login. On a multi-user system or a server with service accounts, this expands your attack surface unnecessarily.
Why This Is a Problem
Every user account is a potential entry point. Service accounts, legacy users, or accounts created by applications may have weak security. You rarely need more than one or two users to have SSH access.
The Correct Approach
Explicitly whitelist SSH users in sshd_config:
AllowUsers youruser admin
Or use groups for better management:
AllowGroups sshusers
Then add permitted users to that group:
groupadd sshusers
usermod -aG sshusers youruser
This approach ensures that even if an attacker compromises a service account, they cannot use it for SSH access.
Mistake 9: Skipping SSH Audit and Log Monitoring
Many admins harden SSH but never check whether their hardening is effective. Failed login attempts, unusual connection patterns, and successful authentications all generate logs that usually go unexamined.
Why This Is a Problem
Without monitoring, you won't know if you're under attack until after a breach. You can't measure the effectiveness of your hardening efforts or spot configuration drift over time.
The Correct Approach
Regularly review SSH logs:
# Recent SSH activity
journalctl -u sshd -n 100
# Failed login attempts
grep "Failed password" /var/log/auth.log
# Successful logins
grep "Accepted publickey" /var/log/auth.log
Implement centralized logging if you manage multiple servers. Set up alerts for suspicious patterns:
- Multiple failed attempts from the same IP
- Successful logins from unexpected geographic locations
- SSH access outside business hours (for production servers)
Periodically audit your SSH configuration:
sshd -T | grep -E 'permitrootlogin|passwordauthentication|pubkeyauthentication|port'
Verify that your actual running configuration matches your intended security policy.
Mistake 10: Using Outdated SSH Protocol Settings
Legacy SSH servers may still accept old protocol versions, weak ciphers, or deprecated key exchange algorithms for backward compatibility. These create unnecessary vulnerabilities.
Why This Is a Problem
Older protocol versions and weak cryptographic algorithms have known vulnerabilities. Supporting them increases attack surface even if you never intentionally use them.
The Correct Approach
Explicitly configure modern, strong cryptographic settings in sshd_config:
# Use only SSH protocol 2
Protocol 2
# Strong ciphers
Ciphers [email protected],[email protected],[email protected],aes256-ctr,aes192-ctr,aes128-ctr
# Strong MACs
MACs [email protected],[email protected],hmac-sha2-512,hmac-sha2-256
# Strong key exchange
KexAlgorithms curve25519-sha256,[email protected],diffie-hellman-group-exchange-sha256
Before applying these settings, verify that your SSH clients support them. Modern systems handle these without issue, but very old clients may require updates.
Conclusion
SSH hardening is not a checklist you complete once and forget. It's a layered approach combining strong authentication, restrictive access controls, proper monitoring, and regular audits. The mistakes outlined here share a common thread: they trade real security for convenience or rely on single points of failure.
Avoid the false confidence that comes from half-measures like changing the port alone or leaving password auth enabled as a backup. Build defense in depth by combining key-based authentication, firewall restrictions, privilege separation, and active monitoring. Test your changes carefully to avoid lockouts, verify your configuration regularly, and keep your SSH software updated. Your VPS security depends on getting these fundamentals right.