Most compromised servers I've seen in support tickets shared the same pattern: default SSH ports, password auth still enabled, and firewalls that allowed everything. The fixes are straightforward. You don't need a dedicated security team or expensive tooling—just a solid checklist and the discipline to work through it.
This guide covers eight practical hardening tasks you can knock out in an afternoon: SSH key rotation, firewall lockdown, automatic kernel updates, intrusion detection with fail2ban, disabling unused services, file integrity monitoring, log centralization basics, and a quick audit framework. Each section includes the exact commands and config snippets I use on production VPS instances.
1. Rotate SSH keys and kill password auth
Password authentication is the easiest vector. Brute-force bots hammer SSH twenty-four hours a day.
Generate a new ED25519 key pair on your local machine:
ssh-keygen -t ed25519 -C "[email protected]" -f ~/.ssh/id_ed25519_new
Copy the public key to your server:
ssh-copy-id -i ~/.ssh/id_ed25519_new.pub user@your-server-ip
Test the new key in a separate terminal session before you lock yourself out:
ssh -i ~/.ssh/id_ed25519_new user@your-server-ip
Once you confirm it works, edit /etc/ssh/sshd_config:
PasswordAuthentication no
ChallengeResponseAuthentication no
PermitRootLogin no
PubkeyAuthentication yes
Restart SSH:
sudo systemctl restart sshd
Rotate your keys every six to twelve months. I keep a calendar reminder. If a laptop gets stolen or a contractor leaves, you revoke one key instead of changing passwords across thirty servers.
2. Lock down your firewall with iptables or firewalld
Default-allow firewalls are an invitation. Start with default-deny and open only what you need.
For iptables, flush existing rules and set the policy:
sudo iptables -F
sudo iptables -X
sudo iptables -P INPUT DROP
sudo iptables -P FORWARD DROP
sudo iptables -P OUTPUT ACCEPT
Allow loopback and established connections:
sudo iptables -A INPUT -i lo -j ACCEPT
sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
Open SSH (change 22 if you moved the port):
sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT
If you're running a web server, open HTTP and HTTPS:
sudo iptables -A INPUT -p tcp --dport 80 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 443 -j ACCEPT
Drop everything else and log it:
sudo iptables -A INPUT -j LOG --log-prefix "iptables-dropped: "
sudo iptables -A INPUT -j DROP
Make the rules persistent. On Debian/Ubuntu:
sudo apt install iptables-persistent
sudo netfilter-persistent save
On RHEL/CentOS, use firewalld instead—it's the default and easier to manage:
sudo firewall-cmd --set-default-zone=drop
sudo firewall-cmd --zone=drop --add-service=ssh --permanent
sudo firewall-cmd --zone=drop --add-service=http --permanent
sudo firewall-cmd --zone=drop --add-service=https --permanent
sudo firewall-cmd --reload
Review your open ports quarterly. I've found database ports exposed to the world because someone tested a config and forgot to remove it.
3. Automate kernel and package updates
Unpatched kernels are how most worms spread. Automate it.
On Ubuntu/Debian, install unattended-upgrades:
sudo apt install unattended-upgrades
sudo dpkg-reconfigure --priority=low unattended-upgrades
Edit /etc/apt/apt.conf.d/50unattended-upgrades and ensure security updates are enabled:
Unattended-Upgrade::Allowed-Origins {
"${distro_id}:${distro_codename}-security";
};
On RHEL/CentOS, use dnf-automatic:
sudo dnf install dnf-automatic
sudo systemctl enable --now dnf-automatic.timer
Edit /etc/dnf/automatic.conf and set:
apply_updates = yes
Kernel updates require a reboot. Schedule maintenance windows or use live-patching services like Ubuntu Livepatch or KernelCare if you can't afford downtime. Most VPS providers send you an email before forced maintenance; read them.
4. Deploy fail2ban for intrusion detection
fail2ban watches your logs and bans IPs after repeated failed attempts. It's lightweight and catches the automated garbage.
Install it:
sudo apt install fail2ban # Debian/Ubuntu
sudo dnf install fail2ban # RHEL/CentOS
Copy the default config:
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
Edit /etc/fail2ban/jail.local and enable the SSH jail:
[sshd]
enabled = true
port = ssh
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600
If you run a web server, enable the nginx or apache jails too:
[nginx-http-auth]
enabled = true
[nginx-botsearch]
enabled = true
Start and enable the service:
sudo systemctl enable --now fail2ban
Check banned IPs:
sudo fail2ban-client status sshd
I've seen single servers block thousands of IPs per month. The noise level is real.
5. Disable unused services and close their ports
Every running service is a potential entry point. Audit what's listening.
List open ports:
sudo ss -tulnp
You'll see something like:
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=1234,fd=3))
LISTEN 0 128 0.0.0.0:80 0.0.0.0:* users:(("nginx",pid=5678,fd=6))
If you see ports you don't recognize, find the service:
sudo systemctl list-units --type=service --state=running
Disable anything you're not using:
sudo systemctl stop <service-name>
sudo systemctl disable <service-name>
Common candidates: cups (printing), avahi-daemon (Bonjour/mDNS), rpcbind (NFS), postfix (if you use an external mail relay). If you're not running a mail server, you probably don't need a local MTA.
What if you need a service temporarily?
Bind it to localhost instead of 0.0.0.0. For example, if you run MySQL only for a local app:
[mysqld]
bind-address = 127.0.0.1
Then firewall rules don't matter—nothing external can reach it.
6. Set up file integrity monitoring with AIDE
AIDE (Advanced Intrusion Detection Environment) takes a snapshot of your filesystem and alerts you to changes. It catches backdoors and modified binaries.
Install AIDE:
sudo apt install aide # Debian/Ubuntu
sudo dnf install aide # RHEL/CentOS
Initialize the database:
sudo aideinit
This takes a few minutes. Once done, copy the database:
sudo cp /var/lib/aide/aide.db.new /var/lib/aide/aide.db
Run a check:
sudo aide --check
Schedule daily checks with cron:
sudo crontab -e
Add:
0 3 * * * /usr/bin/aide --check | mail -s "AIDE Report" [email protected]
After legitimate system updates, reinitialize the database:
sudo aideinit
sudo cp /var/lib/aide/aide.db.new /var/lib/aide/aide.db
I've caught modified /bin/bash binaries this way. Not often, but once is enough.
7. Centralize logs and set retention policies
Logs scattered across /var/log are hard to search when you need them. Centralize them.
If you manage multiple servers, set up rsyslog forwarding to a central log host. On each server, edit /etc/rsyslog.conf and add:
*.* @@log-server.example.com:514
The @@ means TCP. For UDP, use a single @.
On a single server, at least rotate logs and keep them longer than the default:
sudo nano /etc/logrotate.d/syslog
Change rotate 4 to rotate 12 (twelve weeks). Change weekly to daily if disk space allows.
For web server logs, do the same:
sudo nano /etc/logrotate.d/nginx
I keep web logs for ninety days minimum. When a client asks "why did my site go down three weeks ago," you'll need them.
8. Run a monthly security audit checklist
Automation handles the routine stuff, but manual checks catch configuration drift.
Here's my monthly checklist:
- Review sudo access:
sudo cat /etc/sudoersand files in/etc/sudoers.d/ - Check for accounts with empty passwords:
sudo awk -F: '($2 == "") {print $1}' /etc/shadow - List users with shell access:
awk -F: '($7 == "/bin/bash" || $7 == "/bin/sh") {print $1}' /etc/passwd - Review cron jobs:
sudo crontab -land check/etc/cron.d/ - Scan for world-writable files:
sudo find / -xdev -type f -perm -0002 -ls 2>/dev/null - Check listening services:
sudo ss -tulnp - Review firewall rules:
sudo iptables -L -n -vorsudo firewall-cmd --list-all - Verify SSH config:
sudo sshd -T | grep -i passwordauth
Document what you find and fix drift immediately. Configuration creep is slow, then all at once.
What to check first
Start with SSH hardening and firewall rules. Those two alone stop the majority of automated attacks. Then add fail2ban, automate updates, and disable unused services. The rest—file integrity monitoring, log centralization, and monthly audits—layer on top.
You don't need to implement everything in one day. Pick two tasks this week. The threat landscape doesn't require perfection; it requires steady progress and fewer obvious gaps.