Most break-ins happen because servers ship with defaults designed for ease, not security. Port 22 open to the world, root login allowed, no rate limiting on authentication attempts. I've seen root passwords brute-forced in under two hours on a fresh VPS.
You can close those gaps in an afternoon with eight hardening steps that don't require deep expertise. Copy the configs, restart the services, test once. The payoff is a server that resists automated attacks and gives you time to respond when something unusual happens.
1. Lock down SSH immediately
SSH is the front door. Attackers scan for it constantly.
Disable root login and password authentication entirely. Use SSH keys only. Open /etc/ssh/sshd_config and change or add these lines:
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
ChallengeResponseAuthentication no
UsePAM no
Move SSH off port 22 if you want to dodge the noisiest bots:
Port 2849
Pick any unused port above 1024. Update your firewall rules to match before you restart SSH or you'll lock yourself out.
Restart with systemctl restart sshd and test your key-based login in a second terminal before closing your current session. That's the step people skip, then panic.
Add MaxAuthTries 3 and LoginGraceTime 30 to give attackers fewer attempts and less time. Three tries, thirty seconds, then disconnect.
2. Install and configure Fail2Ban
Fail2Ban watches your logs for repeated failed logins and blocks the source IP automatically. It's a second layer behind SSH hardening.
Install it:
sudo apt install fail2ban # Debian/Ubuntu
sudo yum install fail2ban # RHEL/CentOS
Create /etc/fail2ban/jail.local with these settings:
[DEFAULT]
bantime = 1h
findtime = 10m
maxretry = 3
[sshd]
enabled = true
port = 2849
logpath = /var/log/auth.log
Adjust the port to match your SSH config. bantime = 1h blocks the attacker for one hour after three failures in ten minutes. Scale that up to 24h or permanent if you see persistent attacks.
Start it with systemctl enable --now fail2ban and check status with fail2ban-client status sshd. You'll see banned IPs accumulate in the output.
3. Set up automatic security updates
Patching is the part everyone knows matters and still forgets. Automate it.
On Debian and Ubuntu, install unattended-upgrades:
sudo apt install unattended-upgrades
sudo dpkg-reconfigure --priority=low unattended-upgrades
Edit /etc/apt/apt.conf.d/50unattended-upgrades and uncomment the security line:
Unattended-Upgrade::Allowed-Origins {
"${distro_id}:${distro_codename}-security";
};
Add this to get email reports on failures:
Unattended-Upgrade::Mail "[email protected]";
Unattended-Upgrade::MailReport "only-on-error";
On RHEL or CentOS, use dnf-automatic:
sudo yum install dnf-automatic
sudo systemctl enable --now dnf-automatic.timer
Edit /etc/dnf/automatic.conf and set apply_updates = yes under the [commands] section. Without that line it only downloads updates but doesn't install them.
Check logs weekly in /var/log/unattended-upgrades/ or /var/log/dnf.log to confirm patches are landing. If you see errors about held packages, investigate. A broken dependency can silently stop all updates.
4. Configure a stateful firewall with minimal open ports
Close everything except what you actually use.
I prefer ufw on Ubuntu because the syntax is straightforward:
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 2849/tcp comment 'SSH'
sudo ufw allow 80/tcp comment 'HTTP'
sudo ufw allow 443/tcp comment 'HTTPS'
sudo ufw enable
That's it. Three inbound ports, everything else blocked. If you need to open a database port or mail submission, add it explicitly and comment what it's for. In six months you won't remember.
For RHEL or CentOS, use firewalld:
sudo firewall-cmd --set-default-zone=drop
sudo firewall-cmd --zone=public --add-service=ssh --permanent
sudo firewall-cmd --zone=public --add-service=http --permanent
sudo firewall-cmd --zone=public --add-service=https --permanent
sudo firewall-cmd --reload
The drop zone discards packets silently instead of sending ICMP rejections. Marginally quieter in logs.
Check active rules with ufw status verbose or firewall-cmd --list-all. Run nmap from another machine to verify only your intended ports respond.
What if you need temporary access?
Sometimes you'll troubleshoot something that requires opening a port briefly. Don't leave it open.
With ufw, add a timed rule:
sudo ufw allow from 203.0.113.45 to any port 3306 proto tcp
That whitelists one IP. When you're done, delete it:
sudo ufw status numbered
sudo ufw delete [number]
With firewalld, use a rich rule:
sudo firewall-cmd --add-rich-rule='rule family="ipv4" source address="203.0.113.45" port port=3306 protocol=tcp accept' --timeout=3600
It expires automatically after one hour.
5. Enable disk encryption for data at rest
If someone physically pulls your drive or accesses the hypervisor, unencrypted data is readable immediately. Full disk encryption solves that.
Most cloud providers let you create encrypted volumes at provisioning time. Do it. The performance overhead is negligible on modern CPUs with AES-NI.
If you're encrypting an existing disk, back up first. Use LUKS:
sudo cryptsetup luksFormat /dev/sdb
sudo cryptsetup open /dev/sdb encrypted_data
sudo mkfs.ext4 /dev/mapper/encrypted_data
Mount it:
sudo mount /dev/mapper/encrypted_data /mnt/secure
To mount at boot, add a keyfile or passphrase prompt to /etc/crypttab. Store the keyfile with restricted permissions or use a TPM module if the hardware supports it.
For the root filesystem, you need to set up encryption during OS installation. Retrofitting root encryption is possible but painful. Easier to rebuild.
6. Disable unused services and remove unnecessary packages
Every running service is attack surface. Every installed package is a potential vulnerability.
List what's running:
systemctl list-units --type=service --state=running
See something you don't recognize? Check what it does:
systemctl status [service-name]
Disable anything you don't need:
sudo systemctl disable --now [service-name]
Common targets: avahi-daemon, cups, bluetooth, ModemManager. If it's a headless server, you probably don't need any of those.
Remove orphaned packages:
sudo apt autoremove
sudo apt autopurge
On RHEL:
sudo yum autoremove
Less installed software means fewer things to patch and fewer places for an attacker to hide.
7. Harden kernel parameters with sysctl
The Linux kernel has dozens of tunable security knobs. Most distributions leave them at permissive defaults.
Create /etc/sysctl.d/99-security.conf with these settings:
# Ignore ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
# Disable IP forwarding unless you're a router
net.ipv4.ip_forward = 0
net.ipv6.conf.all.forwarding = 0
# Ignore ICMP ping requests
net.ipv4.icmp_echo_ignore_all = 1
# Enable SYN cookies to resist SYN flood attacks
net.ipv4.tcp_syncookies = 1
# Disable source packet routing
net.ipv4.conf.all.accept_source_route = 0
# Log martian packets
net.ipv4.conf.all.log_martians = 1
Apply them:
sudo sysctl -p /etc/sysctl.d/99-security.conf
Verify with sysctl net.ipv4.ip_forward or any other key. Should return the value you set.
Disabling ping can make troubleshooting harder, so consider whether that tradeoff suits your environment. I keep it on for internal servers and off for anything public-facing.
8. Set up centralized logging and alerting
You can't respond to what you don't see.
Forward logs to a remote server or SIEM so an attacker can't erase their tracks by deleting local files. rsyslog does this natively.
Edit /etc/rsyslog.conf and add:
*.* @logserver.example.com:514
That uses UDP. For reliability, use TCP:
*.* @@logserver.example.com:514
Restart rsyslog:
sudo systemctl restart rsyslog
On the log server, configure rsyslog to listen:
module(load="imtcp")
input(type="imtcp" port="514")
Store logs in /var/log/remote/ and rotate them aggressively. Disk fills fast.
For alerting, set up simple email notifications with logwatch or integrate with tools like Graylog or Elastic. I've used a cron job that greps for specific patterns in auth logs and sends a message to Slack. Low-tech but effective:
#!/bin/bash
grep 'Failed password' /var/log/auth.log | tail -20 | curl -X POST -d @- https://hooks.slack.com/...
Run that hourly. Adjust the pattern to match what matters in your environment.
Keep testing your defenses
Hardening isn't a one-time checklist. Test every few months.
Run a port scan from outside your network:
nmap -sS -p- your-server-ip
Only your intended services should respond. Everything else filtered or closed.
Check SSH with:
ssh -v -o PasswordAuthentication=yes root@your-server-ip
Should fail immediately. If it prompts for a password, your config didn't take effect.
Review installed packages quarterly and remove anything you added for testing and forgot about. Check which users have shell access with cat /etc/passwd | grep /bin/bash and delete stale accounts.
In support tickets I handled, the usual culprit was a service someone installed six months ago for a one-off task and never disabled.
How often should I update the kernel?
Apply kernel updates as soon as they're available if they fix security issues. Otherwise, monthly is fine. You'll need to reboot, so schedule it during low-traffic windows. Check your provider's maintenance calendar if you're on managed infrastructure.
Does disk encryption slow down the server?
Barely. Modern CPUs with AES-NI acceleration handle encryption in hardware. You might lose one or two percent of throughput. The protection against data theft is worth it.
Can I automate all eight steps with a script?
Yes, but test the script on a dev server first. One wrong firewall rule and you'll lock yourself out. I've done it. Ansible or shell scripts work fine, just include rollback logic and always keep a second SSH session open while testing.
What if Fail2Ban blocks my own IP?
Add your IP to the ignore list in /etc/fail2ban/jail.local:
[DEFAULT]
ignoreip = 127.0.0.1/8 ::1 203.0.113.45
Restart Fail2Ban. To unban immediately, run fail2ban-client set sshd unbanip 203.0.113.45.
Where to go from here
These eight steps handle the most common attack vectors: brute force, unpatched software, open ports, and physical access. They won't stop a determined attacker with zero-day exploits, but they'll block the automated scans that compromise most servers.
Start with SSH and the firewall today. Add the rest over the next week. Check your logs in a month to see how many attacks you're deflecting. The number will surprise you.