Every fresh server is a target the moment it touches the internet. I've seen boxes compromised within hours because someone skipped the basics—default SSH port still open, root login allowed, no firewall rules.
This checklist walks you through twelve concrete steps to harden a new Linux server. Most take minutes. A few require ongoing attention, but all are standard practice for any production machine.
1. Update the system immediately
First thing: pull down security patches.
apt update && apt upgrade -y # Debian/Ubuntu
dnf upgrade -y # RHEL/Rocky/AlmaLinux
Set up automatic security updates so you're not manually patching every week. On Debian-based systems install unattended-upgrades:
apt install unattended-upgrades -y
dpkg-reconfigure --priority=low unattended-upgrades
For RHEL-family boxes, dnf-automatic does the same job:
dnf install dnf-automatic -y
systemctl enable --now dnf-automatic.timer
Automatic updates won't catch every CVE instantly, but they close the window on known exploits without you remembering to log in.
2. Create a non-root user with sudo privileges
Never work as root day-to-day. Create a dedicated user, add it to the sudo group, and use that account for admin tasks.
adduser deploy
usermod -aG sudo deploy # Debian/Ubuntu
usermod -aG wheel deploy # RHEL/Rocky
Test the new account before you lock root out—open a second SSH session, log in as deploy, and confirm sudo whoami returns root.
3. Enforce SSH key authentication and disable password login
Passwords are guessable. Keys are not.
On your local machine, generate an SSH key if you don't have one:
ssh-keygen -t ed25519 -C "[email protected]"
Copy the public key to the server:
ssh-copy-id deploy@your_server_ip
Now edit /etc/ssh/sshd_config on the server:
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
ChallengeResponseAuthentication no
Restart SSH:
systemctl restart sshd
Before you close your current session, open a new terminal and confirm you can still log in with your key. If something broke, your original session is still alive so you can fix it.
4. Change the default SSH port
Port 22 gets hammered by bots. Moving SSH to a high port cuts noise in your logs and reduces attack surface.
Pick a port above 1024 that's not already in use—something like 2234 or 4422. Edit /etc/ssh/sshd_config:
Port 2234
If you're running a firewall (and you should be), open the new port before restarting SSH or you'll lock yourself out:
ufw allow 2234/tcp
systemctl restart sshd
From now on, connect with ssh -p 2234 deploy@your_server_ip.
5. Configure a firewall and deny by default
A firewall blocks everything except the services you explicitly allow. On Ubuntu ufw is simplest; on RHEL-family systems use firewalld.
Using ufw (Debian/Ubuntu)
ufw default deny incoming
ufw default allow outgoing
ufw allow 2234/tcp # your SSH port
ufw allow 80/tcp
ufw allow 443/tcp
ufw enable
Check the rules:
ufw status verbose
Using firewalld (RHEL/Rocky/AlmaLinux)
firewall-cmd --set-default-zone=public
firewall-cmd --permanent --add-service=http
firewall-cmd --permanent --add-service=https
firewall-cmd --permanent --add-port=2234/tcp
firewall-cmd --reload
List active rules:
firewall-cmd --list-all
Only open the ports your services actually need. If you're not running a mail server, SMTP ports stay closed.
6. Disable unused services
Every running service is an attack vector. List what's enabled:
systemctl list-unit-files --type=service --state=enabled
Stop and disable anything you don't recognize or need. Common candidates:
systemctl disable --now avahi-daemon
systemctl disable --now cups
systemctl disable --now bluetooth
Be careful with this step—don't disable sshd or networking unless you enjoy console access.
7. Install and configure fail2ban
Fail2ban watches your logs for repeated login failures and temporarily bans the offending IP.
apt install fail2ban -y # Debian/Ubuntu
dnf install fail2ban -y # RHEL/Rocky
Create a local config at /etc/fail2ban/jail.local:
[DEFAULT]
bantime = 3600
findtime = 600
maxretry = 5
[sshd]
enabled = true
port = 2234
logpath = /var/log/auth.log # or /var/log/secure on RHEL
Start the service:
systemctl enable --now fail2ban
Check banned IPs:
fail2ban-client status sshd
In support tickets I handled, fail2ban caught brute-force attempts within minutes of going live.
8. Set up log monitoring and retention
Logs tell you what happened when something breaks or gets breached. Make sure they're being written and kept long enough to investigate.
Check that rsyslog or systemd-journald is running:
systemctl status rsyslog
Configure log rotation in /etc/logrotate.d/ so logs don't fill your disk. A typical /etc/logrotate.d/syslog looks like:
/var/log/syslog
{
rotate 7
daily
missingok
notifempty
compress
delaycompress
}
For centralized logging, consider shipping logs to an external service or a dedicated log server. That way an attacker can't just delete /var/log to cover their tracks.
9. Enable SELinux or AppArmor
Mandatory access control systems add another layer by restricting what processes can do, even if they're compromised.
SELinux ships with RHEL-family distributions. Check its status:
getenforce
If it says Permissive or Disabled, set it to Enforcing in /etc/selinux/config:
SELINUX=enforcing
Reboot for the change to take effect. SELinux can break applications if policies aren't configured, so test thoroughly.
AppArmor is the default on Ubuntu. Verify it's active:
aa-status
Most packages ship with AppArmor profiles. If you compile something custom, you'll need to write a profile or switch it to complain mode during testing.
10. Disable unused network protocols and services
If your server doesn't need IPv6, turn it off. Same for services like NFS or RPC that you're not using.
Disable IPv6 by adding to /etc/sysctl.conf:
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
Apply the change:
sysctl -p
Mask unused services so they can't be started accidentally:
systemctl mask rpcbind
systemctl mask nfs-server
Fewer listening sockets means fewer ways in.
11. Harden kernel parameters
A few sysctl tweaks make the kernel less helpful to attackers. Add these to /etc/sysctl.conf:
# Ignore ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
# Ignore source-routed packets
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
# Enable SYN cookies for SYN flood protection
net.ipv4.tcp_syncookies = 1
# Log suspicious packets
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1
# Disable IP forwarding if this is not a router
net.ipv4.ip_forward = 0
Reload:
sysctl -p
These won't stop a determined attacker but they close off some easy reconnaissance and DoS tricks.
12. Schedule regular security audits
Hardening isn't a one-time task. Set reminders to review user accounts, open ports, and installed packages every quarter.
Run a quick port scan from outside your network:
nmap -sS -O your_server_ip
Check for packages with known vulnerabilities. On Debian/Ubuntu:
apt list --upgradable
On RHEL/Rocky:
dnf updateinfo list security
Review sudo logs in /var/log/auth.log or /var/log/secure for unexpected privilege escalations. Look for unfamiliar user accounts in /etc/passwd.
Consider running an automated scanner like Lynis:
apt install lynis -y
lynis audit system
It'll flag misconfigurations and suggest additional hardening steps specific to your setup.
What order should you follow?
Start with steps 1-3: updates, a non-root user, and SSH keys. Those three stop the most common attacks.
Next set up your firewall (step 5) and change the SSH port (step 4). Once the network perimeter is locked down, work through the remaining steps in any order.
If you're managing multiple servers, script the process. Ansible, Puppet, or even a shell script will save you from repeating these steps manually every time you spin up a new box.
FAQ
Do I need all twelve steps on every server?
Yes, unless you have a specific reason to skip one. Defense in depth means layering protections—if one fails, the others still hold.
Will changing the SSH port really help?
It won't stop a targeted attack, but it drastically cuts down log noise and automated scanning. Your auth logs will thank you.
Can I automate security updates on a production server?
For security patches, yes. Configure unattended-upgrades or dnf-automatic to install security updates only, not all package updates. Always test major updates in staging first.
What if SELinux breaks my application?
Run in permissive mode temporarily and check /var/log/audit/audit.log for denials. Use audit2allow to generate a policy module that allows the required actions. Never just disable SELinux permanently.
How often should I review firewall rules?
Every time you add a new service, and at least quarterly otherwise. Ports you opened for testing six months ago might still be exposed.
Start with the network perimeter
If you only have ten minutes, do this: update the system, add SSH keys, disable password authentication, and turn on the firewall. Those four steps block the vast majority of opportunistic attacks.
The rest of the checklist adds depth. Fail2ban catches brute-force attempts. Kernel hardening and mandatory access controls limit what an attacker can do if they get in. Regular audits catch configuration drift before it becomes a breach.
None of these steps are complicated. Most are copy-paste. But skipping them is how servers end up mining cryptocurrency for someone else or serving phishing pages by Tuesday.