Most hardening guides stop at installing fail2ban and calling it a day. That leaves the real attack surface untouched. I have spent years in support tickets, and the difference between a server that survives an audit and one that gets pwned usually comes down to a handful of advanced moves. This guide skips the basics and focuses on deep optimization, edge cases, and performance tuning for experienced admins.
1. Harden SSH Before You Expose It
SSH is the front door. If it is weak, everything else matters less. Start with key-based authentication only. Disable password logins, root login, and empty passwords. Then tune the cipher and KEX settings to match modern standards.
Edit /etc/ssh/sshd_config and apply this:
PasswordAuthentication no
PubkeyAuthentication yes
PermitRootLogin no
ChallengeResponseAuthentication no
UsePAM no
KexAlgorithms [email protected],diffie-hellman-group16-sha512
exchange_limit:1024
Ciphers [email protected],[email protected],[email protected]
MACs [email protected],[email protected]
Restart the service and test from a second terminal before locking yourself out. Verify with:
sshd -t
A common edge case: cloud providers sometimes require password login for initial access. Set up your key, test the connection, then disable passwords. If you lose the key, you will be stranded. Keep a console session open until you confirm the new settings work.
2. Restrict SSH Access with a Firewall
A firewall alone is not enough. You need to limit which source addresses can reach your SSH port. This stops automated scanners from probing every IP on the internet. Use iptables or nftables, depending on your distribution.
Example iptables rule:
iptables -A INPUT -p tcp --dport 22 -s 203.0.113.0/24 -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -j DROP
For nftables, the syntax is cleaner:
table inet ssh_filter {
chain input {
type filter hook input priority 0; policy drop;
iifname "eth0" tcp dport ssh ip saddr 203.0.113.0/24 accept
iifname "eth0" tcp dport ssh drop
}
}
If you use a cloud provider, their network ACLs add another layer. Check the provider console and align it with your local firewall. Do not rely on one layer alone.
3. Deploy Fail2ban with Custom Filters
Fail2ban blocks brute force attacks, but the default config is too loose for many setups. Tune the thresholds and create filters for your specific services. Check the logs regularly to see what is actually being blocked.
[sshd]
enabled = true
port = ssh
maxretry = 3
bantime = 3600
findtime = 600
For services like Nginx or Apache, write custom filters that match your access log format. A misconfigured filter can ban legitimate users or miss attackers entirely. Test filters with:
fail2ban-regex /var/log/nginx/access.log /etc/fail2ban/filter.d/nginx.conf
Watch the jail status:
fail2ban-client status
4. Harden the Kernel and System
The Linux kernel has many tunable parameters. Some controls are critical for security. Adjust settings in /etc/sysctl.conf or /etc/sysctl.d/99-security.conf.
net.ipv4.ip_forward = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.log_martians = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 4096
kernel.randomize_va_space = 2
Apply with:
sysctl -p /etc/sysctl.d/99-security.conf
Disable unused modules. If you do not need NFS, block it at the firewall. If you do not need Bluetooth, unload the module. Every service you remove is a surface you do not have to patch.
5. Harden File Permissions and Ownership
Permissions are a frequent source of privilege escalation. A world-writable file can allow an attacker to modify code or configuration. Run a regular audit.
find / -xdev -perm -0002 -type f -print 2>/dev/null
find / -xdev -perm -4000 -type f -print 2>/dev/null
Fix common issues with chmod and chown. Never run a web server as root. Apache and Nginx should run as dedicated users. PHP-FPM pools should map to a non-root user.
6. Harden Applications and Services
Every service you run is a potential entry point. Keep software updated, but do not stop there. Disable features you do not need. Remove modules that are not required. For example, disable CGI if you do not use it.
For Nginx, limit the number of worker connections and enable rate limiting:
http {
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
server {
location /api/ {
limit_req zone=api burst=20 nodelay;
}
}
}
For WordPress, disable XML-RPC if you do not use it. It is a common target for brute force and DDoS attacks. Restrict file edits in wp-config.php and use a security plugin that enforces least privilege.
7. Secure Email and DNS
Email deliverability depends on security. SPF, DKIM, and DMARC protect your domain from spoofing. A misconfigured DMARC policy can cause legitimate mail to be rejected. Start with a monitor policy and adjust after reviewing logs.
DNS security matters too. Use DNSSEC to prevent cache poisoning. Configure your resolver to ignore malformed queries. If you run a public DNS server, restrict recursion to trusted clients.
8. Monitor Logs and Set Up Alerts
You cannot fix what you do not see. Centralize logs on a separate host or use a managed logging service. Watch for failed logins, kernel warnings, and unusual outbound traffic.
tail -f /var/log/auth.log
journalctl -u sshd --since today
Set up alerts for repeated failed logins, new user accounts, and changes to cron jobs. An attacker who gains a shell will often try to hide their tracks. Monitor for new cron entries and suspicious binaries.
9. Back Up and Test Recovery
A hardened server is still a target. Backups are your last line of defense. Test restores regularly. A backup that has not been verified is just a file.
tar -czf /backup/server-$(date +%F).tar.gz /etc /var/www
Store backups off-site or on a separate volume. Encrypt them if they contain sensitive data. Automate the process, but verify the scripts run successfully.
Common Edge Cases
Full Disk
A full disk can break security. Log rotation stops, and services fail to write logs. An attacker can exploit this to avoid detection. Monitor disk usage and set up alerts before you hit 80 percent.
Misconfigured Firewall
A firewall that blocks everything can lock you out. Always test with a second connection. Keep a console session open until you confirm the rules work. Use a rule that allows your IP explicitly before applying the drop policy.
SSH Lockout
n If you lock yourself out of SSH, use the provider console. Do not try to guess the firewall rules from memory. A wrong iptables command can take down your entire session.
Expired SSL Certificates
SSL certificates expire. A server with a broken certificate may refuse connections or show warnings. Check expiration dates regularly and automate renewals. A failed renewal can take down a public-facing service.
What to Check First
Start with SSH, firewall rules, and kernel parameters. Then audit file permissions and service configurations. Monitor logs and back up regularly. These steps give you the most protection for the least effort. Review your setup monthly, and adjust as your infrastructure changes.