Most server breaches happen because someone skipped the basics. I've seen production boxes compromised within hours of deployment because root SSH was left open with password auth. The good news? You can lock down a fresh VPS in a single afternoon with seven systematic steps.
This isn't theory. These are the exact configurations I walk clients through when they hand me a newly provisioned server and ask me to make it production-ready.
1. Harden SSH Before Anything Else
SSH is your front door. Secure it first.
Start by creating a non-root user with sudo privileges. Never run daily operations as root—it's too easy to make a catastrophic typo.
adduser deployuser
usermod -aG sudo deployuser
Switch to key-based authentication immediately. Generate an SSH key pair on your local machine if you haven't already:
ssh-keygen -t ed25519 -C "[email protected]"
Copy the public key to your server:
ssh-copy-id deployuser@your_server_ip
Now edit /etc/ssh/sshd_config and make these changes:
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
Port 2849
Changing the default port from 22 to something random cuts automated bot attacks by roughly 90% in my experience. Pick any unprivileged port above 1024 and below 65535. Write it down—you'll need it.
Restart SSH carefully. Do NOT close your current session until you've tested the new config in a second terminal:
sudo systemctl restart sshd
Open a new terminal and verify you can connect with your key on the new port:
ssh -p 2849 deployuser@your_server_ip
Only after that works should you close the old session.
2. Configure a Firewall and Default-Deny
A firewall is your second line of defense. Default-deny means you explicitly allow only the ports you need and block everything else.
On Ubuntu/Debian, use ufw. On CentOS/RHEL, use firewalld. I'll show ufw here because it's simpler:
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 2849/tcp # your SSH port
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
Check the status:
sudo ufw status verbose
You should see your three allowed ports and a default deny policy. If you're running a mail server, database, or custom app, open only those specific ports. Don't leave 3306 (MySQL) or 5432 (PostgreSQL) open to the world—bind them to localhost or use a private network.
For firewalld on RHEL-based systems:
sudo firewall-cmd --set-default-zone=drop
sudo firewall-cmd --permanent --add-port=2849/tcp
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload
3. Automate Security Updates
Patching is boring until you get breached. Automate it.
On Debian/Ubuntu, install unattended-upgrades:
sudo apt install unattended-upgrades
sudo dpkg-reconfigure --priority=low unattended-upgrades
Edit /etc/apt/apt.conf.d/50unattended-upgrades to enable automatic security updates. The defaults are sensible—security patches only, no major version jumps.
On RHEL/CentOS, use dnf-automatic:
sudo dnf install dnf-automatic
sudo systemctl enable --now dnf-automatic.timer
Edit /etc/dnf/automatic.conf and set apply_updates = yes under the [commands] section.
I still check servers weekly, but automated patching closes the window between disclosure and exploitation from days to hours.
4. Lock Down User Access and Sudo
Every additional user is a potential attack vector. Create accounts only when necessary and disable or delete them when people leave.
When you do need to grant sudo, be specific. Don't give blanket ALL=(ALL:ALL) ALL. Instead, use targeted rules in /etc/sudoers.d/:
sudo visudo -f /etc/sudoers.d/deployuser
Add a line like:
deployuser ALL=(ALL) NOPASSWD: /usr/bin/systemctl restart nginx, /usr/bin/systemctl reload php*-fpm
This lets deployuser restart nginx and PHP-FPM without a password prompt, but nothing else. For developers who need occasional root access, require a password and log it:
developer ALL=(ALL) ALL
Always use visudo or visudo -f to edit sudoers files—it checks syntax and won't let you save a broken config that locks you out.
5. Install and Configure Fail2Ban
Fail2Ban watches log files for repeated authentication failures and temporarily bans the offending IP. It's like an automated bouncer.
Install it:
sudo apt install fail2ban # Debian/Ubuntu
sudo dnf install fail2ban # RHEL/CentOS
Create a local config file at /etc/fail2ban/jail.local:
[DEFAULT]
bantime = 1h
findtime = 10m
maxretry = 5
[sshd]
enabled = true
port = 2849
logpath = /var/log/auth.log
This bans an IP for one hour if it fails five SSH login attempts in ten minutes. Adjust the port to match your SSH config.
Start and enable Fail2Ban:
sudo systemctl enable --now fail2ban
Check active jails and banned IPs:
sudo fail2ban-client status
sudo fail2ban-client status sshd
I've seen Fail2Ban block thousands of brute-force attempts per day on publicly facing servers. It won't stop a determined attacker, but it raises the bar.
6. Disable Unnecessary Services
Every running service is a potential entry point. List what's listening:
sudo ss -tulnp
You'll see a table of open ports and the processes behind them. If you spot something you don't recognize—rpcbind, cups, or legacy daemons—disable it:
sudo systemctl stop <service_name>
sudo systemctl disable <service_name>
On a typical web server, you should see SSH, HTTP/HTTPS, and maybe a database bound to localhost. Anything else deserves scrutiny.
Also check for running xinetd or inetd services:
sudo systemctl list-units --type=service --state=running | grep inet
Disable them if they're present. Modern systems rarely need inet daemons.
7. Enable Logging and Monitor Regularly
Logs are your security camera footage. Configure centralized logging or at minimum ensure rsyslog or journald is running and retaining logs for at least 30 days.
Check disk space for /var/log:
df -h /var/log
If it's tight, configure log rotation in /etc/logrotate.d/. Most distros ship sane defaults, but verify auth logs are being kept:
sudo ls -lh /var/log/auth.log* # Debian/Ubuntu
sudo ls -lh /var/log/secure* # RHEL/CentOS
You should see at least a few rotated files. If not, check /etc/logrotate.conf and the drop-in files.
For real-time alerts, set up a simple cron job that emails you if someone uses sudo:
sudo nano /etc/sudoers
Add this line under Defaults:
Defaults mail_always
Defaults mailto="[email protected]"
You'll get an email every time sudo is invoked. It's noisy if you have automated scripts, but invaluable on a small server where sudo should be rare.
For larger setups, consider shipping logs to a separate log server or a service like Papertrail or Loggly.
How do I test if my SSH config is actually secure?
SSH into your server as root (from a second session, just in case). If it rejects you, your PermitRootLogin no is working. Try connecting with a password; it should fail if PasswordAuthentication no is set. Run sudo sshd -T to dump the effective config and verify your changes.
What if I lock myself out while hardening SSH?
This is why you test in a second terminal before closing the original session. If you do lock yourself out, most VPS providers offer a web-based console or recovery mode. Boot into single-user mode, mount the filesystem, and revert /etc/ssh/sshd_config. Always keep a second session open during SSH changes.
Should I install an antivirus on Linux?
Generally no. Linux malware exists, but traditional AV is overkill for a server. Focus on hardening, patching, and monitoring. If you're hosting email or file shares that serve Windows clients, consider ClamAV to scan for Windows malware, but it's not a substitute for proper system security.
How often should I review firewall rules?
Every time you deploy a new service. Also audit quarterly. I've found forgotten test ports left open months after a project ended. A quick sudo ufw status or sudo firewall-cmd --list-all takes 10 seconds.
What's the one thing most people skip?
Changing the SSH port and disabling password auth. I'd estimate half the servers I audit still allow password login on port 22. Fix that and you're ahead of most small operations.
Start with SSH and firewall today
You don't need to implement all seven steps at once. If you do nothing else, harden SSH and enable a default-deny firewall this afternoon. That alone will stop the majority of opportunistic attacks.
The rest—Fail2Ban, automated patching, service audits—can follow over the next week. Security isn't a one-time checkbox. But these seven steps give you a defensible baseline that takes a few hours and makes your server a much harder target.