A critical CVE drops and suddenly you're racing to patch dozens of servers before an exploit hits the wild. The pressure to move fast runs headlong into the reality that a bad patch can break production harder than the vulnerability itself. I've seen teams skip testing, apply patches blindly, and spend the next twelve hours rolling back broken services.
The workflow below lets you close critical vulnerabilities quickly while keeping production stable. It's the process I use when a high-severity CVE lands in my queue.
Step 1: Confirm the vulnerability affects your systems
Not every CVE announcement means you're vulnerable. Check three things before you start patching.
First, verify the affected software versions are actually running on your infrastructure. Use package managers to list installed versions:
# Debian/Ubuntu
dpkg -l | grep package-name
apt-cache policy package-name
# RHEL/CentOS/Rocky
rpm -qa | grep package-name
yum info package-name
Second, check whether the vulnerable code path is reachable. A critical OpenSSH vulnerability doesn't matter if SSH is firewalled off from the internet. A PHP library flaw is irrelevant if you're running Python apps.
Third, look for compensating controls already in place. WAF rules, network segmentation, and disabled services all reduce exposure. Document these so you can justify patch priority to management if needed.
Skip patches that don't apply to your environment. Focus energy on real risks.
Step 2: Assess criticality and set a patching deadline
Severity scores alone don't tell you how fast to move. A critical-rated vulnerability in a non-internet-facing database needs a different timeline than a remote code execution bug in your public web server.
Rank by these factors:
- Exploitability: Is there working exploit code in the wild? Check security mailing lists and exploit databases.
- Asset exposure: Internet-facing systems get patched first, then internal infrastructure, then isolated test boxes.
- Data sensitivity: Servers handling customer data or payment processing jump the queue.
- Business impact: Core revenue services take priority over internal tools.
For truly critical, remotely exploitable vulnerabilities with public exploits, your deadline is hours, not days. For high-severity issues without known exploits, aim for a 48-72 hour window. Everything else can follow your normal patch cycle.
In support tickets I handled, the usual mistake was treating all "critical" CVEs identically and burning out teams with constant emergency patching.
Step 3: Read the patch release notes and changelogs
Patches sometimes introduce breaking changes or new bugs. Ten minutes reading release notes saves hours of troubleshooting.
Look for:
- Configuration file format changes
- Deprecated features or removed options
- New dependencies or library version requirements
- Known issues or recommended workarounds
- Required service restarts or system reboots
Check the vendor's security advisory, not just the CVE description. Vendors often include specific upgrade instructions or compatibility warnings.
If the patch is for a kernel or core system library, expect a reboot. Plan accordingly.
Step 4: Test the patch in a non-production environment
Never apply critical patches directly to production, even under time pressure. A broken patch that takes down your infrastructure is worse than the vulnerability.
Your test environment should mirror production as closely as possible: same OS version, same software stack, same configuration files. If you don't have a proper staging environment, spin up a minimal test VM with the same package versions.
Apply the patch and verify:
- The package installs without errors
- Services restart cleanly
- Core functionality works (test authentication, database connections, web requests)
- Logs show no new errors or warnings
- Resource usage (CPU, memory, disk) stays normal
Leave the patched test system running for at least an hour. Some issues only appear under sustained load.
If you're patching a web server, run a quick load test. If it's a database, run your query test suite. Real traffic patterns expose problems that manual testing misses.
So what if testing reveals a problem?
Document the failure, check for vendor errata or updated patches, and escalate to the vendor if needed. Sometimes you'll need to choose between a known vulnerability and a breaking patch. That's a business decision, not a technical one.
In those cases, apply compensating controls while you wait for a fixed patch: tighten firewall rules, enable additional logging, deploy WAF rules, or temporarily disable the vulnerable feature if the business can tolerate it.
Step 5: Schedule a maintenance window and notify stakeholders
Even if you're moving fast, tell people before you patch production. A two-sentence Slack message is better than surprised users filing outage tickets.
Your notice should include:
- Which systems are being patched
- Expected start time and duration
- Whether the service will be unavailable or just slower
- Rollback plan if things break
For after-hours emergency patching, notify your on-call team and get at least one other engineer on standby. You want backup if the patch goes sideways.
Schedule patches during low-traffic periods when possible, but don't delay a critical fix by 72 hours just to hit a perfect maintenance window. If the CVE is actively exploited, patch now.
Step 6: Apply the patch with a rollback plan ready
Before you run any update commands, take a snapshot or backup of the system state. On physical servers, document the current package versions. On VMs, take a snapshot. On cloud instances, create an AMI or image.
# Snapshot current package state
dpkg --get-selections > /root/packages-before-patch-$(date +%F).txt
rpm -qa > /root/packages-before-patch-$(date +%F).txt
Apply the patch using your distribution's package manager:
# Debian/Ubuntu
apt-get update
apt-get install --only-upgrade package-name
# RHEL/CentOS/Rocky
yum update package-name
After installation, restart affected services and verify they're running:
systemctl restart service-name
systemctl status service-name
journalctl -u service-name -n 50
Test the same functionality you tested in staging: web requests, database queries, authentication, whatever is critical to your application.
Monitor logs and metrics for at least 15 minutes after the patch. Watch for:
- Increased error rates
- Slower response times
- Failed health checks
- Memory leaks or CPU spikes
- Failed cron jobs or background tasks
If you see problems, roll back immediately. Revert the package to the previous version:
# Debian/Ubuntu
apt-get install package-name=previous-version
# RHEL/CentOS/Rocky
yum downgrade package-name
Restoring from a VM snapshot is faster if you have that option.
Step 7: Document the patch and verify across all systems
Once the first server is stable, patch the rest of your infrastructure using the same process. Don't assume one successful patch means the others will go smoothly—each environment has unique configurations.
Track your progress in a spreadsheet or ticket system:
- Server hostname/IP
- Patch applied (yes/no)
- Timestamp
- Applied by whom
- Any issues encountered
Run a post-patch audit to confirm every affected system was updated:
# Check patched version across multiple servers
for host in server1 server2 server3; do
echo "$host:"
ssh $host 'rpm -q package-name'
done
Document any systems that couldn't be patched (legacy software dependencies, vendor support contracts, etc.) and what compensating controls you put in place. Your auditors will ask.
Update your runbooks with lessons learned. If you hit an undocumented issue, add it so the next person doesn't waste time.
What about automated patch management?
Automation speeds up routine patching but critical CVEs still need human oversight. I've seen automated tools apply kernel updates that required manual intervention to boot, and security patches that broke application dependencies.
Use automation for the inventory and deployment steps—let tools identify vulnerable systems and push patches to test environments. Keep the testing and production deployment steps manual until you've built confidence in your rollback procedures.
Configuration management tools like Ansible, Puppet, or Salt are excellent for consistent patch deployment once you've validated the process. Start with automated inventory scanning and gradually expand to deployment as your process matures.
How do I know if a CVE is truly critical?
Check whether exploit code is publicly available and whether the vulnerable service is exposed to untrusted networks. Remote code execution or privilege escalation vulnerabilities on internet-facing systems are your highest priority. Local vulnerabilities or denial-of-service issues on internal systems can usually wait for the next maintenance cycle.
Should I patch development servers as urgently as production?
Patch dev and staging environments first—they're your test beds. But if a dev server has internet-facing services or handles real customer data, treat it like production. Attackers pivot through compromised development systems to reach production networks.
What if the vendor hasn't released a patch yet?
Apply workarounds from the security advisory: disable vulnerable features, restrict network access, enable additional authentication. Monitor vendor channels for the patch release and apply it the same day it drops. Zero-day vulnerabilities are exactly when your compensating controls earn their keep.
How long should I wait before patching after a vendor releases the fix?
For critical, remotely exploitable vulnerabilities, test and patch within 24-48 hours. Waiting a week to "let others test it first" gives attackers a week to compromise your systems. The testing you do in your staging environment is your safety net.
Check your patch inventory monthly
Critical CVE patching is crisis management. Better to never reach that point. Run monthly vulnerability scans, subscribe to security advisories for every piece of software in your stack, and keep a current inventory of what's running where.
When the next critical CVE drops, you'll know exactly which systems need attention and you'll have a tested workflow to patch them fast without breaking things. That's the difference between scrambling in a panic and executing a plan.