The morning a client calls and says "our entire production environment is encrypted" is when your backup strategy either saves the business or proves worthless. I've walked too many hosting customers through ransom demands because their backups lived on the same network the attackers owned.
A real ransomware backup recovery plan separates teams that restore in hours from those who spend days negotiating with criminals or rebuilding from scratch.
Why most backup strategies fail during ransomware attacks
Ransomware operators know you have backups. They hunt for them before triggering encryption. Modern attacks dwell in networks for weeks, mapping backup repositories, escalating privileges, and poisoning recovery points. The typical signs I've seen: backup jobs still run green in monitoring dashboards while attackers quietly delete snapshots or encrypt the backup target itself.
Cloud storage buckets configured with full delete permissions? Compromised. NAS devices mounted read-write across the domain? Encrypted alongside production. Backup retention policies that auto-delete old versions? Attackers wait until your good snapshots age out.
The attacks are patient. Your recovery plan needs to assume the attacker already has access to anything reachable from your production network.
Step 1: Implement true offline backup storage
Offline means physically or logically disconnected from systems an attacker can reach. Not a mapped drive. Not a share mounted 24/7.
Tape cartridges ejected and stored off-site still work. Many hosting teams dismissed tape as legacy tech, but you can't encrypt a cartridge sitting in a safe. LTO-9 drives handle 18 TB native capacity and modern tape automation libraries slot/unslot media without manual intervention for weekly or monthly fulls.
Air-gapped storage servers work too: a backup target that powers on via scheduled script, accepts the replication job, then powers down. Some shops use IPMI or BMC interfaces to wake the backup host, run the job over a dedicated VLAN, then hard-power-off when done. That box stays dark 23 hours a day.
Physically disconnected external USB or eSATA drives remain simple for small environments. Rotate two or three drives: one on-site attached during backup windows, one off-site in a safe deposit box or secondary location, one in transit. The drive that isn't plugged in can't be touched.
Object storage with carefully locked-down permissions serves as another layer. Write credentials stay in a secrets vault only the backup service can access, and those credentials have zero delete permissions. A separate break-glass admin account, stored offline and protected by hardware MFA, holds delete rights for emergencies only.
Step 2: Enable immutable snapshots and append-only modes
Immutability means backups can be created but not modified or deleted for a fixed period, even by administrator accounts. Compliance locks and object versioning enforce this at the storage layer.
S3 Object Lock, Azure Blob immutable storage, and Google Cloud Bucket Lock all offer WORM (write once, read many) guarantees. Configure governance or compliance mode retention periods that exceed your recovery time objectives by a comfortable margin—30 days minimum, 90 days better.
For on-premises targets, ZFS holds snapshots you can mark read-only at the filesystem level, and many enterprise backup appliances include hardened operating modes that prevent snapshot deletion even from root. Veeam calls it immutability with hardened repositories; Commvault has ransomware protection mode; Rubrik defaults everything immutable.
Append-only repositories let backup software write new data but never overwrite or remove existing blocks. Configure your backup software to authenticate to the repository with credentials that literally lack delete API permissions. An attacker who steals those creds can create garbage backups but can't destroy the good ones already written.
Test it. Try to delete a snapshot with admin credentials during the lock period. The attempt should fail hard.
Step 3: Build recovery runbooks and test them quarterly
A plan you've never executed is just a Word document. Real recovery happens when someone half-asleep at 2 AM can follow the runbook and actually get services running.
Your incident playbook needs four sections:
Detection and containment checklist: How do you confirm it's ransomware and not a hardware failure? Which network segments do you isolate first? Who calls law enforcement, legal, and insurance? This section stops the bleeding before you start recovery.
Restoration priority matrix: Which systems come back first? Usually domain controllers from a known-good snapshot, then DNS, then database primaries, then application servers. Each entry needs the snapshot identifier format, storage location, and estimated restore time. A spreadsheet works—just keep it printed and in the on-call binder because you might not have access to Google Sheets.
Step-by-step restore procedures: Exactly how do you pull a backup from offline storage and mount it? What boot media do you use for bare-metal restores? Which config files need manual edits (IP addresses, hostnames) after restore? I write these as if I'm explaining it to someone who's competent but unfamiliar with our specific environment.
Validation gates: How do you verify each restored system before moving to the next? Check logs for suspicious persistence mechanisms left by the attacker. Compare file hashes. Confirm services actually answer requests. Don't bring a compromised domain controller back just because it boots.
Quarterly tests mean picking a subset of production systems and actually restoring them to isolated VMs or a separate environment. Full DR exercises once a year; partial component tests every quarter. Track restore times. The tested restore time is the only honest RTO you have.
What if your backups are also compromised?
Sometimes the attacker got there first and corrupted your snapshots weeks ago. You won't know until you try to restore and find encrypted files inside the backup itself.
File integrity monitoring helps catch this early. Tools that hash critical files and compare against known-good baselines will flag tampering inside backup archives before you're in a crisis. Run validation jobs that actually restore a few random files from each backup generation and scan them—don't just trust that the backup software reported success.
Keep multiple generations with different retention schedules. Daily incrementals for two weeks, weekly fulls for three months, monthly archives for a year. Attackers often trigger encryption before they've poisoned every generation. The older the snapshot, the more likely it's clean.
Version control for critical config files (SSH keys, database configs, web server vhosts) in a separate Git repository gives you another recovery path. That repo should live in a different infrastructure silo—different cloud account, different credentials—so compromising production doesn't hand over your config backups too.
Step 4: Segment backup infrastructure from production networks
Backup servers shouldn't sit on the same VLAN as the workloads they protect. Dedicated backup networks with firewall rules that permit only the specific protocols needed (typically TCP 10000-10005 for backup agents, or NFS/SMB shares restricted by IP) reduce the attack surface.
One-way data flows work even better: production hosts push backups to a target they can write to but not read from or delete. The backup repository never initiates connections back into production. If an attacker owns a production server, they can send garbage to the backup target, but they can't pull existing backups to study them or delete them.
Separate authentication domains help too. Backup infrastructure joined to its own directory service, not the production Active Directory that ransomware typically compromises first. Service accounts that run backup jobs should have minimal privileges—read access to the data being backed up, write access to the backup target, and nothing else. No domain admin. No sudoers.
Jumpboxes or bastion hosts provide the only management access to backup infrastructure, and those bastions require hardware MFA. I've seen cases where attackers pivoted from a web server to the backup network because both management interfaces were accessible from the same admin workstation. Don't let that be you.
Step 5: Establish a backup monitoring and alert system
Silent backup failures are how attackers buy time. Your monitoring needs to scream when expected jobs don't complete, when snapshots disappear, when disk usage on the backup target drops suddenly (mass deletion), or when restore tests fail.
Key metrics to track:
- Successful completion of each scheduled job, with alerts on failure or skip
- Growth rate of backup repository (sudden flatline suggests incremental jobs are failing to capture changes)
- Snapshot count per retention policy (rapid decrease indicates deletion)
- Test restore success rate from automated validation jobs
- Login attempts and privilege escalations on backup infrastructure
Forward backup logs to a separate SIEM or log aggregation system the attacker can't easily tamper with—not the same syslog server that collects production app logs. Cloud-based log platforms or a physically separate logging appliance work.
Set up independent monitoring from outside your network. A simple cron job that curls a healthcheck endpoint and pages you if it fails can catch an outage faster than internal monitoring tools the attacker might have disabled.
Step 6: Practice recovery under time pressure
Your best test happens when you force yourself to restore a critical system in under two hours with no preparation beyond grabbing the runbook. Announce a surprise DR drill, shut down a development database, and make the on-call engineer restore it from backup. Time it. Document what went wrong.
Common problems surface immediately: Backup media is in a storage unit two hours away. The decryption key for the backup repository is in a password manager nobody can access because the SSO provider is down. The runbook says "restore from Tuesday's snapshot" but doesn't explain how to identify Tuesday's snapshot in the object storage bucket.
After each drill, update the runbook with the fixes. The goal is a process so mechanical that even a junior admin can execute it at 3 AM during an adrenaline spike.
Rotate who runs the drill. If only your senior engineer knows how to work the tape library, you have a bus-factor problem. Cross-train the team until three people can restore production from backups without help.
Build restore time into your SLAs and communicate it honestly. If your actual tested restore time for the database tier is six hours, don't promise customers a four-hour RTO. Under-promise, over-deliver.
Coordinating with incident response and forensics
Restoration isn't just technical. Legal, insurance, and forensics teams often need access to compromised systems before you wipe them. Your runbook should define who makes the call to start restoration versus preserving evidence.
Image compromised systems before restoring over them when possible. A bit-for-bit disk copy to separate forensic storage preserves evidence and lets investigators figure out how the attacker got in. That intelligence prevents reinfection after you restore.
Bring in your cyber insurance carrier early. Many policies require notification within hours and offer access to forensics and negotiation teams. Some even cover the cost of restoration if you can prove you followed documented security practices, which means you need that tested backup plan documented before the incident.
Coordinate restore timing with the security team's remediation work. Bringing systems back online while attackers still hold credentials or persistence mechanisms just gives them a fresh environment to encrypt. Confirm all known attacker access paths are closed—revoked credentials, patched vulnerabilities, rebuilt compromised jump hosts—before you restore production.
Building the right backup stack for ransomware resilience
Your backup tooling matters less than how you configure and operate it. That said, some features directly support ransomware defense:
Incremental forever with synthetic fulls avoids the backup window problems that tempt teams to skip full backups, which means you always have a recent complete snapshot to restore from.
Built-in malware scanning inside the backup software can flag encrypted or suspicious files before they hit the repository. Not a substitute for endpoint protection, but a useful last-ditch check.
Instant VM recovery or mount features let you run workloads directly from the backup storage while permanent restoration happens in the background, cutting RTO dramatically for virtualized environments.
Integrated replication to secondary sites multiplies your recovery options. Local backup appliance for fast restores, plus replicated copies at a colo or cloud region an attacker can't reach from the primary environment.
Open-source tools like Restic, Borg, or Duplicati handle encryption, deduplication, and multiple backends well enough for many self-hosted environments. Pair them with scripts that enforce your offline and immutability requirements.
What happens when you can't restore everything
Sometimes the backups are too old, too incomplete, or too compromised. You'll rebuild some systems from scratch. Your runbook should include a priority list: which services get restored from backup vs. redeployed fresh, and which data you can afford to lose.
Application servers often rebuild faster than restoring if you have infrastructure-as-code and deployment automation. Spin up new instances, deploy from your Git main branch, and move on. Stateful components—databases, file storage, email spools—are where backups matter most.
Document acceptable data loss windows for each system. Email can lose a few hours. Financial transaction databases can't lose any. Those tolerances dictate backup frequency and whether you need continuous replication in addition to periodic snapshots.
Customer communication becomes critical when restoration takes longer than expected. A status page updated every hour with realistic estimates beats silence. Customers tolerate downtime they understand; they abandon services that go dark without explanation.
Start with your most critical system
You don't need to implement all six steps across your entire infrastructure this week. Pick your most business-critical system—the database or service that losing would shut down revenue—and build a complete ransomware recovery plan for just that one component.
Document the restore procedure, configure immutable snapshots, test a full restore to a separate environment, and time it. Once that system's plan is solid, replicate the pattern to the next priority tier.
Incremental improvements ship. Perfect plans don't. Start small, test often, and expand coverage as you validate each piece. When the call comes at 2 AM, you'll have a plan that actually works instead of a document that sounded good in a planning meeting.