"We have backups" is not a recovery plan

Almost every team will tell you they have backups. Far fewer can tell you how recent the last restorable copy is, how long a full recovery would take, or when they last proved a restore actually works. That gap — between having backups and being able to recover — is where outages turn into business-ending events. A backup is an artifact; disaster recovery is a tested capability. Confusing the two is how organizations discover, mid-crisis, that their backups were silently failing for months.

This matters whether the trigger is a ransomware payload, a fat-fingered DROP TABLE, a corrupted disk, or a cloud region going dark. The cause varies; the question is always the same: how much data did we lose, and how long until we're running again?

RPO and RTO: the two numbers that define everything

Before you design anything, you need two targets, and they should be set by the business, not guessed by IT:

  • RPO (Recovery Point Objective) — how much data you can afford to lose, measured in time. An RPO of one hour means backups must capture state at least hourly; anything more recent than the last backup is gone. RPO drives backup frequency.
  • RTO (Recovery Time Objective) — how long you can afford to be down. An RTO of four hours means a full restore — provisioning, data, validation — must complete inside four hours. RTO drives backup architecture.

These numbers aren't aspirational; they're constraints that decide cost. A near-zero RPO and RTO means continuous replication and warm standby, which is expensive. A day of acceptable data loss and a day of acceptable downtime means cheap nightly snapshots. Most systems sit somewhere between, and different systems deserve different tiers — your customer database is not your internal wiki.

The 3-2-1 rule, and why immutability got added

The durable shorthand for backup resilience is 3-2-1: three copies of the data, on two different media types, with one copy off-site. It survives because it defends against independent failure modes at once — a deleted file, a failed disk, and a site-wide disaster won't all take out the same copy.

But ransomware rewrote the threat model. Modern attackers specifically hunt and encrypt or delete backups before they fire the payload, because they know a clean restore is what defeats them. So 3-2-1 has effectively become 3-2-1-1-0: one of those copies must be immutable or air-gapped — write-once storage or an offline copy the attacker's credentials cannot reach — and you want zero errors on a verified restore test. An online backup sitting in the same account with the same admin credentials as production is one stolen password away from being useless.

A backup you have never restored is a hope

This is the single most ignored discipline in disaster recovery: untested backups fail at the worst possible moment. Silent corruption, an incomplete backup scope that missed a critical volume, an encryption key nobody can find, a restore procedure that assumes a server that no longer exists — these don't show up until you try to restore, and the middle of an outage is the wrong time to find out.

  • Test restores on a schedule — quarterly at minimum for critical systems — and restore to a clean environment, not the live one.
  • Time the restore. If it takes nine hours and your RTO is four, you don't have a four-hour RTO; you have a documentation problem. Measuring it is how you find out.
  • Validate the data, not just the process. A restore that completes but produces a corrupt or partial dataset passed the wrong test.
  • Treat a successful restore test as audit evidence — timestamped, archived, mapped to your recovery control. It's proof the capability is real, not just documented.

Disaster recovery is bigger than data

Restoring data is necessary but not sufficient. A full disaster-recovery plan answers the operational questions a database snapshot can't:

  • Order of recovery. What comes back first? Dependencies matter — restoring the app before its database just produces a different kind of broken.
  • Who does what. A runbook with named roles, contact paths that work when email is down, and a declared decision-maker who can call it. This is the same muscle your incident response plan exercises, and the two plans should reference each other rather than contradict.
  • Where it runs. If the primary site or region is gone, the recovery target has to exist somewhere it isn't.

Make recovery a monitored control, not a quarterly memory

The failure mode for backups is drift: a new database gets added and never makes it into the backup scope, a job starts silently failing, retention quietly lapses. Treat backup health like any other security signal — backup success and recency belong on a dashboard, and a missed or stale backup should surface as a finding the same way a missed patch does, not as a surprise you discover during the disaster itself. Continuous verification is the same instinct behind continuous compliance monitoring: a control you check once is a control you've already stopped trusting.

The companies that recover gracefully aren't the ones with the most backups — they're the ones who restored from them on a calm Tuesday, before they ever had to do it on the worst day of the year.