Yandex Cloud data loss: 2.84% gone after drone strikes

“We restored access to more than 97% of customer data.” That’s the headline Yandex Cloud chose for its update this week. Read it again. If a bank told you it had found 97% of your money, you wouldn’t call that a success story. The Yandex Cloud data loss is exactly what the small print says it is: 2.84% of customer data is gone for good, because the hardware it lived on no longer exists.

The context is grim. Ukrainian drones have been hitting Yandex data centers for weeks now, at least three sites so far, and the company sometimes called Russia’s Google is learning in public what “disaster recovery” means when the disaster is literal.

Yandex Cloud data loss, in numbers

Yandex says the remaining 2.84% “cannot be restored due to critical damage to equipment,” as Tom’s Hardware reported after spotting the admission. Nearly three percent sounds small until you’re the customer whose production database was in that three percent. There’s no partial credit for a VM that’s 97% there.

The strikes are a reminder that a cloud is still a building. Steel, concrete, cooling, and a roof that can be hit. I wrote about the Kyivstar attack that wiped a telecom’s core a while back; this is the mirror image, and far less subtle. No malware required, just explosives.

Blaming customers for missing backups

Here’s the part that made me put my coffee down. Yandex explains that the unrecoverable data was lost partly because of “the lack of customer backups on the platform.” Customers who used the Yandex Cloud backup tools, the post adds, “did not lose any data.”

So the cloud provider whose data center got blown up would like you to know that, really, you should have planned for that. Technically true. Morally, it’s a bit rich. People buy cloud storage precisely so they don’t have to think about where the disks physically are. Replication across zones is the whole pitch, and this incident suggests plenty of workloads were sitting in a single room.

Yandex has at least changed one thing: every new virtual machine now ships with backups enabled by default. It’s the right call, about one war too late. And it’s also opening a new availability zone, ru-central1-f, at an undisclosed location, with a reduced set of services for now. Undisclosed being the operative word when the previous addresses turned out to be targets.

The 3-2-1 rule still beats any SLA

You’re probably not hosting on Yandex. Doesn’t matter. The lesson travels. Your provider’s “99.99% durability” is a statistical promise about disks failing one at a time, not about a building failing all at once. Fire did it to OVH in Strasbourg in 2021. Drones are doing it to Yandex now. Something else will do it to someone else next year.

Three copies, two media, one off-site: the boring 3-2-1 rule has survived every one of these incidents, and it costs a fraction of what you’d lose. Keep the off-site copy with a different company, in a different country if you can. We’ve already seen how fragile the physical side of this industry is when $11 million of Nvidia GPUs vanished from a truck, and how a few remote commands can cook a supermarket’s freezers. Infrastructure is a lot more flammable than the brochures suggest.

And if your provider ever leads with the percentage it saved rather than the percentage it lost, you already know which side of that number you’re on.