Site Outage Friday 3rd February 2023

[MagnetON]

At approximately 10:40am on Friday morning the platform providing all the servers for TenTenths, along with other projects was subjected to a "ransomware" attack.

Hundreds of servers around the world have been impacted, not just Tenths. You can read more about it here.

This attack effectively wiped out the Tenths web and database servers.

Shortly after analysing the attack I commissioned new hardware for Tenths and started the process of re-creating the servers.

We have a very robust backup / disaster recovery plan for Tenths with multiple generations of backup stored off-site from our hosting provider (including a 20Tb NAS in my attic!)

It takes a bit of time to create new database and web servers, once these are created the backups have to be uploaded to them, the database restored, and all the rest of the web server files put back in to place.

It's time consuming because there's a fair bit of data to ship around.

From experience it takes around 24 hours to get everything moved and done.

In this case we returned the site to functional in around 19!

There's more work to be done, but hopefully it won't involve any long interruptions.