Bit embarrassing, but one of the UK’s favourite hosts and registars, 123-Reg, did a minor update over the weekend and a small code error managed to delete an unspecified number of their customer’s websites.
Mistakes happen, but this one is a bit of a disaster, as well as deleting the websites there are no backups for customers who haven’t paid for a back up service. If that’s you and you don’t have your own server image backup then your website has gone for the moment.
In a service update 123-Reg says “We are currently working on restoring your VPS packages using data recovery tools; in other words, we are attempting to recover your packages bit by bit, while checking data integrity at the same time”.
They’ve called in specialist data recovery experts to try to rebuild the deleted websites. The data loss affects websites hosted on a Virtual Private Servers (VPS), effectively multiple websites are hosted on a single server cluster but for the website admin it looks like you have your own server. By wiping the VPS it wiped multiple customers websites at a single keystroke.
The recovery work is underway, but 123-Reg are having to work on multiple physical servers in multiple locations to recover data wherever they can to rebuild the deleted websites. Due to this they’re unable to prioritise recovery of specific hosts but are having to work methodically, so if Aunties blog is restored before your ecommerce site it’s not that she has had special treatment.
Lessons to be learned
123-Reg is a major host with some 800,000 UK customers. Data loss can happen with any host through human error, catastrophic hardware failure or malicious intent.
Back up your data. Back it up with either a different host, cloud storage, or offline on a local hard drive. Don’t rely on your host having back ups because they might get wiped out too. Customers that have a local backup of their 123-Reg VPS can rebuild their servers, 123-Reg have instructions on how to do this.
Pay for a back up service. Those 123-Reg customers who opted to pay a little extra for a back up service can be back online the quickest.
Use a professional 3rd party to manage your website and make sure that a back up and restore agreement is built into your service level agreement. Nowadays it’s all too easy to assume that a website is forever and the ISP will be infallible. Normally they are, but the 123-Reg incident should be a wakeup call. It’s bad enough losing your photographs or your blog. Losing your ecommerce website and your livelihood is much more serious.
It’s all too easy to think that it’s the ISP’s job to look after your data and website
How to get help from 123-Reg
We feel for the guys at 123-Reg who are working around the clock to restore the servers and naturally their support staff are overloaded. If you wish to get an update on your own server you can contact them at [email protected] and they will update you as soon as they know the status of your data and website.
2 Responses
Oh BUGGER!
Just read about this on the BBC due to their headline ”football club’s website deleted” – turns out it was Premier league side Ross County, who lost their site on Saturday and got it back on Tuesday afternoon.
Hope none of the tamebay readers lose too much business because of this, as times are tough enough already.