We are happy to report that all restorations have been completed on Kvmhost69. Please contact our support team if you are unable to connect to your server.
Posted Jan 31, 2018 - 20:10 GMT
Update
We are roughly half way through the restorations for this KVMhost. The remaining servers left to be restored are expected to be complete in the next 24-48 hours.
Some users may notice that their servers are back up, but in a read-only state. We would advise you power the server off, and then boot into recovery mode and run a file-system check. If you're unsure on how to do this, or if you have issues connecting to your server following this incident, please contact our support teams.
Posted Jan 31, 2018 - 13:10 GMT
Update
Good progress is being made in the restorations now, and with some of the larger server images restored we're hoping the process will now speed up.
Posted Jan 31, 2018 - 08:13 GMT
Update
The restoration work is still ongoing, we apologise for any inconvenience caused.
Posted Jan 31, 2018 - 02:09 GMT
Update
The restoration work is still ongoing, we apologise for any inconvenience caused.
Posted Jan 30, 2018 - 20:02 GMT
Update
The restoration work is still ongoing, we apologise for any inconvenience caused.
Posted Jan 30, 2018 - 15:30 GMT
Update
The restoration work is still in progress and it is unfortunately hard to give any 'ETA' at present, but we are doing all we can to speed up the progress.
Posted Jan 30, 2018 - 08:08 GMT
Update
The recovery process is still underway, however the restoration speed has slowed down due to the amount of data activity on the backup server network, as a result the estimated completion time is no in accurate, we are however doing our best to get the affected services back online as soon as possible. We will update this page with the latest once we have any further information.
Posted Jan 29, 2018 - 21:57 GMT
Update
The recovery process is still continuing, and we cannot offer a firm time-line however we're currently estimating around 24-48 hours for this process to complete. However, this time-frame is subject to change, and we will update this status page once we have any further information, or any updates to the time-line.
Posted Jan 29, 2018 - 16:54 GMT
Update
The first few machines have been brought back online now, progress continues with the rest. At the moment restoration speeds are too variable to give a timeline for 100% completion.
Posted Jan 29, 2018 - 09:41 GMT
Update
The backup restore is still running and may take some time to complete. It is hard to give an estimate at this time, but we will the situation in full when our senior system administrators are in the office.
Posted Jan 29, 2018 - 06:34 GMT
Update
The kvmhost is currently back online, backups are in the process of being restored as such customers should see each individual servers coming back online automatically after they've been restored.
Should you encounter any problems with the restored servers i.e failing to boot up, or issues with services starting up, please contact our support team with a detailed respond stating the issue so that we can provide some assistance.
We thank you all for your continued patience.
Posted Jan 28, 2018 - 17:02 GMT
Update
Unfortunately it was not possible to recovery from the hardware failure without data loss, due to there being more than 2 failed disks in the raid array we suffered minor data loss on this host.
As a result we've had to make the decision to restore the affected servers (virtual servers) from our carefully taken disaster recovery backups, this restoration is currently being prepared and will shortly begin, the affected virtual servers will come back online after the restoration has completed.
Additionally, customers may notice a slight roll-back in the restored data, as the backups being restored from can range from 1 - 7 days old, dependant on the virtual server in question.
Once again, we do apologise for the inconvenience, and we hope to have this service back online as soon as possible.
Posted Jan 28, 2018 - 13:22 GMT
Update
Update: 2018-01-28 10:27
The failed disks in the array have been swapped out with new ones, we are currently int he process of rebuilding the new disks before adding them into the array. We apologise for the inconvenience.
Posted Jan 28, 2018 - 10:29 GMT
Identified
kvmhost69 has suffered a disk failure, of which has forced the file-system to mount in read-only mode, we are currently investigating this issue and do expect to have this back to normal operation as soon as possible.
Servers on this host are currently in a shut-off state whilst we replace the failed drives in the Raid-10 array.