web130, web131, web132, web133 abnormal server performance
Incident Report for Heart Internet
Resolved
This incident has been resolved.
Posted Jul 14, 2017 - 12:38 BST
Update
The backup restore has now fully completed. There is a possibility of restore problems for some sites, again as per the last update if you have a website on one of these servers which is still offline please contact our Support Team for further assistance.
Posted Jul 10, 2017 - 01:34 BST
Monitoring
The majority of affected websites should now be restored. If you have a website on one of these servers which is still offline please contact our Support Team for further assistance.
Posted Jul 09, 2017 - 12:22 BST
Update
Restores are unfortunately taking longer than expected, however, they are being processed by our system.

We apologise for the continued delay and for the inconvenience caused.
Posted Jul 07, 2017 - 10:44 BST
Update
The backup restore commenced at 01:30am and we have checked this is running as expected. There is a huge amount of data to restore, which could unfortunately take a few days to fully complete. If you do have your own backups of your files you can of course upload this if you wish, however at some point it may get overwritten by the backup restore currently running. We do sincerely apologise for the inconvenience caused.
Posted Jul 06, 2017 - 02:02 BST
Update
The new network attached storage device (NAS) has been built and is ready for data to be restored to it. We are currently reviewing the safest and quickest way to do this, and then the restore will be started.
Posted Jul 06, 2017 - 22:46 BST
Update
Our systems administrators are not able to repair the filesystem, we are working on building the server again and restoring from backups. This will mean minimal data loss, but any changes made since the last backup run will be lost. The majority of websites will be restored from our backups run at approximately 2am this morning, in some cases these may be from an older date.

If you need clarification for this on a per site basis, contact our support team after this has been completed and we can advise when the site was restored from.

We do apologise for any inconvenience caused here.
Posted Jul 05, 2017 - 22:46 BST
Identified
The cause has been identified as a filesystem error on the network attached storage. Our systems administrators are working on getting this restored.

Further updates will be provided as more information is gathered.
Posted Jul 05, 2017 - 21:45 BST
Investigating
We are currently having adverse performance on this server, and are investigating the causes.
We expect to have it back to normal operation soon, and apologise for any inconvenience caused.
Posted Jul 05, 2017 - 21:38 BST
This incident affected: Webservers.