Cloud server down.......

Kingdom · #1 (**permalink**) 24-10-11, 11:26 PM

......and then up, and then down! I've chatted with Perry who straight away said they are working on the problem without me even giving him details so I guess it's a system wide issue. I run my own monitoring (OpenNMS) which shows that DNS, FTP, HTTP and then the entire server have been going offline and then back on since 23:45hrs and now at 00:23Hrs they are still down.

Could someone at WHUK confirm what has happened and when we will see services returning to normal. If they are still flakey when my clients start to use them in the morning I'm going to have a lot of angry people to deal with tomorrow.

Could WHUK also confirm that we will be compensated for this further outage?

**John** · #2 (**permalink**) 25-10-11, 01:26 AM

Apologies, this should have been posted on the status pages. A large RAID rebuild was taking place at around this time on Cluster 4. Although this isn't usually service impacting, VMs which are heavily I/O intensive will notice an unusual slowness. A large RAID rebuild takes approximately 10 hours.

Most clients have scheduled backups taking place between midnight and 1am as well. Heavy I/O usage, a backup job running and a RAID Rebuild taking place is going to cause a small bottleneck period.

It will be finished by the morning, as we selected the overnight period to do it. The most intensive park finished shortly after you posted this message. If an individual service happens to crash, you can just restart it and it will come online quickly.