VPS High load 21 March 2008

[optrex]

Hey gusy,

I had a very nice chat with Malcolm this morning when my node was restarted due to high loads (apparently several users were backing up at the same time this morning).

I've been out all day since, have come back and I'm getting high loads again. Is there anything that can be done to find the source?

I don't think its with me, it may be another VPS user, but I was under the impression other people servers activity should not affect mine. Here is the latest info from fld.

Apache Server Status:

Current Time: Friday, 21-Mar-2008 18:38:30 GMT
Restart Time: Friday, 21-Mar-2008 12:11:00 GMT
Parent Server Generation: 0
Server uptime: 6 hours 27 minutes 29 seconds
Total accesses: 56097 - Total Traffic: 78.4 MB
CPU Usage: u15.6 s63.24 cu.12 cs0 - .34% CPU load
2.41 requests/sec - 3535 B/second - 1465 B/request
13 requests currently being processed, 4 idle workers


VMStat:

Output from vmstat:
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
4 0 0 282452 0 0 0 0 6 1 0 177 0 0 98 2 0


I also have a PS, but its too long to post here.

[Administrator]

Please let me know if you have opened a ticket so far. I would like to have a ticket # to get it sorted from our VPS Team.

I would like to assure you that we will go through deep analysis of this problem and we will get this sorted permanently.

[optrex]

Hi Admin,

Thanks for the response, I dont have a ticket as I've not opened one.

TBH, after the nightmare of my last host, you guys have been so reliable I've not been overly concerned.

Would you like me to open one or shall I just leave it with you?

[Administrator]

Originally Posted by optrex

Hi Admin,

Thanks for the response, I dont have a ticket as I've not opened one.

TBH, after the nightmare of my last host, you guys have been so reliable I've not been overly concerned.

Would you like me to open one or shall I just leave it with you?

I need a ticket as that helps me to get things sorted. If someone ignores your ticket then they need to go through complicated steps designed by our QA department.

[optrex]

OK, I've just opened one, its CGK-23160-802

Its mainly curiosity as I didn't think other peoples actions on their servers should affect load on mine as although the resources are shared, they are a guaranteed allocation.

Anyway, thanks for looking into it for me

[optrex]

ooops, looks like I tempted fate, all services offline atm

[Administrator]

Originally Posted by optrex

ooops, looks like I tempted fate, all services offline atm

Node hosting your VPS is facing quota problems. Swsoft Engineers have provided some fix for this problem which is being implemented right now. It will take 30 minutes more for all services to get back to normal. We apologize for this unscheduled maintenance.

[optrex]

OK thanks. I dont mind outages as long as there is communication and I know whats going on - its just one of those things.

Today I've experienced my first outages since joining you and the communication has been excellent. I'm cool

[Administrator]

Originally Posted by optrex

OK thanks. I dont mind outages as long as there is communication and I know whats going on - its just one of those things.

Today I've experienced my first outages since joining you and the communication has been excellent. I'm cool

Thanks for your understanding. I hope there won't be any other problem left on your node once the quota fix gets completed.

[optrex]

Just to open this one back up as I've become aware of regular high server loads between 03:00 and 04:30. At this point my site traffic is down to 2% and most of what is being access is cached, so there is little in the way of load. My backups have completed by 02:30 and the only other thing that may run is the automatic cpanel scripts update and possibly a full node backup.

I am also getting random high loads during the day for no apparent reason. When I check my WHM it will show an average load that remains high (around 4 to 6) whilst my PID views will be like 5.0% max with the rest at 0.3% or 0%

A quote from my support ticket AQX-11544-166 states

"It is nothing worrying if you are receiving some timeouts on the VPS at 3.00 to 4.30 as this is the time when the VPS under goes backup operation as the standard cPanel server starts the process at 3.00. There are some other VPSs on this node that perform the backup operation during this period and some starting from 2.00 so it is obvious for the hardware node to be under specific load."

At face value, I was happy with the explaination, but after some thought I became concerned.....this being my reasonsing;

As a user, you step up from shared hosting to VPS to be free of the restrictions of your site being affected by the misuse of others on your shared platform. You move to VPS to operate under a set number of guaranteed resources.

I am experiencing high loads, which is usually associated with CPU. When going for a VPS you generally check CPU level as being guaranteed or of "equal share" so that you are not affected by other servers on your node (unless the node goes down of course).

In checking the sales page

VPS Hosting technology presents multiple isolated server environments on a single physical server. Each VPS uses its own hard disk space, RAM, CPU, Dedicated IP addresses and it can be rebooted independently. Virtual Private Servers allows multiple customers to share the expense of hardware and network connections without sacrificing privacy, performance or preference.

Combined with "Dedicated CPU and RAM" under the features tab, and everything you read would indicate a dedicated or "equal share" CPU resource.

However the answer given by the technical department indicates the CPU is a shared resource where someone can, if they so desire, use all the available share, thus resulting in high loads for everyone else. To back this up, the knowledge base suggests the following...

How much of the processor power is guaranteed to me and how much will it burst to if I need more?
Ans: The processor is shared between all users so there is no guaranteed amount of processor power alloted to anyone. You can utilize the full processing power provided you do not overload the server.

In summary I am confused. Sales indicates an equal share or guaranteed amount, where other users on the node will not affect my VPS. But experience, support and the kb article indicate its shared and its use by others will affect my server.

The latter of course is not ideal and may indicate further quota issues.

[Administrator]

Processing Power cannot be equally distributed between all VPS's hosted on a specific node. If we think of distributing 2.4 GHz CPU between 12 VPS's then you won't get more than 200 MHz out of it which is insufficient to run even a single website. CPU resources include many other factors, out of which few are shared and few are dedicated for each VPS customer.

[optrex]

OK that makes sense, thanks. But surely its not reasonable to expect timeouts as "normal" whatever time of day it is, even if it is during a backup?

[Administrator]

Originally Posted by optrex

OK that makes sense, thanks. But surely its not reasonable to expect timeouts as "normal" whatever time of day it is, even if it is during a backup?

If all VPS's demand more CPU power at night for completion of Backup then CPU load is bound to rise. We can think of separating backup crons of all VPS's by 30 minutes so that all VPs's don't overload main hardware node at same time.

I will request our VPS Team to look into this.

[optrex]

Thanks, I will keep an eye on things over the next few days. The spikes during the day I was getting, seem not to have arisen today, so thats good news.

[optrex]

After my post last week things have improved steadily. Within 24 hours, there was a noticable reduction in the context switches and since there has been no adverse daytime slowing of the site. (see interrupts-week.png)

During the overnight routines, the load has remained steady, with a peak of around 3.0. Initially this still had some detrimental effects on the site (see load-week.png)

However, the day after the announcement that the carrier had been changed http://www.webhosting.uk.com/forums/...0&postcount=27 this slowness has also reduced (from 50-60 second page load to 15-20 seconds), whilst the load itself remains the same. (see load-day.png)

The information has been graphed using Munin server monitoring tool, from within WHM, see Munin server monitoring