Identifying and resolving CPU % ready issues with vFoglight - vFoglight Blog - vFoglight

Author: James Kahn, Systems Engineer, Vizioncore

Hi, I’m James Kahn, Systems Engineer for Vizioncore out of Australia.

Today I want to share how to identify and resolve CPU contention (also known as % ready) issues with vFoglight. CPU % ready issues can impact environments of any size, and can be present even if overall CPU utilization is low. I’ve seen major CPU % ready issues in customer environments where the aggregate CPU utilization is below 25%.

What is CPU % Ready?

CPU % Ready is the amount of time a virtual machine spends waiting in line for the processor when it has work to do. For example, 10% CPU Ready Time means that a virtual machine spends 10% of its time waiting for processor cycles – and it’s not getting access to them quickly enough. CPU % ready is a real issue that can impact on the performance of your virtual machines and the applications running within them. As a general rule, CPU % ready above 5% is a bad situation for production virtual machine to be in.

Okay, so how do I diagnose them?

In your vFoglight installation, you may see an alarm against the cluster like the following:

The alarm above indicates that some virtual machines in the cluster “Lab Cluster” are waiting on CPU cycles – which we known as a CPU % Ready issue.

To investigate, we open vmExplorer view in vFoglight and navigate to the cluster “Lab Cluster”. Navigate to the “VMs” tab, and you will see a screen similar to this:

I can see from the above that my vFoglight virtual machine is being impacted by the CPU % ready issue, from the orange bar in the graph and the 7.4% Ready indicated in the table below. As mentioned earlier, anything above 5% is considered sub-optimal.

Can vFoglight identify the VM with the issue directly with an alarm?

Yes, you can alarm on CPU % Ready for individual virtual machines be enabling the alarm VMW Virtual Machine Percent Ready. It’s not enabled to default to ensure that CPU % Ready issues don’t impact on any SLAs configured.

How do I resolve it?

The goal of resolving CPU % Ready is to grant the suffering virtual machines access to the processor when they need it. Exactly how to do this depends on the cause of the CPU % Ready issue.Some potential resolutions include:

Investigate other virtual machines on the same ESX host as the VM with the issue for excess vCPUs. For time slicing reasons, VMs should only be given one vCPU unless they need more. While ESX time slicing improved significantly from ESX 2.5 to ESX 3+, this can still have a big impact.
Add more processor cores to the host. The processor time slices are then spread over more cores, allowing VMs to gain access to the processors more quickly.
Increase the CPU shares assigned to the VM with the CPU % ready issue. This will prioritize the VM above other VMs on the ESX host it resides on. You can use the shares view within vFoglight to help assist with this.

If you have any questions or comments, post them below.

- JK