Tech FAQ: Why Is CPU Utilization Low, Yet the VMs Keep Lagging?

In virtualized platform operations, performance issues and system sluggishness are not necessarily due to the CPU being fully occupied. During troubleshooting, O&M teams tend to focus on visible metrics such as CPU utilization and memory usage, while overlooking CPU ready wait time percentage, a hidden performance killer in virtualization.

What Is CPU Ready Wait Time Percentage?

In a virtualized environment, CPU overcommitment is a common strategy for improving CPU resource utilization. In this setup, multiple virtual machines share and compete for host CPU resources. When vCPU allocation is too high, host resources are overloaded, or a sudden business peak occurs, a VM may need to wait for physical CPU scheduling, which affects the performance of the services it carries. This waiting duration is CPU ready time, and the monitoring metric of which is CPU ready wait time percentage.

CPU ready time: The total time during which a VM has completed task preparation and is ready to run, but has to queue for scheduling because physical CPU resources are occupied. In plain terms: the VM is ready to go, but cannot get a turn on the physical CPU and is forced to wait.
CPU ready wait time percentage: The ratio of the time a VM spends waiting for required CPU resources during runtime to the total runtime. It is used to determine whether the VM is frequently waiting for CPU scheduling. When the statistical period is one second, CPU ready wait time percentage = CPU ready time / 1 second.

Generally, when the CPU ready wait time percentage exceeds 10%, the VM is facing serious CPU resource contention. Services are more likely to lag, and O&M teams need to pay attention and intervene promptly. Focusing only on visible metrics such as CPU utilization at this stage can easily lead to missing the root cause.

❌ Common misconception 1: Low CPU utilization = no performance issue
✔️ Correct view: Even if CPU utilization is only 50%, a high CPU ready wait time percentage can still cause service lag.

Why Does CPU Ready Wait Time Percentage Increase?

In production environments, three factors mainly drive increased CPU ready time and VM sluggishness:

Excessive vCPU allocation: Some users habitually configure VMs with more vCPUs, such as 8 vCPUs or 16 vCPUs, even when the actual workload only needs 2-4 cores. Excessive vCPU allocation can intensify scheduling contention at the physical layer and increase the probability that the VM has to wait in the scheduling queue.
Host resource overload: When cluster node CPUs remain under high load for a long time, for example when overall utilization continuously exceeds 80%, multiple VMs compete for limited physical core resources at the same time. This increases CPU scheduling pressure and, in turn, the CPU ready wait time percentage.
Sudden business peaks: During peak scenarios such as e-commerce promotions, month-end settlement, and financial closing, compute demand can surge instantly and cause short-term CPU resource scheduling congestion.

❌ Common misconception 2: The more vCPUs a VM has, the better its performance
✔️ Correct view: Balanced resource allocation is better than over-provisioning. Overprovisioned vCPUs only add scheduling overhead.

❌ Common misconception 3: Alert thresholds can remain fixed for the long term
✔️ Correct view: Adjust alert thresholds dynamically based on business cycles such as major promotions, month-end closing, and off-peak seasons to achieve more effective resource governance.

Customer Story: Resolving Periodic Lag in a Financial System

Background

A customer’s financial system ran on a core business VM. During account closing and settlement on the 1st day of every month, the system experienced periodic lag: forms loaded slowly, voucher submission was delayed, and office efficiency was significantly affected.

From conventional metrics, the VM’s CPU utilization was only 60%, and memory usage showed no obvious anomalies. Therefore, relying on CPU utilization alone made it difficult to identify the root cause in time.

Based on the multidimensional metric monitoring capability of SmartX Enterprise Cloud Platform (SmartX ECP) and its alerting mechanism for multi-channel, second-level delivery, the O&M team found that during the monthly settlement peak, the VM’s CPU ready wait time percentage surged to 45%. This indicated that although the VM was ready to run, it could not obtain physical CPU scheduling for an extended period and was frequently left waiting. Analysis of host and cluster resource utilization points to two key issues:

Excessive vCPU allocation: The VM was configured with 16 vCPUs, but the actual workload did not require such high concurrent compute resources. Too many vCPUs instead increased underlying scheduling pressure.
Tight host resources: During the settlement peak, multiple VMs on the same host entered a high-load state at the same time, and the cluster CPU overcommitment ratio reached 4:1, further intensifying contention for physical CPU resources.

Optimization Measures

To address these issues, the O&M team took three optimization measures:

Configuration right-sizing: The VM configuration was adjusted from 16 vCPUs to 8 vCPUs, reducing unnecessary scheduling resource consumption.
Resource isolation: Before the monthly settlement date, the VM was preferentially live migrated to a lower-load node. If the target node had insufficient resources or contention still occurred after migration, CPU pinning was temporarily enabled during the settlement period to bind the VM to fixed physical CPU cores, ensuring resource determinism and performance stability during business peaks.
Alert hardening: Emergency-level alert rules were configured so that administrators would be notified immediately when the CPU ready wait time percentage exceeded 10%.

Results

After configuration adjustment and resource scheduling optimization, the financial system ran stably in subsequent settlement cycles:

CPU ready wait time percentage remained below 10%.
Overall financial settlement efficiency improved by 40%.
The periodic lag issue was fully resolved.