VMware Self-Service- Determining and changing the rate of timer interrupts a guest operating system requests

Many aspects of timekeeping are influenced by the rate of timer interrupts that a guest operating system requests. This article lists the default values for a number of kernels and how to measure the rate.

Many guest operating systems keep track of time by programming a periodic timer interrupt and incrementing the current time by the period of the interrupt every time an interrupt is received. Such periodic timer interrupts are often referred to as "ticks" and this method of timekeeping is known as tick counting.

Windows

Windows generally requests 64 or 100 interrupts per second depending on which HAL is use. There is a Windows multimedia timer API that allows applications to raise this to 1024 interrupts per second. There is no easy way to determine what the timer interrupt rate rate is from within the virtual machine. Timekeeping in a VMware Virtual Machine describes how to use TimeTrackerStats to determine the rate of timer interrupts the virtual machine is requesting.

Linux

Linux kernels request a timer interrupt from the Programmable Interval Timer (PIT) and an interrupt from each Local APIC Timer, of which there is one per virtual CPU. These are all requested at the same rate, but that rate varies from kernel to kernel. The requested rate of timer interrupts is referred to in the Linux kernel as HZ. The total number of timer interrupts requested by a Linux kernel is HZ * (N + 1) where N is the number of virtual CPUs in the virtual machine.

Default mainline values

Different values can be selected at compile time. Vendor kernels based on a given mainline kernel may have a different value than the default.

Linux 2.4: HZ = 100Hz.
Linux 2.6.0 - 2.6.13: HZ = 1000Hz.
Linux 2.6.14 onward: HZ = 250Hz.

Tickless Linux Kernels

Starting with Linux 2.6.18, Linux kernels began transitioning away from tick counting. Some new kernels use aperiodic interrupts rather than programming a periodic timer interrupt. These kernels are called tickless kernels. For tickless Linux kernels, HZ still influences the rate at which aperiodic timer interrupts are requested, but since these interrupts are not counted for timekeeping, the value of HZ does not impact timekeeping.

Generally tickless Linux kernels do not use the PIT at all, and instead just use aperiodic Local APIC timer interrupts.

Affected Vendor kernels

Redhat Enterprise Linux 4: HZ = 1000Hz
Redhat Enterprise Linux 5: HZ = 1000Hz
SUSE Linux Enterprise Server 9: HZ = 1000Hz
SUSE Linux Enterprise Server 10: HZ = 250Hz
Ubuntu 6.10: HZ = 250Hz
Ubuntu 7.04 and 7.10 Desktop: HZ = 250Hz

"divider" kernel command line option

Redhat Enterprise Linux 5 starting with version 5.1 and Redhat Enterprise Linux 4 starting with version 4.7 support a kernel command line option that allows a divider to be supplied that makes the kernel request HZ/divider timer interrupts per second. The syntax is divider=x where x is the divider.

To change the requested rate to 100Hz, pass divider=10 on the kernel command line. RHEL 5.1 64bit kernels have a bug in their implementation of the divider= option. The bug is fixed in Redhat RHBA-2007-0959. Do not use the divider= option with 64bit RHEL 5.1 unless this fix is applied.

Recompiling with a different HZ value

In standard 2.6 kernels, the timer interrupt rate is fixed at kernel compile time and cannot be changed by command line parameters. You can, however, recompile your kernel with a lower timer interrupt rate. 100Hz is adequate for most applications in a Linux guest. See the documentation for your Linux distribution for detailed instructions on how to build and run a custom kernel. Some version of Linux allow the value of HZ to be selected during the "make config" stage. Other kernels require the source to be modified. For those kernels, before recompiling the guest kernel, locate the following line in include/asm-i386/param.h orinclude/asm-x86_64/param.h: #define HZ 1000 . Change the value of HZ to 100: #define HZ 100 .

Measuring HZ

To measure the actual rate of timer interrupts in a Linux virtual machine:

Compare /proc/interrupts at the beginning and end of a 10 second interval.
From a shell prompt run cat /proc/interrupts ; sleep 10; cat /proc/interrupts .

There are several columns. The first is the IRQ number. The PIT is the device that generates the interrupts that Linux counts for time keeping. This is IRQ 0. Also of interest is "LOC" which is the Local APIC timer. For each vCPU there is a column listing the number of interrupts that device has raised on that vCPU. The final two columns give the interrupt type and a name for the devices on that IRQ.

To compute the rate of timer interrupts:

before = Sum of all PIT interrupts received on all vCPUs before sleep.
after = Sum of all PIT interrupts received on all vCPUs after sleep.
timer rate = (after - before) / 10 seconds

For example:

CPU0 CPU1

0: 125251 79291 IO-APIC-edge timer

1: 591 585 IO-APIC-edge i8042

8: 0 0 IO-APIC-edge rtc

9: 0 0 IO-APIC-level acpi

12: 67 8 IO-APIC-edge i8042

14: 753 643 IO-APIC-edge ide0

169: 2840 142 IO-APIC-level ioc0

177: 748 19 IO-APIC-level eth0

NMI: 43 35

LOC: 204282 204830

ERR: 0

MIS: 0

CPU0 CPU1

0: 134539 80039 IO-APIC-edge timer

1: 592 585 IO-APIC-edge i8042

8: 0 0 IO-APIC-edge rtc

9: 0 0 IO-APIC-level acpi

12: 67 8 IO-APIC-edge i8042

14: 771 715 IO-APIC-edge ide0

169: 2840 147 IO-APIC-level ioc0

177: 800 19 IO-APIC-level eth0

NMI: 43 36

LOC: 214314 214862

ERR: 0

MIS: 0

before = 125251 + 79291 = 204542

after = 134539 + 80039 = 214578

timer rate = (214578 - 204542) / 10 seconds = 1003/sec

This matches the 1000Hz rate for Redhat Enterprise Linux 4 that is expected.

If no PIT interrupts are seen in 10 seconds, the kernel is a tickless kernel and the HZ value does not matter for timekeeping.

For more information, see Timekeeping best practices for Linux (1006427).

Knowledge Base

Search the Knowledge Base:

Search the Knowledge Base:

Determining and changing the rate of timer interrupts a guest operating system requests

Purpose

Resolution