Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

Time in a Linux 2.6 guest operating system runs faster than real time due to lost tick overcompensation (1006113)

Symptoms

  • When running Linux 2.6 in a virtual machine, time in the guest may run faster than real time
  • Time in a guest operating system is ahead of real time
  • Time runs faster in a guest operating system

Resolution

Background

Many guest operating systems keep track of time by programming a periodic timer interrupt and incrementing the current time by the period of the interrupt every time an interrupt is received. Such periodic timer interrupts are often referred to as "ticks" and this method of timekeeping is known as tick counting.
 
One drawback to this approach is that timer interrupts may be lost and when they are, the guest operating system's notion of time falls behind. Many Linux 2.6 kernels attempt to compensate for this by attempting to detect when a tick is lost and compensate for it. The guest reads a timer device counter from the timer interrupt handler to detect whether more than one timer period has elapsed since the last timer interrupt. If this is detected, it assumes that a timer tick was lost and compensates for that by adding in an appropriate number of timer ticks. Unfortunately, this concept is flawed. A "lost" timer tick detected in this way may not be lost, but still pending in the interrupt controller and about to be delivered as soon as the interrupt handler returns. Spuriously detecting lost timer ticks is a real bug that can occur on physical hardware as well as in a virtual machine. In a virtual machine the overcompensation is much more severe because of the different timing characteristics.
 
Affected Kernels
  • Mainline Linux kernels 2.6.0 - 2.6.17
  • Vendor kernels
    • SLES 9, 10  
    • SLED 9, 10
    • SUSE Linux 9.1 - 10.1  
    • Ubuntu 5.04 - 6.10
    • Mandriva Corporate Desktop 4.0
    • Mandriva Corporate Server 4
    • Mandrake 10, 10.1
    • RHEL 3 (64bit only)
    • RHEL 4
    • RHEL 5.1 (64bit only), 5.2 (64bit only)
    • CentOS 5.1 (64bit only), 5.2 (64bit only)
    • OEL 5.1 (64bit only), 5.2 (64bit only)
    • Turbolinux 10 Desktop
    • Turbolinux 10 Server
    • Asianux 3.0
Note: Mainline Linux kernels 2.6.18 and later are not affected.

Solution

The Linux kernel has different implementations of lost tick compensation depending on whether the timer device is the TSC or the PM timer, and whether the kernel is i386 (32bit) or x86_64 (64bit). The implementations using the TSC are much more prone to overcompensation.
 
For i386 kernels, using the kernel command line option clock=pit disables lost tick compensation.
 
For x86_64,  use the notsc kernel command line option. This disables use of the TSC and causes the kernel to fall back to the PM timer, which has much less severe lost tick overcompensation.
 
Note: Unfortunately, not all affected 64bit Linux kernels support the notsc kernel command line option. In particular, RHEL 4 Updates 0-2, SLES 9 (all service packs), Suse Linux 9.1, Suse Linux 9.2, Ubuntu 5.04, Mandrake Linux 10.1, Turbolinux 10 Desktop, and Turbolinux 10 Server do not.
 
Note: Unfortunately, not all affected 32 bit Linux kernels support the clock=pmtmr command line option. In particular Mandrake 10.1, Mandrake 10, and Turbolinux 10 Desktop do not.
 
Warning: Do not use the kernel command line option acpi=off . The kernel command line option acpi=off disables support for the PM timer (which is part of the ACPI subsystem), so if you provide this option together with notsc, the latter option is ignored and the kernel uses the TSC anyway. It is not necessary to use acpi=off in a virtual machine.
 
clock=pit has a bug that can cause the result of gettimeofday to temporarily go backward. If this is an issue for you, use clock=pmtmr instead. This selects the PM timer. For more information, see Time in a virtual machine jumps backward when using clock=pit (1006086).
 
When running a kernel that is doing some form of lost tick compensation (PM based or TSC based), it is important to run NTP in the guest to correct for the error introduced by lost tick overcompensation. VMware Tools periodic time synchronization does not correct for time in the guest getting ahead of real time.
 
When running a kernel that has lost tick compensation disabled, there are occasional lost ticks. Because of this, it is important to run NTP or VMware tools in the guest to correct for them. For more information, see Time runs slower than real time due to lost timer interrupts (1006088).
 
SLES 9 64bit kernel versions 2.6.5 - 7.312 and later support a kernel command line option, ignore_lost_ticks , that disables lost tick compensation.
 
For details on editing kernel configuration, see the Editing Kernel Configuration section of Timekeeping best practices for linux (1006427).

Additional Information

For detailed Linux timekeeping best practices, see Timekeeping best practices for linux (1006427).
 
Another issue with similar symptoms is described in, Time in virtual machine drifts due to hardware timer drift (1006072).

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 16 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 16 Ratings
Actions
KB: