Markus Kuhn, Computer Laboratory, University of Cambridge
A plea to the developers in charge of the real-time clock driver in Microsoft operating systems.
What is Universal Time? Universal Time (UT), also known as Greenwich Mean Time (GMT), is the astronomical local time on the prime meridian. Its atomic-clock representation, known as Coordinated Universal Time (UTC) and defined in ITU-R Recommendation TF.460-4, is the official international reference time scale, the basis all time announcement services and most national definitions of time. Universal Time is not affected by any national Daylight Saving Times. It has long become the global reference time in distributed computer systems and many other applications.
What is the PC’s Real Time Clock? Every IBM PC compatible computer features a battery-backed "Real Time Clock (RTC)" chip. This contains a 32768 Hz crystal oscillator that drives a date/time counter. It also contains 114 bytes of battery-backed CMOS RAM that are reserved for BIOS use. The chip typically used is compatible to the Dallas Semiconductor DS1287 and Motorola MC146818B RTCs.
How does MS-DOS relate to this? The original IBM PC was released with the first Microsoft DOS and Basic versions in the early 1980s. At that time, the engineers involved did not consider modern issues such as networking across time zones, mobile computing, automatic clock synchronisation, automatic daylight saving time updates, 24/7 database server operation, dual-boot installations, virtual-machine servers, etc. Microsoft and IBM established at the time the (now highly problematic) convention to keep the RTC adjusted to whatever the PC’s user considers her local time zone. MS-DOS never had any notion of Universal Time or any time zone other than "local time". The question of time zones only emerged when MS-DOS C compilers began to implement the standard C API function gmtime(), which applications can use to query Universal Time. They followed a practice now documented in IEEE POSIX.1 Section 8.3, namely interpreting the environment variable TZ as a description of local time relative to Universal Time, in order to convert the local time provided by MS-DOS into Universal Time.
This never worked properly, especially not near the repeated hour after the end of a daylight-saving-time period. Since an entire hour is repeated once every year, there is no unambiguous way for a function to convert local time into Universal Time, leading to wild jumps in the output of gmtime() under MS-DOS each fall.
MS-DOS never was designed for running mission-critical 24/7 servers reliably, so this remained a minor concern. For those who cared, replacement RTC drivers (CLOCK.SYS) appeared eventually for MS-DOS, such as clk360rs.zip, which allowed users to run the RTC directly in UTC and which determined the local time based on a configured TZ string when the system.
The POSIX platform and C programming language had been developed from early on for use in global networks across many time zones. Most POSIX systems keep their hardware real-time clock in Universal Time, and that is also the only time zone that the kernel of a standard POSIX system is aware of (e.g. in file modification time stamps). Conversion to any displayed local time zones is performed on POSIX systems by the C run-time library, either based on the TZ environment variable or time zone information stored in configuration files.
Eventually, the Microsoft Windows API also introduced a notion of time zones and application access to Universal Time, but even today all Microsoft operating systems still continue with the dangerous old MS-DOS practice of running the RTC in local time. Even the current Microsoft Windows 7 and the server versions do that by default!
The continued use of local time in the RTC by Microsoft operating systems causes various problems:
I would like to urge Microsoft: please follow the time-proven POSIX practice of running the RTC in Universal Time. Ideally, this should become the standard configuration for the Microsoft platform. At least optional support would be extremely desirable. This way all described problems are solved easily:
It seems to me a really rather simple change to add to the Windows RTC driver in every Microsoft operating system a little configuration option that determines whether the time stored in the RTC is Universal Time or local time. I very much hope, Microsoft will consider implementing that at least as an option. Many discussions of this proposal on the comp.protocols.time.ntp newsgroup and the tz mailing list showed enthusiastic support from other computer time keeping experts for this simple proposal.
Initially, it should perhaps only be an option for administrators of 24/7 servers in data centers and users multi-OS-version machines, but eventually, there is really no reason why the Universal Time RTC shouldn't become the standard for every new or updated Microsoft installation. After all, correct and reliable knowledge of Universal Time by applications is required by numerous Internet standard protocols (e.g., RFC 822). The time and time zone have to be configured correctly by the user anyway during the installation of a new operating system, which is a good opportunity to set the RTC to Universal Time.
In particular, no changes are necessary in existing PC hardware and BIOS firmware in order to run the RTC in UT. This is entirely up to the operating system. The BIOS is already completely ignorant about what time zone its RTC runs in, and it can continue to remain so when the operating system decides that this timezone always be UTC.
The perhaps only change BIOS vendors might eventually want to make is in the documentation of the BIOS configuration menus, which should then encourage users to enter Universal Time and not some local time. But non-technical users are today no longer expected anyway to use the BIOS configuration menus, and should certainly not be encouraged to do so in the future. Perhaps, access to the battery clock from BIOS menus should be moved generally into "for expert users only" sections.
While browsing through the Windows 2000 SP2 kernel binary with the "strings" tool, I noted a UTF-16 encoded string "RealTimeIsUniversal" (NTOSKRNL.EXE:bbd4, NTKRNLPA.EXE:9304). It turned out that Windows NT tests a long forgotten undocumented registry entry
for instructing the kernel that the RTC runs in UT (type REG_DWORD, values 0 for local time or 1 for Universal Time). When you set RealTimeIsUniversal=1, the kernel initialization code will jump after reading the CMOS clock over the section that translates local time into CMOS time and will take the CMOS time directly as the Universal Time kept by the NT kernel clock.
2001-07-09: I got a reply from someone in Microsoft's Base Kernel Team who got interested in RealTimeIsUniversal and they had a look at the relevant parts of the NT kernel source code. The RealTimeIsUniversal flag is there (a leftover from the days when NT still ran on RISC machines with UTC RTCs), but its implementation seems now incomplete and it is currently not covered by Microsoft's documentation and regression test suite, therefore using it is not recommended at this time. A couple of potential RealTimeIsUniversal bugs have been identified over the past few days, there might be more. For instance, the kernel debugger assumes that the CMOS time is local time and will get the time wrong when RealTimeIsUniversal=1. There might be a similar problem in the code that resumes processing after the CPU was suspended or in the code that calculates DST change times. I hope they will look into fixing these problems, but they haven't made any promise yet that RealTimeIsUniversal=1 will be officially supported. In any case, it is unfortunately too late at this stage for a fix to get into the forthcoming Windows XP release. Perhaps RealTimeIsUniversal=1 can be established as the default for new platforms such as IA64 where there is no DOS-compatibility requirement, and then it would be fully supported again?
2004-09-02: A blog post by Windows developer Raymond Chen suggests that (a) the Windows development team really hasn't grasped the magnitude of problems caused by day-light switching the RTC (especially in dual-boot environments), and that (b) Microsoft might be more inclined to move to UTC if BIOS manufacturers simply removed the RTC clock display from their setup menus, because some Microsoft folks appear to think that displaying UTC in the built-in BIOS config menu (next to other arcane settings such as the type of floppy drive used) is still way too scary for mere mortals. (But none of what he says justifies not even fixing the RealTimeIsUniversal=1 support, which could simply be used only by UTC-aware customers who know exactly what they are doing.)
2006-07-04: Various Microsoft Windows Vista beta testers have told me that this next-generation operating system still is not capable of running the CMOS clock in UTC.
2006-11-02: Microsoft and Novell announce broad collaboration on Windows and Linux interoperability and support – full support for RealTimeIsUniversal=1 would seem like something that ought to be high on their list ...
2008-10-31: Hurray! Someone from Microsoft's Core Operating Systems Division hints in an email to me that both Vista SP2 and Windows 7 will fix the problems in the RealTimeIsUniversal=1 support that have made running the CMOS clock in UTC so far not practical with Windows (i.e., the time was wrong after resuming a suspended/hibernating Windows). He warns though that this improved support for UTC in the CMOS clock may not immediately be widely documented and details may change in the future. I'm also being told that when RealTimeIsUniversal=1, then Windows Vista SP2 and 7 will no longer update the CMOS clock, so you may want to run an NTP client that does that in the other operating system. This should make life substantially easier for people who use Windows in a dual-boot environment (e.g., alongside MacOS).
2012-02-10: Microsoft Support released a bug warning and advises for the time being not to set RealTimeIsUniversal: System may become unresponsive during Daylight Saving Time (DST) changeover if RealTimeIsUniversal is set.
2013-03-04: Microsoft Support released Hotfix 2800213 for Windows Server 2008 and Windows 7, fixing a bug in the way hardware abstraction layer (HAL) uses the CMOS clock time, which could have led to 100% CPU utilization after a DST changeover if RealTimeIsUniversal=1.
Looks like we are very slowly getting there ...
Various Microsoft KnowledgeBase articles provide examples of the rather serious conceptual problems that the use of local time in Microsoft APIs has created:
The use of local time in the RTC is only the most easy to fix problem in a whole range of related issues. None of the above problems would have appeared had Windows followed the POSIX practice of using UTC everywhere and providing conversion to the local time in any specifyable time zone only as a completely separate library functionality. Local time should be completely irrelevant for the operation of an operating system, it is just a convenient display format for application user interfaces. Microsoft API architects might in that context also be interested in the Proposal for a New Time API for ISO C 200X that solves a range of other problems in the current POSIX and ISO C time API.
Thanks to the folks at Microsoft (Fabien Petitcolas, Rob Earhart) and on comp.protocols.time.ntp and tz who provided feedback on this memo.
Please don't hesitate to contact me if you have any related information, ideas, questions or suggestions. I'll keep this text up-to-date as new information arrives.
created 2001-07-02 – last modified 2012-03-12 – http://www.cl.cam.ac.uk/~mgk25/mswish/ut-rtc.html