Time is an illusion, Unix time doubly so...

As you well know, on Unix systems we measure time as the number of seconds since "the epoch": 00:00:00 UTC on January 1st, 1970. This has made a lot of people very angry and been widely regarded as a bad move.

For starters, this definition is not based on something sensical such as, say, the objective frequency of vibration of a Cesium-133 atom, but on a convenient fraction of the time it takes a particular large rock to complete a full rotation around its own axis.

You see, in Unix time each day is guaranteed to have exactly 86400 seconds, and we pretend that this number monotonically increases. When it turns out that because our reference rock actually took a bit longer than what was convenient for us to complete its rotation and we need to add a leap second, we simply pretend that didn't happen, and our timestamp mechanism ceases to identify a unique point in time.

The other thing that continues to cause problems here is that as we try to count seconds, we run into data storage and representation issues, because honestly, computers aren't really all that good at numbers, it turns out. Talk about "epoch fail".

So how did we get here? It all began back in 1971, when the First Edition Unix Programmer's Manual defined Unix time as "the time since 00:00:00, 1 January 1971, measured in sixtieths of a second":

That's right, the original Unix epoch was 1971-01-01T00:00:00. What timezone, you ask? Well, it sure wasn't "UTC", because that didn't replace GMT as the standard time until 1972. Secondly, note that time was measured in 1/60ths of a second, not in seconds. Why would that be?

Remember that at that time, Unix was being developed in the US on a PDP-11. These systems had a Line-Time Clock (LTC), which takes the AC power frequency to generate an interrupt for the processor. This interrupt is then used to update the system clock.

The funny thing here is that this interrupt frequency thus depends on the utility frequency of its power supply. In much of Asia and Europe, this frequency is 50 Hertz, but in the US, the Westinghouse Electric Corporation observed that the carbon arc lamps in use at the time had less flicker at 60 Hz and standardized on that frequency. (Fun fact: Westinghouse, an early competitor to Thomas Edison and General Electric, would later buy CBS and was then bought by Viacom.)

A weird side effect of this US/Europe split is that in Japan, both are used: in the East, where the first generators were purchased from the German AEG and installed in Tokyo, the utility frequency is 50 Hz; in the West, they were installed in Osaka from G.E., using 60 Hz. As a result, the country now uses several High-Voltage, Direct Current (HVDC) electric power transmission systems to convert the elecitricity between the two regions!

Aaaaanyway... so yeah, the 1st Edition UNIX measured time at those 60 Hz using a 32-bit integer, thus only being capable of counting 2^32 / 60 ticks/second * 60 s/m * 60 m/h * 24 h/d * 365 d/y = 2.3 years, as noted in the original manual. Some time in 1971, the time measurement was redefined to count seconds, now capable of representing 136 years of time, and the Unix epoch date was -- contrary to the common misconception that it represents the birthdate of Unix -- somewhat arbitrarily picked to be 1970:

(The actual birthdate of Unix lands somewhere in 1969, when Ken Thompson ported "Space Travel" to the PDP-7, so it's easy to see why the Unix epoch seems like it would represent the birth of Unix.)

Leap Seconds

Ok, so now we have our 32-bit seconds-since-the-epoch counter, which monotonically increases at a rate of 1 Hz, and promising us to count exactly 86400 seconds for any 24 hour period. But our silly reference rock appears to be slowing down in its rotation, making it necessary to fudge this number a bit every so often.

Now we can't really blame the Unix epoch time for not accounting for leap seconds initially, since those did not exist up until 1972. Since then, we've had 27 positive leap seconds (as of 2022), the last one being introduced at the end of 2016. Whenever a positive leap second occurs, our Unix epoch time simply pretends it didn't, with the result being that two dates map to the same epoch timestamp (epoch-time.c):

From epoch to time, via gmtime(3) to strftime(3):
1483228798 2016-12-31T23:59:58
1483228799 2016-12-31T23:59:59
missing leap second here
1483228800 2017-01-01T00:00:00

From time to epoch, via strptime(3) to mktime(3):
2016-12-31T23:59:58 is 1483228798
2016-12-31T23:59:59 is 1483228799
2016-12-31T23:59:60 is 1483228800
2017-01-01T00:00:00 is 1483228800 duplicate timestamp here

(And let's forget all about negative leap seconds, or the fact that some Unix system define the range of tm_sec from [0-61], accounting for a mystical "double leap second" that never actually existed.)

Year 2038 Problem

Now even counting monotonically forward and ignoring leap seconds, the time_t data type used to measure seconds since the epoch is eventually going to overflow. Good thing we have standards to save us here! Sure, POSIX is quiet on this, but the good thing about standards is that there are so many to choose from, right? Oh wait *taps ear*, what's that? I see.

Bad news everybody: standards are not going to save us. For example, the C standard says:

Well, ok then. With a 32-bit time_t, we can then account for roughly 136 years starting in 1970. Except, time_t is a signed 32-bit integer, so we only get 68 years in either direction, leaving us with what you all know as the "Year 2038 Problem" (check progress).

That is, the largest date that can be represented as a time_t using a 32-bit signed integer is epoch 2^31 - 1 = 2147483647, or 2038-01-19T03:14:07Z:

Easy to fix, right? We "just" change the data type of a time_t to be a 64-bit signed integer, yielding a theoretical maximum epoch date of 2^63-1 = 9223372036854775807 after January 1st, 1970. As people like to point out, that represents a date around 292 billion years in the future, or roughly 22 times the estimated age of the universe, and thus officially Somebody Else's Problem.

But -- there's always a "but", isn't there? -- is that really what happens? Why don't we give it a try and see how different systems behave when we hand them not-quite-so-reasonable times to chew on.

mktime(3) wrapping

While time is represented as a time_t, the other common format in use to represent broken down time is a struct tm, from which you can then get a time_t by calling mktime(3). The distinct elements of the struct tm are noted by POSIX to be:

int    tm_sec   Seconds [0,60]. 
int    tm_min   Minutes [0,59]. 
int    tm_hour  Hour [0,23]. 
int    tm_mday  Day of month [1,31]. 
int    tm_mon   Month of year [0,11]. 
int    tm_year  Years since 1900. 
int    tm_wday  Day of week [0,6] (Sunday =0). 
int    tm_yday  Day of year [0,365]. 
int    tm_isdst Daylight Savings flag.

The funny thing about the ranges noted there are that they are... advisory at best. POSIX specifically notes:

So what exactly happens when you feed mktime(2) a struct tm with values outside of these ranges? Consider the following program:

int main() {
        struct tm t;
        time_t epoch;

	/* 2022-12-31 */
        t.tm_year = 122; t.tm_mon = 11; t.tm_mday = 31;

	/* 22:57 */
        t.tm_hour = 22; t.tm_min = 57;

        t.tm_sec = 3785;

        if ((epoch = mktime(&t)) == -1) {
                err(EXIT_FAILURE, "mktime");
        }

        (void)printf("%s", ctime(&epoch));

	return 0;
}
$ cc -Wall -Werror -Wextra t.c
$ ./a.out
Sun Jan  1 00:00:05 2023
$

Here, before we set tm_sec, our struct tm represented December 31st, 22:57:00, 2022. But then we set tm_sec to 3785, i.e., 1 hour, 3 minutes, and 5 seconds. This then leads to tm_min to be incremented by 3 minutes, which leads to tm_hour to be bumped up by one. Then we increment tm_hour once more (from the remaining 3600 seconds from tm_sec), which wraps that value over to 00 and leads to tm_mday needing to be incremented. tm_mday now wraps to 01 with tm_year incrementing, ultimately leading to the final date being 2023-01-01T00:00:05.

This normalization of timestamps gets even more confusing once you add negative values (e.g., tm_mday = -1 means the last day of tm_mon - 1) or (again) with leap seconds in play and can make your head hurt. It's best to avoid this situation.

Funny date(1)

Now let's look at epoch dates at or around the epoch. We'll ignore the fact that epoch times prior to 1972 are not well defined, because our epoch time is defined to be in UTC, but (see above) UTC had not yet been standardized before 1972. We'll just be like Unix with leap seconds and pretend that doesn't concern us.

Displaying an arbitrary date using the date(1) command is easy enough. You just give it the date in the format CCyymmddHHMM.SS. Unless you're using Linux, where GNU date(1) wants to use what is perhaps the most infuriating format (MMDDhhmmCCYY.ss), even using inconsistent capitalization of the date fields compared to the strftime(3) % sequences. Why would you stick the year in between minutes and seconds?

But using either format is not useful for our purposes anyway, because at some point we want to go beyond four-digit years, so instead let's specify seconds since the epoch directly ("-r <seconds>" for BSD date(1), "--date @<seconds>" for GNU date(1)).

(GNU date(1) using AM/PM instead of a 24-hour clock here is annoying, but ok.)

Let's see what happens if we try to probe for 32-bit time_t s. As discussed above, the Year 2038 Problem would be triggered at epoch 2147483648 / -2147483649:

Oh, hey, look at that. OmniOS seems to have problems with epoch dates beyond 2^31. I wonder what's going to happen if we not only display the date, but set it? Let's loop, set the date, and print the date:

omnios$ for s in 5 6 7 8; do sudo date -u 011903142038.0$s; date; done
January 19, 2038 at 03:14:05 AM UTC
January 19, 2038 at 03:14:05 AM UTC
January 19, 2038 at 03:14:06 AM UTC
January 19, 2038 at 03:14:06 AM UTC
January 19, 2038 at 03:14:07 AM UTC
December 13, 1901 at 08:45:52 PM UTC
ld.so.1: date: fatal: /lib/libc.so.1: Value too large for defined data type
Killed
omnios$ date
ld.so.1: date: fatal: /lib/libc.so.1: Value too large for defined data type
Killed
omnios$ ls
ld.so.1: ls: fatal: /lib/libc.so.1: Value too large for defined data type
Killed
omnios$ sudo reboot
sudo: unknown uid 100
sudo: error initializing audit plugin sudoers_audit
omnios$

Nice. Note how the date actually wraps around before the OS freaks out. Now the other systems should not have any problems setting a date. Right?

That is, on NetBSD we can't set the date to before the epoch, while on Linux (since kernel version 4.3) we can't set the date before the current uptime. This is, settimeofday(2) will return EINVAL, because of guarantees made by its "CLOCK_MONOTONIC" clock that, according to POSIX "represents the amount of time since an unspecified point in the past":

Along those lines, it looks like OmniOS calculates uptime based on boot time relative to the system date, while NetBSD and Linux, for example, keep separate counters:

omnios$ sudo date -u 010100001970.00
January  1, 1970 at 12:00:00 AM GMT
omnios$ uptime
00:01:46    up -49 min(s),  1 user,  load average: 0.00, 0.00, 0.02
omnios$ sudo date -u 011803142038.00
January 18, 2038 at 03:14:00 AM GMT
omnios$ uptime
03:14:11    up 5567 day(s), 7 min(s),  1 user,  load average: 0.24, 0.16, 0.07

Trying to set the date to before the epoch yields different results as well. On NetBSD and Linux, it'll simply fail, but on OmniOS, it will set the month, day, hour, minute, and second, but pin the year to 1970:

omnios$ sudo date -u 020100001969.00
February  1, 1970 at 12:00:00 AM GMT
omnios$ date
February  1, 1970 at 12:00:01 AM UTC
omnios$ sudo date -u 121308451901.52
December 13, 1970 at 08:45:52 AM GMT
omnios$ date
December 13, 1970 at 08:45:54 AM UTC

But so what is the largest date we can set our clock to on systems that use a 64-bit time_t? As noted above, we'd expect to be able to set it to epoch 2^63 - 1 = 9223372036854775807. Let's first display the date, then try to set it:

So this is interesting. Even though we were promised a 64-bit time_t, we can't set the time to 9223372036854775807. We seem to max out at 67768036191676799 in the year 2,147,485,547. While that seems initially arbitrary, you may notice that 2147485547 is 2^31 -1, and suddenly this makes sense: even though the time_t is 64-bit, the struct tm's tm_year is still 32-bit, and so the maximum value that can be represented is thus the last second of the year 2,147,485,547.

But what's the deal with FreeBSD? Somehow there the epoch of 67768036191676799 is invalid, but epoch 67767976233532799 represents what on the other platforms is 67768036191676799? If you do the math, you'll notice that the difference between these two epoch times is 1900 years -- so apparently FreeBSD bases it's tm_year not on the year 1900 (as the struct tm in <time.h> claims), but on the year 0? Since I couldn't figure this out, I filed a bug report. ¯\_(ツ)_/¯

When we try to set the date, we first observe that using NetBSD date(1) it's impossible to set a date beyond the year 9999 (another bug report), but, fun fact, that doesn't really matter because it turns out we can't go higher than the year 4147 anyway:

netbsd# date 414708200732.17
date: settimeofday: Invalid argument
netbsd# date 414708200732.16; date +%s
Sun Aug 20 07:32:16 UTC 4147
68719476736

The reason for that is that NetBSD has a hardcoded limit of 2^36 = 68719476736 on the tv_sec value it will accept when setting the time, because larger value made KUBSAN unhappy (code):

        /*
         * The time being set to an unreasonable value
         * will cause unreasonable system behaviour.
         */
        if (ts->tv_sec < 0 || ts->tv_sec > (1LL << 36))
                return EINVAL;

linux$ sudo date -s @8277292036
date: cannot set date: Invalid argument
Wed Apr 18 11:47:16 PM UTC 2232
linux$ sudo date -s '@8277292035'
Wed Apr 18 11:47:15 PM UTC 2232
linux$ sleep 10; date +%s
8277292045
linux$ sudo date -s @$(date +%s)
date: cannot set date: Invalid argument
Wed Apr 18 11:47:25 PM UTC 2232

The reason for this limitation is found in the source as to be such that it can accommodate a 30-year uptime before the counter wraps:

#define NSEC_PER_SEC    1000000000L
#define KTIME_MAX       ((s64)~((u64)1 << 63))
#define KTIME_SEC_MAX   (KTIME_MAX / NSEC_PER_SEC)

/*
 * Limits for settimeofday():
 *
 * To prevent setting the time close to the wraparound point time setting
 * is limited so a reasonable uptime can be accomodated. Uptime of 30 years
 * should be really sufficient, which means the cutoff is 2232. At that
 * point the cutoff is just a small part of the larger * problem.
 */
#define TIME_UPTIME_SEC_MAX    (30LL * 365 * 24 *3600)
#define TIME_SETTOD_SEC_MAX    (KTIME_SEC_MAX - TIME_UPTIME_SEC_MAX)

Interestingly, this means that on Linux, the signed 64-bit max time value (2^63 - 1 = 9223372036854775807) does not represent the seconds since the epoch, but counts the nanoseconds since the epoch, bringing Linux's theoretical max date (KTIME_SEC_MAX) to a mere 9223372036 epoch seconds or the date 2262-04-23T11:47:16 -- a fair bit away from the "22 times the estimated age of the universe" and a bit more in the realm of possibly becoming an actual problem.

Lastly, when trying to see what value FreeBSD lets us set the clock to, I found that the value differed between the two latest releases, but either had the unpleasant problem of leading to a spontaneous reboot of the system:

freebsd# date -u -f "%s" 49282253052249598
Fri Dec 31 23:59:58 UTC 1561694399
49282253052249598
freebsd# sleep 1; date; date +%s
Fri Dec 31 23:59:59 UTC 1561694399
49282253052249599
freebsd# sleep 1; date; date +%s
Sat Jan  1 00:00:00 UTC 1561694400
49282253052249600
[ system reboots ]

Note: I'm able to initially set the date to values larger than 49282253052249598, but about three seconds later, the system reboots. If I set the date to one second earlier, i.e., 49282253052249597, then the system will not reboot, even as the system time progresses beyond the next value. Aren't computers great?

Oh, and one more thing...

We all love how macOS is a UNIX -- one of only 6 operating systems currently registered (the others being AIX, EulerOS (a commercial Linux distribution made by Huawei), HP-UX, Xinuos (previously UnixWare, via the old AT&T Unix System Laboratories + Novell, SCO, Caldera, and UnXis), and z/OS). But Apple's Core Foundation framework does not use the Unix epoch as it's basis for time. Instead, it uses 2001-01-01T00:00:00 GMT as it's reference date:

So if you want to convert those timestamps into Unix epoch, you need to add 978307200.

As I said, time is an illusion, and Unix time doubly so. And while the Year 2038 Problem may not concern you (unless you happen to use OmniOS, I suppose), perhaps I was able to show you that there are plenty of other surprises lurking. And we haven't even touched upon the insanity that is timezones and Daylight Saving Time...

Signs of Triviality