
Proposal: Monotonic Elapsed Time Measurements in Go - Zikes
https://github.com/golang/proposal/blob/master/design/12914-monotonic.md
======
jkn
This a pretty magical solution.

\- Backward compatible. No API change.

\- Naive, idiomatic and wrong use of the API automatically becomes naive,
idiomatic and correct: Existing code that was using wall-clock time will now
be using monotonic time when it should, and only when it should [1]

\- No change to the memory footprint on 64-bit systems

\- No change to the range of representable dates [2]

"No API change" means if for some reason it turns out to be a bad idea, they
can still revert it and stay backward compatible (though of course, the
documentation will mention how monotonic times are used to calculate better
time differences and that would no longer be true after a revert).

Very impressed with the extensive survey of existing code which didn't find a
single case where the change would cause an issue.

[1] Except when a user got out of their way to calculate a time difference in
a non-idiomatic way.

[2] The range is only restricted when monotonic time information is present,
which cannot sensibly be the case outside the restricted range.

------
knodi
Well written and an excellent in-depth break down of the problem and
solution(s). Would have expect nothing but excellence from Russ Cox.

------
gbrown_
First off, yay easy access to monotonic time! This is good.

Now whilst the author acknowledges Go can't rely on systems to smear leap
seconds it was hopelessly naive to think they could in the first place.

    
    
        I hoped that the trend toward reliable, reset-free computer clocks would
        continue and that Go programs could safely use the system wall clock to measure
        elapsed times. I was wrong. Although Akamai, Amazon, and Microsoft use leap
        smears now too, many systems still implement leap seconds by clock reset.
    

I'm not trying to make a dig at the author here it's just a topic that grinds
my gears so to speak. I hope others don't pick up this idea that you can make
assumptions about how the system clock will behave.

------
twic
> Go 1 compatibility keeps us from changing any of the types in the APIs
> mentioned above.

But apparently allows them to change the _semantics_ of those types. I am
somewhat dubious about this. The point of the substantial appendix is to
demonstrate that the change in semantics is safe in existing code, but it only
looks at a subset of existing code. It's still possible that someone out there
has made assumptions which will now be invalid, and their code will be broken
by this change.

~~~
kevhito
> someone out there has made assumptions which will now be invalid

I was thinking about this as well, but I think the key is that they guarantee
only the semantics that are _documented_. If you make assumptions beyond the
documentation, then no, of course they can't guarantee compatibility,
otherwise it would essentially forbid any changes whatsoever. Still, it is a
good thing the Go team takes the extra step to ensure that even changes that
fall inside their compatibility promise are _still_ vetted against a vast body
of real-world code, and to check that common (non-documented) assumptions
won't be violated if they can help it.

------
justinsb
I had hoped the underlying monotonic clock would have been exposed directly.

~~~
bcgraham
I was persuaded by the proposal. Do you still prefer directly exposing the
monotonic clock? If so, why?

~~~
justinsb
Simply that the proposal is complex and involves subtleties of behaviour, and
I worry that there may be further unexpected consequences found later (like
the equality surprises). I do think that the proposal nicely works around the
problem of back-compatibility for the APIs which were using a time.Time
instead of a time.Duration.

But: I expect that many of the uses of time.Now to measure elapsed time were
accompanied by a comment along the lines of "// TODO: use a monotonic clock
when go exposes it".

I see little harm in exposing `time.MonotonicNow` or similar, in addition to
these changes, particularly as this is what the original github issue
requested:
[https://github.com/golang/go/issues/12914](https://github.com/golang/go/issues/12914)

This could also be done at very low risk in the `golang.org/x` packages, and
the hypothesis that the go community does not understand monotonic clocks
could be tested, by looking at the rate of adoption.

~~~
rspeer
> But: I expect that many of the uses of time.Now to measure elapsed time were
> accompanied by a comment along the lines of "// TODO: use a monotonic clock
> when go exposes it".

You may be too optimistic. I think most developers haven't heard about
monotonic clocks.

Cox describes this in his proposal: "Providing two APIs makes it very easy
(and likely) for a developer to write programs that fail only rarely, but
typically all at the same time."

------
twic
> The wall clock is for telling time; the monotonic clock is for measuring
> time.

Spot on! But it's not as if they're unrelated - there is a deterministic
formula for converting monotonic clock time to wall clock time, taking into
account everything from leap seconds, via the calendar, to leap years.

The problem is that today's systems conflate the two kinds of time. The
proposed solution here is to treat them as completely separate, which seems to
me almost as great a mistake as treating them as the same.

What i'd like to see is:

1\. Computers keep time using a monotonic clock, based on TAI [1].

2\. When the local clock drifts from a master clock, it is corrected by
slewing, ie making the local clock run slightly faster or slower for a while
until it is back in sync with the master clock (as NTPD does now).

3\. Computers keep track of the times of leap seconds, obtained via NTP or the
tz file or whatever.

4\. To convert from a TAI time to a wall clock time, the time is first
converted to UTC by applying the leap seconds, and then converted to a human-
readable date using the calendar.

Essentially, i want to move the handling of leap seconds from the clock to the
formatting. If you can do this, the whole question of the impact of leap
seconds on time measurement goes away.

It's still not perfect, because although the clock is monotonic, it doesn't
always run at the most accurate possible rate: during slewing, it is
deliberately deviated from this. You could solve this with another layer of
indirection: run the clock at the best estimate of the true rate, and then
keep track of drift over time, so that when converting to a wall clock time,
you first apply drift correction, then leap seconds, then the calendar.
Essentially, you're formally recognising the separate existence of local and
global clocks.

You could potentially even change the drift correction retrospectively, so
that a given local time would map to different wall clock times at different
computation times. That starts to get pretty weird, though.

[1]
[https://en.wikipedia.org/wiki/International_Atomic_Time](https://en.wikipedia.org/wiki/International_Atomic_Time)

~~~
kevhito
> there is a deterministic formula for converting monotonic clock time to wall
> clock time

I think "deterministic" is not really right here, since you can only convert
monotonic times in the past, not the future, and only after you have been
given the (completely arbitrary) leap second data. If you call that
"deterministic", then what isn't?

> The proposed solution here is to treat them as completely separate

Did you read the proposal? Because the Go folks are essentially trying to
present a single, unified API for both, whereas lots of comments here on HN
are suggesting the separate-API approach instead. Also, the Go folks
specifically call out Google's efforts to eliminate the leap second problem
from their systems and how/why it hasn't caught on outside of the big players.
Can you compare your proposal to Google's time handling solution? Does it
suffer the same problems mentioned in the the post?

~~~
twic
> I think "deterministic" is not really right here, since you can only convert
> monotonic times in the past, not the future, and only after you have been
> given the (completely arbitrary) leap second data. If you call that
> "deterministic", then what isn't?

You're right that the "deterministic" formula doesn't extend all the way into
the future. If you believe Wikipedia [1], then leap seconds are added at two
points in the year, and we've always had about six months' warning, so there's
a window of about six to twelve months ahead where all the leap seconds are
known. I don't think it's useful to get into what "deterministic" means, but
you're quite right that you can't reliably convert monotonic times to human
times, or vice versa, beyond that window. I'm not sure how much that matters.

> Did you read the proposal?

You're not supposed to ask that! But yes, i did.

> Because the Go folks are essentially trying to present a single, unified API
> for both, whereas lots of comments here on HN are suggesting the separate-
> API approach instead.

No, they are presenting a single type which conceals both kinds of time. This
will lead to surprising behaviour if you ever rely on both kinds. For example:

[https://news.ycombinator.com/item?id=13574235](https://news.ycombinator.com/item?id=13574235)

[1]
[https://en.wikipedia.org/wiki/Leap_second#Insertion_of_leap_...](https://en.wikipedia.org/wiki/Leap_second#Insertion_of_leap_seconds)

------
dolzenko
The only downside would be time.Time structures grow by 8 bytes, but this is
really minor because if you're storing times you're likely storing timestamps
and not time.Time structures.

~~~
leaveyou
Actually, it only grows with 4 bytes on the 32 bit systems (no growth on the
64 bit). It's right there in the doc:

>On 64-bit systems, there is a 32-bit padding gap between nsec and loc in the
current representation, which the new representation fills, keeping the
overall struct size at 24 bytes. On 32-bit systems, there is no such gap, and
the overall struct size grows from 16 to 20 bytes.

------
ex3ndr
Why not to learn from java (android) and make simple function like
"time.UptimeMillis" that will return elapsed time from system boot?

~~~
bradfitz
The linked document addresses this.

