
Timestamps done right - mollmerx
https://getkerf.wordpress.com/2016/01/19/timestamps-done-right/
======
philsnow
When you build that syntax into the language / tokenizer, you're making a
bunch of choices for the user.

As many have said elsewhere in the thread, what is `m` supposed to mean,
minute or month? If the smallest value you can represent with this special
syntax is 1s, you're excluding people who work with sub-second timestamps.
Unless, you want them to do like 1m0.0000415s or something like that, but now
you have the same problem as you started with (long series of digits being
hard to read for humans).

More problems: different cultures have different concepts of a "month"
(because they have different calendars). Also, the units that you want to use
depend heavily on the application. `1y` is different from `52w` is different
from `365d`, but all of those concepts are very similar to users. I expect
people to create bugs where these tokens are mixed. When you write `now + 1y`
do you mean "this time on this date, one year from now" or do you mean "this
instant plus 60 _60_ 24*365 seconds" (which is slightly greater/less, I forget
which way)? How do you communicate that to the user? If they can't control
which sense ("human" time vs "physical" time) the larger units represent, then
those people have to calculate the time offsets by hand.

Just pick nanos or micros since some Epoch (the Unix one is one reasonable
choice), use them everywhere internally, and let the user pretty-print them.
If you want to support high-energy timeseries where the time scale is smaller,
either make your nanos fractional or use femtos/attos/zeptos/plancks.

~~~
Jabbles
The ambiguity of "now + 1y" caused an outage for Azure:

[https://azure.microsoft.com/en-us/blog/summary-of-windows-
az...](https://azure.microsoft.com/en-us/blog/summary-of-windows-azure-
service-disruption-on-feb-29th-2012/)

------
ehartsuyker
> I mean, just try it in Java.

For ages JodaTime actually nailed it, and the Java 8 date API was based off
this.

> Not an add on type as in R or Python or Java.

Again, let's talk about the modern version of the language and not act like
prior screw ups are the end all for a language.

Also

> 2012.01.01 + 1m1d

How is that more clean than:

> new DateTime(2012, 1, 1).plusMonths(1).plusDays(1)

~~~
skywhopper
By my count it's over 30 characters less "clean". The Java syntax obscures
some meaning and requires a lot more boilerplate in favor of less magical (and
thus complex) syntax.

I agree that this sort of first-class datetime-type representation may not be
appropriate for every language, but myself, I find it refreshing and
brilliant, and I'd love to see more languages support this sort of syntax
instead of overloading strings or using complicated objects or APIs.

It's like comparing the power and ease of using regular expressions in Perl or
Ruby versus in Java or Python. In Perl and Ruby, regexes are built-in to the
syntax of the language itself. They're a truly first-class type, like strings
and integers are in all four languages, and like lists and
dicts/hashes/associative arrays are in Perl, Ruby, and Python.

I'd love to see datetime objects promoted to similar first-class native syntax
support in this way in more languages. It won't be appropriate everywhere, but
in the right language it'd be amazing.

~~~
stanmancan
How is plusMonths(1).plusDays(1) obscure in any way? You could show that to my
grandparents and they would be able to guess what it does. "+1m1d" on the
other hand they wouldn't have a clue.

Length, in either direction, does not correlate to "clean". Clarity and intent
does. Clean Coder ([http://www.amazon.ca/Clean-Code-Handbook-Software-
Craftsmans...](http://www.amazon.ca/Clean-Code-Handbook-Software-
Craftsmanship/dp/0132350882)) does a fantastic job talking about this, it's
worth picking up if you haven't read it in the past.

~~~
nickpsecurity
"Length, in either direction, does not correlate to "clean". Clarity and
intent does."

Exactly. So, the first example starts with a date in a way of representing
dates that will register immediately for even a lay person. The developer
intends to add time to that date. The example does this with an addition
operator then a value with letters representing recognizable units of time.
Matter of fact, this was so obvious that I knew what the author was doing
before I read the explanation. I'd probably do "min," "sec," "hr," etc to aid
intuition, though. Esp avoid confusion on months vs minutes for m.

Then, there's the other example. It appears to create an object. It then calls
a method on that clearly adds one month. It also calls a method of that
method, that object... idk that language so I don't really know the semantics
of what it's doing... to add 1 day.

One is definitely more clear and intuitive than the other. It also has the
rare property of being easier to type. Epic win over whatever the other thing
is. Not to say the other one was bad: still pretty clear. Just not as much as
a straight-forward expression.

~~~
ehartsuyker
In the former, typing '2012.01.01' implicitly creates a date object
(probably), and '1m1d' implicitly creates some sort of duration object
(probably) and uses an operator to combine them.

The Java was isn't really that different. Just more verbose, but again, I
don't mind trading a few key strokes for clarity.

~~~
nickpsecurity
What clarity? That's what I mean. The first example starts with a date then
adds values to it. Second does same thing. I use neither language nor do tjme
series but still knew tge intent. So all you're getting is extra keystrokes
with same clarity.

------
stygiansonic
This is just an aside, and not a knock on the article, but Java _does_ have a
built-in Timestamp type, [1] and has had it for some time.

Maybe it's not "first-class" (not sure what this means in context?), but it's
definitely there and not in a third-party JAR or anything.

However, it's bad for other reasons, the first being that it extends
java.util.Date (the Javadoc seems to admit this) and combined with the related
java.sql.Date (which also extends java.util.Date) makes for a very confusing
API.

For this reason, Oracle recommends just using the new Date APIs, [2] and
mapping a SQL TIMESTAMP to LocalDateTime.

1\.
[https://docs.oracle.com/javase/8/docs/api/java/sql/Timestamp...](https://docs.oracle.com/javase/8/docs/api/java/sql/Timestamp.html)

2\. [http://www.oracle.com/technetwork/articles/java/jf14-date-
ti...](http://www.oracle.com/technetwork/articles/java/jf14-date-
time-2125367.html)

~~~
alblue
The original Java date and time API was probably one of the worst APIs ever
invented, as I describe in my answer on StackOverflow:
[http://stackoverflow.com/a/1969651](http://stackoverflow.com/a/1969651)

However the new javax.time APIs (created by Stephen Colebourne) are excellent
and probably one of the best designed APIs. It's funny what twenty years
difference can make :)

~~~
stygiansonic
Totally concur. But date/time is a hard thing to get right!

However, getting it wrong the first time is different than getting wrong
again, as you pointed out with the Calendar type. 0 == JANUARY, but most other
things are indexed from 1...

Third time's the charm, I guess :) Getting the input from Stephen Colebourne
(of Joda-Time fame) was undoubtedly a key point in getting this done.

~~~
jandrese
Numbering months from 0 is obnoxiously common in these time APIs. This is one
of the few edge cases where IMHO we should index from 1, because people
frequently just use the numeric version for output and it is way too easy to
forget to add or subtract 1 from the month value. The memory lost by having an
empty 0 spot in the month name array is absolutely not a concern in any sane
project.

------
DiabloD3
Why are we not using unix timestamps (seconds since the epoch) for timestamps
on everything that doesn't require sub-second precision?

~~~
NyxWulf
They are difficult to read for humans. Without using a computer can you tell
me when this timestamp was taken? 1453383978

Ultimately that was the point of the article, but the tone of the article
about how they got it right and everyone else is wrong bugged me.

My approach has been to store everything in epoch form, do all of my
calculations and manipulations from there, then build tools that make
converting back to a human readable representation when and where it is
needed. I think the idea that you have a problem completely solved though just
prevents the search for any improvements.

~~~
JadeNB
Skimming the article doesn't make it clear whether they've addressed this,
but, if you want to add units of time of non-constant size, then you _cannot_
just use time-stamps. For example, `(today + 1 year) - today` is longer than
`(today + 2 years) - (today + 1 year)` [0], so that one can't think of them as
just `(today.unix + seconds_in_year).human` and `(today.unix +
2*seconds_in_year).human`.

[0] Assuming that today + 1 year is defined to be 2017-01-21, and today + 2
years is defined to be 2018-01-21; and, if not, then one faces other problems
with intuition.

~~~
lfowles
Ugh. Every time I see time code, I'm thankful I work with monotonic subsecond
times.

------
unabashedturtle
I work with timestamps A LOT. In fact, I often have to deal with streams of 5
million + /second. Even worse, for most of them, microsecond precision is
critical.

I'm convinced most languages do timestamps wrong. Specifically, they separate
out the "time" component from the "date" component.

What's a more common operation? Counting how many events happened in a span of
time? Or shifting every timestamp by 15 days?

Timestamps should generally be designed for extremely fast and lightweight
comparison, but keep enough information that a shift is doable. From my
experience, all you need is: a unix timestamp, a microsecond (or nanosecond)
offset, and the source timezone.

In this case, if you want to find elapsed time between two timestamps you
simply subtract the unix timestamps and offsets. Very CPU friendly and easily
vectorizable. You can do this even if the timestamps originated at different
timezones (since everything is UTC under the hood).

What if you want to shift the date? Or group by date? Turns out computing the
date on the fly is a very cheap operation. Easily can do hundreds of
millions/second on a single high-end server core.

an use whatever calendar system floats your boat.

Any timestamp system that relies on year/month/day semantics is rarely going
to be optimized for the most common operations users do with timestamps. Even
worse, for simple comparison you run into all the weird edge-cases that you
wouldn't have cared about if you stuck with a unix timestamp under the hood.

------
de_Selby
I'm surprised the author is so big on kerf but doesn't mention q/kdb+[1] at
all, which kerf took almost all these ideas from and which has a much bigger
user base.

1) [https://kx.com/](https://kx.com/)

~~~
acveilleux
This should enlighten you why:
[https://scottlocklin.wordpress.com/2015/12/15/an-
introductio...](https://scottlocklin.wordpress.com/2015/12/15/an-introduction-
to-kerf/)

Basically, he's big on Kerf because he's involved in the development. The
above blog post will tie it with Kx and other APLs.

[https://getkerf.wordpress.com/](https://getkerf.wordpress.com/) is the
official Kerf blog. And Kerf is not meant to be free...

~~~
nickpsecurity
It's _definitely_ an improvement in reader comprehension. I'll admit one of
the reason I didn't learn those languages is that they looked like gibberish.

------
falsedan
Doing timestamps right: use a library or class to abstract away the way the
data is stored from the intent of the actions you want to perform on them.

What the article sez: GET KERF IT'S THE BEST AND ONLY WAY TO SOLVE THIS

------
MikeTLive

      Numericaly sortable date [YYYY+][mm][dd].[HH][MM][SS],[NNNNNNNNN]
      date -u +%Y%m%d.%H%M%S,%N
      20160122.044145,052215000
    
      Absolute dates with nanos as seconds since epoch
      date -u +%s.%N
      1453437715.409682000
    
      Hyphenated,"chunks", with nanos and TZ offset.
      date -uIns
      2016-01-22T04:41:03,121501000+0000

------
lostcolony
This is interesting, and I can see that adding 1d10m is easier than adding 24
* 60 * 60000 + 10 * 60000 to a (millisecond) timestamp. Though I have to guess
m is minute, not millisecond or month.

But...the display and storage of a timezone is a datetime, and it doesn't
appear to have a timezone attached. Meaning I'd still rather just
store/retrieve/work with millis since epoch, since that avoids any ambiguity
about what timezone the timestamp takes place in. With just a datetime...I can
~assume~ it's GMT, but...is it? Millis since epoch are the same across all
timezones, there is no ambiguity; datetimes vary, and I have to make
assumptions, and make sure my library/driver/etc handles conversions
correctly.

------
smartmic
I wonder if a closed source language for data processing tasks can thrive on
the long run. The concept for timestamps is nice though. But I do not want to
spend time of learning proprietary languages.

~~~
mollmerx
I think that's a very interesting question, to which the answer probably
depends on how you define 'thrive'.

It seems unlikely to me that any language could become mainstream without
being open source. The expectation that a compiler or interpreter should have
its source available is only growing.

At the same time, kx's kdb+ is an example of a product that sits in a niche
and has generated significant revenue despite being closed source. kdb+ has
achieved this by primarily targeting financial services, which is a sector
that's less sensitive to closed source than most, and is able to spend money
on whichever product solves their problem.

If kerf manages to gain an edge over its competitors for a particular set of
problems then sure, it can thrive in the long run the way kdb+ has.

~~~
xixi77
It could change in the future, but there are a plenty of languages that got
pretty mainstream while closed source -- in terms of working with numbers and
data, I'd definitely call matlab and sas mainstream, stata and mathematica are
nearly mainstream, R of course is open source which is a huge reason for its
popularity, but it's a clone of closed-source s+ which was fairly popular
already. In the general-purpose world, c# and swift got most of their traction
while still closed source, etc. Surely there are more examples in other
domains... But yes, there is definitely a very welcome trend to open source
stuff.

------
kevinmgranger
This post doesn't load at all for me without javascript.

~~~
mikegerwitz
I had a related comment earlier this week:

[https://news.ycombinator.com/item?id=10930663](https://news.ycombinator.com/item?id=10930663)

------
jedisct1
Regarding timestamps:
[https://howtologatimestamp.info/](https://howtologatimestamp.info/)

------
DannoHung
How's your treatment of timezones?

------
tempodox

      date -u +'%FT%H:%M:%S %Z'

~~~
Zash

        date -u +%FT%TZ
    

:)

~~~
zimpenfish

        date -uIs
    

Shorter but sadly not RFC3339 compliant although it is ISO8601 compliant.

