
Google's CardDAV server isn't standards compliant (2014) - bariumbitmap
https://evertpot.com/google-carddav-issues/
======
jacquesm
I can confirm just about all of this because of a recent job in this space.
Whatever you do, do not do _two way_ sync with Google's CardDAV servers or you
will end up with a lot of lost data.

I wonder if they should even be allowed to call it CardDAV since you'd expect
it to actually adhere to the standard.

Google is large enough not to care about this though, as long as it works with
their address books on Android phones it is probably good enough for them.

Note that this post seems to be from 2014 and the situation isn't much better
today!

~~~
leephillips
"Google is large enough not to care about this"

They also operate "SMTP servers" that violate the applicable RFPs, have done
so for many years, and are not concerned about the potential damage that can
result: [https://lee-phillips.org/gmailRewriting/](https://lee-
phillips.org/gmailRewriting/)

~~~
ebikelaw
You don’t seem to understand the standards you are complaining about. You are
using mail submission, not relaying. RFC5321 specifically says that it does
not cover submission, and defers to RFC4409 (3.6.1) which specifically allows
From header rewriting (8.8).

If you’re going to complain about the standards-compliance of other people's
services it’s important to understand the standard first.

~~~
leephillips
I discuss exactly this in the article. I even referred to sec. 8.8 (of the
current 6409; 4409 is superseded (but 8.8 is the same)).

~~~
ebikelaw
Yeah but you don’t seem to really grok it.

“ The MSA MAY rewrite local parts and/or domains in the SMTP envelope and,
optionally, in address fields of the header, according to local policy.“

Generally MSAs are permitted to do almost anything. Your reading of the RFC
which requires MSAs to merely reject instead of modifying messages is not
backed by any of the language in the RFC. Essentially, you seem to
misunderstand “may” “must” and “should” as used in these documents.

~~~
leephillips
You're extracting one sentence out of several paragraphs. The following
paragraph restricts the permissible rewriting, and Google is going beyond what
is permitted. If your interpretation differs from mine, that's fine. I don't
think you need to support your view by selective quotations and accusing me of
not being aware of what I explicitly dealt with.

The meaning of the whole section is clear, and I stand behind what I wrote.

~~~
ebikelaw
Again, the only thing that seems clear here is that you don't know how to read
an RFC. The only "must not" in 8.8 is the prohibition of changing the
destination local parts. "must not" appears nowhere else in section 8, and 8
is not strongly prescriptive as it "describes a number of such modifications
that are often considered useful". The described modifications are not thought
to be exhaustive. Moreover, 5321 describes MSA as "arrangements are private
and fall outside the scope of this specification, they are not described
here." 5321 does not control MSA and 4409/6409 do not defer to 5321 on any
point. 4409/6490 do _refer_ to 5321, of course.

There are only 5 MUST NOTs in 6409 and there are zero SHOULD NOTs. The
standard permits the MSA to do pretty much anything it wants to do. I think
you should begin by reading 2119 "Key words for use in RFCs to Indicate
Requirement Levels".

~~~
leephillips
'The only "must not" in 8.8 is the prohibition of changing the destination
local parts.'

That's not even true. If anyone cares about this they should just read the
texts. I'm certainly not going to argue this any further. We're looking at the
same page and seeing different words.

It actually says this:

"However, only addresses, local-parts, or domains that match specific local
MSA configuration settings should be altered"

In other words, the opposite of what you claim. You can keep banging on this
if you find it useful, but I'm not going to read any more of your comments.

~~~
frankchn
According to RFC2119, which defines these terms, "should" and "should not" are
recommendations with the caveat that individuals not following said
recommendations need to understand the full implications what they are doing.

On the other hand, the words "must" and "must not" specify absolute
requirements and prohibitions of the standard, respectively.

~~~
shkkmo
I think the confusion comes from parsing the poorly worded "only X should be
altered" which doesn't really make it clear how the "should" applies.

The clear wording would be "... should only alter X" which makes it clear that
altering things other than X needs be be done with full understanding of the
implications.

------
nolok
I think to get past such issues with providers like this, you first need to
understand what it is and what it is not.

Forget the brochure, the ads and the nice tutorial, those are NOT
CardDAV/SMTP/IMAP/... servers. What they are instead are ways to access their
own proprietary special thing using standard protocols that your software and
tools already supports, if you can't use the dedicated api.

It applies to pretty much every system for which Google has made their own
thing, with their own api (contact api, mail api, ...), and then offer an
alternate way to access their system by standard protocols.

And what this means is that the ruling system is the underlying one, the one
which can be accessed in several ways, including that standard protocol. If
their contact system discards the field "foobar" because it doesn't want it,
then it doesn't matter if CardDAV says it should be kept.

Sure Google should totally make this clearer, and sure as an user I'm not a
big fan, and yes I totally understand and agree with the "but as a result they
break the standard, lose data and cause unexpected behavior and that's
terrible", but from a purely engineering perspective it also makes total sense
as to why those things happen this way, and is actually rather predictable.

You're feeding data to a special system that just so happens to have a
CardDAV/SMTP/IMAP/... endpoint plugged on top of it. Of course that endpoint
doesn't rule how the special system will behave.

And frankly any one who has had to build/support that kind of "api on top of
an api" for some time has faced that sort of situation before.

In fact you don't need to go that far; I mean how many of your web
applications take PUT or DELETE requests ? The HTTP protocol certainly says it
should work and the http server you're using supports it. And let's not talk
about all those sites that give a full page answer to a HEAD request.

PS: again, I am most definitely NOT saying I like that it works that way, I'm
just saying ... "well no big surprise there".

~~~
untitaker_
The distinction between being a "real CardDAV server" and being a server that
exposes CardDAV is undefined and irrelevant, possibly non-existent. There is a
spec, and if you implement it, you have a server that implements that spec.
That's it.

You can of course deviate from certain parts of the spec... dropping unknown
fields as you mentioned might be a reasonable tradeoff. But then that
deviation has to be documented. It isn't.

You're completely leaving out the other issues that the author describes:
Timing out TCP sessions, discarding UIDs. Both things are not reasonable
design choices in any world. And then the response to bug reports.

~~~
cortesoft
I am not sure what you mean by 'the deviation has to be documented.'

Has to be documented in order to do what? To claim to be standards compliant?
Do they ever make that claim?

------
Tepix
Quote from the DAVdroid project:

Address books, calendars and task list contain private and sensitive data by
their nature. We believe that individuals and companies should be able to be
in full control of their own data. This means the freedom to choose where and
how the data are stored, and freedom in choice of software. CalDAV/CardDAV are
open protocols and thus can be used with various server and client software.

Setting up your own CalDAV + CardDAV server such as Radicale is easy. Chances
are if you are reading HN, you can do it yourself. Do it.

~~~
Avamander
Can you recommend any good webGUIs when running my own CalDAV server? I'm
struggling to find one and I really need one.

~~~
1wilkens
I've found infcloud [0] to be decent.

It's a bit messy to set up but the config file is extensively documented and
you generally only need to touch it in the initial setup.

[0]: [https://www.inf-it.com/open-source/clients/infcloud/](https://www.inf-
it.com/open-source/clients/infcloud/)

------
smhg
Slightly related: try to sync an external calendar feed in iCalendar format
(also used in CalDAV) with Google Calendar.

Google Calendar will use random intervals to synchronize. Often taking more
than 24 hours... which is, in most use cases, quite useless.

Sync in the same way with Facebook's event feed (same format) and changes
appear within minutes.

This has been the case for years. Google Groups are full of questions about
this.

I don't want to demand anything from a free product (Google Calendar is very
useful in many ways). But somehow some feedback about 'shortcomings' like
these would be hugely appreciated.

------
gumby
This (good, but sad article) is actually about "How" it's different, not
"Why".

Why don't they care at all about standards? I don't know; they talk a good
game. At least some companies who violate standards are open about it --
Google trumpets it but often simply pays lip service.

------
yosamino
> this isn’t just a 400 Bad Request [...] Any (valid) vCard that the CardDAV
> server does not understand, will result in our HTTP requests timing out

That's just terrible. Why would such a thing be implemented ?

The only valid explaination I can think of, is to deter abuse. But even for
that there are better solutions than this.

~~~
djsumdog
Never attribute to security deign what can more easily be attributed to not
giving a shit.

~~~
skarap
Exactly. Asynchronous programming + promises + some unhandled failure case
somewhere and you end up with timeouts instead of error responses.

------
captn3m0
Relevant: The author of the post worked on Sabre/dav, an open source
CardDAV/CalDAV server (linked at bottom of the post). Unfortunately, it is no
longer under development: [http://sabre.io/blog/2017/development-on-
hold/](http://sabre.io/blog/2017/development-on-hold/)

~~~
treve
A bunch of people have since picked up development and are working on new
releases.

------
drawkbox
Google doesn't like existing standards, and have generally been bad stewards
of their power in that place, they like to make new ones though that they
expect everyone to jump on. I am wondering if the engineers and product
developers are still in charge at Google?

------
splitrocket
Ahh, the good old days of “embrace and extend” are back.

Delightful.

------
nik736
In the CalDAV space a lot of people do whatever they want, so there is a
standard but most people are not compliant with it.

I would imagine the same is true for CardDAV, not only at Google.

------
privateSFacct
My impression is that

[https://developers.google.com/people/v1/libraries](https://developers.google.com/people/v1/libraries)

Is the current recommended solution for this sort of thing.

Libraries look to be available for

GO JAVA JAVASCRIPT .NET NODE.JS OBJ-C PHP PYTHON RUBY

and you can roll your own using HTTP.

CardDAV is not well maintained, and even the contacts API using GData is
superseded with the above.

With over 1 billion active users - my guess is engineering efforts are focused
on serving that customer base, especially the growth in paying customers
through google apps. Not sure where strict compliance with CardDAV would fall
in the analysis.

A note that actually using some of these API's through their client libraries
isn't totally the nightmare described - though my experience is very limited.

------
thrillgore
I get the sense that Google cared about open protocols, and then Android
changed that, to simply being another mechanism to lock users into their
walled garden.

~~~
djsumdog
I don't think you can pin the change on Android. It's a slow progression at
Google from the days of the unofficial "don't be evil." Even their official
standards have been awful. IMAP was, and still is, hopelessly broken on gmail.
Where GTalk once had XMPP with federation, they silently removed federation
and XMPP entirely. I didn't realize how bad their Card/Caldav implementations
where because I switched to Radicalie back in 2013.

There just isn't any money is letting users access and control their own data
in a standard way.

------
torarnv
Tangent: What are people's experience with Google's CalDAV support? Is it just
as bad?

~~~
untitaker_
It isn't. CalDAV is a separate team. The support is quite good. The deviations
from the standard are documented.

------
forgotmypw
Embrace, Extend, Extinguish.

