
Open Letter to Calendar Developers about HTTP - bpierre
http://lists.w3.org/Archives/Public/www-archive/2013Aug/0021.html
======
lazyjones
What do we learn from this? If you provide a public-facing API or service for
direct use by applications over the net, use a unique host name / DNS entry
for it so you can point it elsewhere or delete it at leisure. DNS entries are
cheap, millions of HTTP hits might cost a bit more.

~~~
shabble
For even less fun, there have been numerous incidents of consumer
accesspoints/routers hardcoding NTP servers by IP address[1][2].

[1] [http://pages.cs.wisc.edu/~plonka/netgear-
sntp/](http://pages.cs.wisc.edu/~plonka/netgear-sntp/)

[2]
[http://web.archive.org/web/20060408150213/http://people.free...](http://web.archive.org/web/20060408150213/http://people.freebsd.org/~phk/dlink/)

------
smackfu
I would guess the problem is that most sites are going to serve up a 404 in
this situation, and handling 404s is a bit trickier, since they might be
transient (like a proxy gets in the way, or the hosting service has temporary
issues). So they just don't handle it, since polling a server and getting a
404 is going to work well enough for most use cases. Maybe put a little
exclamation point next to the calendar or something, but still poll it.

And then they handle a 410 the same as a 404, even though a 410 has very
little chance of being transient. It's just rare enough that they don't even
realize they don't support it properly.

~~~
Wilya
I'd wager 99% of codes have a big if(200) { // Do stuff } else { // Try again
later }

Actually, a bunch of them probably don't even check for response code.

Edit: If dataaccessd is really the iOS native CALDAV integration, that's a bit
annoying. Of all people, they should be able to handle that correctly.

~~~
pyre

      if(200) { // Do stuff } else { // Try again later }
    

That's like an "idiot-filter" interview question. "Is the following code
valid?"

~~~
Groxx
Seems like it should be paired with a "Is the following code useful?" about
code which specifically handles every error code listed here
[http://en.wikipedia.org/wiki/List_of_HTTP_status_codes](http://en.wikipedia.org/wiki/List_of_HTTP_status_codes)
and contains code to handle unlisted ones.

Then "where do you draw the line?". Sometimes if(200){}else{} is plenty.

~~~
pyre
I was superficially commenting on the fact that '//' comments everything to
the end of the line[1], but as @jimbobimbo pointed out there are multiple
layers here, and it would be a good lead-in to a conversation in an interview.

[1]: It just struck me as one of those "low bar" technical interview
questions.

~~~
Groxx
haha, that kinda makes me want to make a language that prevents comments from
breaking scopes, so "}" would implicitly also be a comment terminator :)

~~~
pyre
Off the top of my head, Vim has a folding mode where "{{{" and "}}}" are used
to mark the start and end of a fold. So you can see this on the code:

    
    
      // ==== Section 1: Code ==== {{{
    
      BLAH
    
      // ========================= }}}
    

Granted, since this is Vim-specific, you usually only see it used in Vim
plugins, because you're unlikely to have someone editing with another editor.

------
kogir
Am I the only person who after about 3 months would have started returning
random and numerous events? Then clients would voluntarily unsubscribe.

~~~
biot
Nope. I'd be hard pressed to not do a little iCal trolling. I'm sure the moon
would get destroyed by the Death Star at least a few times per year. Not to
mention a new phase of the moon: Space station.

------
jstanley
I have exactly the same problem with a project I run. I generate iCalendar
versions of the timetables for each course at my university and let students
download them. The timetables are static and do not change, yet I still get
hundreds of hits every day from dataaccessd, even in the middle of summer when
nobody has any classes.

EDIT: [http://notlate.co.uk/](http://notlate.co.uk/)

EDIT2: Not that it's enough load to be noticeable, but it's a little wasteful
on the part of dataaccessd IMO.

------
eksith
I have a feeling many of these are from rubbish applications that have mostly
been abandoned by their developers but still remain installed and running in
the background for a lot of users.

~~~
codeka
Except dataaccessd is how iOS's CalDAV sync process identifies itself, so I'd
say it's not exactly a "rubbish" application that's been abandoned...

------
MBCook
Fetching stuff over HTTP can be incredibly stupid. I there is a security list
out there that some little firewall company decided to have their devices
automatically download. The list doesn't change that often, but every single
device that company made likes to check every 10 minutes or so. Nothing can
stop them, they follow Wilya's algorithm [1].

[1]
[https://news.ycombinator.com/item?id=6268919](https://news.ycombinator.com/item?id=6268919)

Side note: it's sad to see that Apple is a huge perpetrator in this one. I
hope they fix that bug.

------
charlesism
I'd never even noticed there was such a thing as a 410 until this. And it's
not surprising - what incentive is there for a team in the real world with a
shipping product to honor it?

There's a pretty obvious incentive to ignore it: it takes developer time to
implement, and the result of the 'feature' is that your customer may get
permanently locked out of a net resource. What happens if a website admin
accidentally puts up a 410, or - worse yet - if a previous owner of the domain
once put up a 410 for a page with the same URI.

I suppose there's an RFC for the code somewhere, but it seems like a bad idea.
I pity the tech support agent who has to explain why his/her customer's
browser refuses to visit foo.com/bar when everyone else can view it and
there's no longer any way of knowing that once, for an afternoon in 1997 the
page threw a 410!

------
evmar
A long time ago I published an XML file on my website for some ill-fated
Google attempt at providing extensible search results. I took the file down
some time later, but the Google crawler continued to hit the URL.

Amusingly, I asked around within the company to try to find who was
responsible for the crawler repeatedly fetching the 404 but ultimately
couldn't figure it out. (It seemed like the decommissioned service had maybe
registered the URL with some still-running automated refetching service?) It
eventually stopped on its own.

------
eli
Google will continue to request 410 resources for quite a while too. As far as
I can tell it treats them the same as 404.

------
ezxs
This is another case for rate limiting

------
rav
This is probably a case where an exponential backoff would be preferable. When
the client is faced with a fetch error the first time, it should wait and try
again in e.g. 10 minutes, and double the wait for each repeated error. This
way, after a week of downtime at the source, each client would send less than
four requests for the resource per month.

~~~
ZoFreX
No, this is a case where obeying the HTTP specification would be preferable.
410 GONE means: Gone. Don't try again.

~~~
Millennium
Agreed. Incremental backoff makes sense for the 5xx-class errors, where the
error is known to be server-side. But 404 is the only 4xx-class error that it
makes any sense for: in any other case, it's pretty certain that your side is
doing something wrong. Even when you're not -i.e. if you're absolutely certain
that this is the right URL for what you want- a 404 should still trigger some
kind of notification so that a person can look into it.

Using incremental backoff for 410, in particular, is to miss the point
entirely. 410 very explicitly means that you should not try again. The fact
that it's so rare only lowers the chance that it's being used incorrectly:
someone savvy enough to set up a 410 knows enough to mean it.

That said, I'm starting to think that if you're going to implement 410 on your
site, you should also start logging activity. When a host keeps looking,
either incrementally or otherwise, start refusing connections from that host
at the firewall level. A program that sets 4xx-class errors to fail silently
probably isn't doing anything about refused connections, so an IP-ban should
wake the developers up. Even if it doesn't, you won't have erroneous requests
clogging your server logs, so you still benefit.

~~~
vidarh
> The fact that it's so rare only lowers the chance that it's being used
> incorrectly: someone savvy enough to set up a 410 knows enough to mean it.

I don't see how your conclusion follows from your premise. I fairly often come
across cases where obscure options are mainly used by those who _don 't_
understand them, and in the absence of data I wouldn't assume that the same
couldn't be the case for HTTP responses.

Exponential backoff is a quite reasonable compromise for those kind of
situations.

