
How Twitter Killed My Fish - dnewms
http://www.coldweatheraquaponics.com/2014/07/10/how-twitter-killed-my-fish/
======
mikeash
More accurate title would be: How I Killed My Fish By Mistaking Twitter For A
Reliable Messaging Platform.

------
ColdWeatherAqua
Hi everyone. Author here. Gotta say I loved all the comments. My blog
followers are all so nice. It feels good to get some real criticism for once
:)

Relying on Twitter was a truly bonehead move, to be sure. Truly a "bang head
here" kind of moment.

However, if trusting Twitter was the only "poor engineering decision" I had
made I'd be a very happy man. It was just one of dozens of bonehead mistakes I
made in building and operating my first aquaponics system.

Aquaponics is one of those things where the path of least resistance is
definitely following a set of plans that are known to work. You can find one
of those on my site, and some other places too. But I've never one to follow
the path of least resistance (eg. Screw the round wheel - I'm making mine with
fractals).

In case anyone's interested, I'm a former electrical engineer who used this as
my first attempt at something with Arduino. More of a power-systems and
lighting guy, software development was and still is a foreign land to me.

I could have tested and researched the reliability of different notification
platforms, but at the time I was building it I was already staying up late
fixing plumbing problems and trying to get those d*&$ one-wire sensors to work
consistently.

Thankfully I eventually got my plumbing and sensors working, and bungled along
into into Zapier which has been absolutely bulletproof.

I think the truly useful lesson here is that if you ever meet anyone in
aquaculture who can tell a good story at a party, plop yourself down and ask
them if they've ever killed fish in any bizarre and interesting ways. Don't
get up for drinks either because you won't want to miss anything. You'll be
there all night.

------
austinz
From the Twitter terms of service:

"The Twitter Entities make no warranty and disclaim all responsibility and
liability for: (i) the completeness, accuracy, availability, timeliness,
security or reliability of the Services or any Content; (ii) any harm to your
computer system, loss of data, or other harm that results from your access to
or use of the Services or any Content; (iii) the deletion of, or the failure
to store or to transmit, any Content and other communications maintained by
the Services; and (iv) whether the Services will meet your requirements or be
available on an uninterrupted, secure, or error-free basis."

Clearly it was never meant as a reliable communications channel for any sort
of M2M application where failure could result in significant economic loss or
a threat to human health or life.

~~~
Houshalter
That just means you can't sue them. I'm sure any service can think of has a
similar warning in their ToS. Did he have any reason to believe Twitter was
more likely to fail than whatever alternative?

~~~
austinz
> Did he have any reason to believe Twitter was more likely to fail

Whether or not Twitter is more or less reliable than any similar alternative
doesn't really matter that much, especially if none of the alternatives offer
anything more than best-effort service with no guarantee of reliability.

The fact that Twitter was (at least superficially) suitable for this sort of
application was probably not intended by Twitter's developers, nor was the
existence of the pathological corner case that ruined OP's fish tank anything
more than a unfortunate coincidence.

It absolutely sucks that OP's fish died, but if Twitter explicitly disavows
any sort of loss because of the possibility that their service might change or
fail, it's doesn't seem reasonable to just ignore that and assume otherwise.
Especially for an application where it doesn't matter if the service is up
_most_ of the time if it ends up failing during the brief period where it's
needed the most. And if you are aware of the limitations of some aspect of
your system, it then becomes possible to design redundancy into the other
parts to compensate. I hope OP's next fishtank adventure keeps this in mind.

------
allegory
And that is a fine example of why you don't rely on such a Rube Goldberg
machine for stuff like this...

A poor engineering decision killed the fish.

I see all sorts of crazy contraptions people have rigged up over the last 5
years since the appearance of the 'maker movement'. Surprised that fish are
the only victim so far!

Sure you can wire up your coffee machine to listen for a tweet and have coffee
ready for when you get in... then when you get there your flat is on fire...

Excuse the cynicism but as a professional engineer it scares me how
dangerously and poorly thought out some of this sort of stuff is.

~~~
wpietri
The number of professional engineers we have is vastly too small to make good
use of the enormous power that computers have handed us in the last 20 years.

I see our options as: a) be condescending dicks every time a novice makes a
novice mistake, b) leave enormous amounts of technical value untapped until
professionals can get around to it, or c) use our professional engineering
smarts to build tools that amateur engineers can succeed with. I favor C.

~~~
mikeash
There are really two versions of A:

A1) Be assholes.

A2) Point out mistakes to share the benefit of your experience.

The problem is that people have a real hard time distinguishing between A1 and
A2, and this drives a lot of people to say that we shouldn't even try for A2,
and just try to build stuff so people can learn from our experience without
the troublesome burden of communicating with us.

~~~
wpietri
Good point. I think A2 can be distinguished from A1 if the writer is willing
to put in a little work. Some of the best lessons I learned were from people
saying, "Oh, yeah, I made a mistake like that once. Here's how I think about
it now. And let me tell you the story of an even bigger mistake I made."

But that sort of compassionate oral tradition only takes us so far. The number
of people who will ever approach the level of a professional engineer is
small. I think if we want to squeeze the full value out of digital technology,
it's going to have to be approachable to the non-expert. Being compassionate
in discussion can help there, but I think it's even more important to be
compassionate in _construction_.

------
rbinv
This is absolutely ridiculous. Why on earth would you rely on something like
Twitter to get critical messages delivered?

~~~
wpietri
Because you are somebody whose expertise is aquaculture, not software
development? Because Twitter is the thing you use? Because the docs weren't
clear? Because the APIs weren't really thought through for use cases like
this?

~~~
kelnos
If the guy put together an Arduino with custom software to control his
aquaponics system and interface with services on the internet, I would venture
to guess that he has _some_ expertise in software development. Enough not to
make such a terrible mistake.

------
Zigurd
More like how not having a heartbeat message that confirms Twitter-based
notifications are working killed the fish. And why Twitter? Seems like any of
email, IM, or SMS would be a better choice.

This is like not starting your emergency generators on a regular schedule and
then being surprised that the fuel filter clogged.

~~~
wpietri
This is one of those comments that relies on hindsight to sound smart, so it's
frustrating to me. The author surely can also think up a solution now that he
knows Twitter can't be trusted. In fact, as he explains in the comments, he
switched to SMS months ago.

~~~
mikeash
Realizing that Twitter is not reliable is not hindsight! It is blatantly
obvious! In terms of reliability, it's one step above putting a message in a
bottle and tossing it in the ocean. If you didn't already know this then
you're not paying attention.

~~~
wpietri
Here I take it "blatantly obvious" is code for "thing I know" or, less
charitably, "thing I can claim to know so I can sound superior, now that it's
obvious in hindsight". If you want to persuade me that it's really obvious,
that only a fool would make this mistake, you'd have to show me where in the
Twitter developer docs they say, "Hey, this is unreliable, so don't think
we're going to actually deliver your messages". And then that a reasonable
person would have had to see that page, read it, and ignore it before
proceeding.

Half of people are below average. Everybody is ignorant about most things, and
they're always learning. I have no reason to think that this guy didn't make a
reasonable novice mistake. I'd rather encourage novices to try things. Telling
them after the fact, "Fool, you should have known!" only discourages them and
other novices.

Personally, I appreciate him producing an interesting article that will help
other novices avoid his mistake.

~~~
mikeash
Twitter's terms of service are located here:
[https://twitter.com/tos](https://twitter.com/tos)

Two choice bits:

"We may suspend or terminate your accounts or cease providing you with all or
part of the Services at any time for any reason...."

"The Twitter Entities make no warranty and disclaim all responsibility and
liability for: (i) the completeness, accuracy, _availability_ , timeliness,
security or _reliability_ of the Services or any Content...."

Emphasis added.

Now I guess you'll say that a reasonable person won't read the TOS anyway so
it doesn't count. And I say, that's all well and good when you're tweeting
about last night's dinner, but when you're building something where you
actually care about the consequences when it fails, you have to actually
perform some form of research if you want reliability.

I'm rather astonished that you're still acting like Twitter's unreliability is
not only non-obvious, but something I wasn't aware of until I read this post,
but am pretending that I knew before so I could act superior. Do you just
assume that every random internet service out there is reliable until proven
otherwise? Because that's the opposite of what's correct. Everything is
completely unreliable until proven otherwise. This is, yes, obvious if you've
been paying attention to the internet for any reasonable length of time.
Twitter hasn't proven otherwise, therefore it is unreliable. I personally
think that more than passes the threshold for "blatantly obvious".

I disagree with you on the impact of pointing out that this was a bad
decision. The fact that you can't rely on random internet services is an
important lesson, and I don't think he learned that lesson. He just switched
from one random unreliable internet service to another. When Zapier or Xively
go down or change authentication requirements or decide he's a spammer and
shut down his account, does he have some way to find out?

I would much rather have someone tell me "that was pretty stupid of you, this
was your real mistake here" than hide the truth from me because they're afraid
of discouraging me. I want to be wrong for as little time as possible.
Apparently this is an unusual sentiment, which I have a hard time coming to
terms with.

~~~
wpietri
Every TOS says something like that, and there are reasonable interpretations
that don't include "we will silently break your shit". Even if he had read it,
this particular failure case might not have occurred to him.

The third option, one between "you're stupid" and hiding the truth, is to not
be a dick about it. The only way people get to the point of being experts is
by making a lot of mistakes. If you'd like somebody to listen to your wisdom,
there are better approaches than starting out with "that was pretty stupid of
you". Especially when it was not in fact pretty stupid of him.

~~~
mikeash
Tell me more about these reasonable interpretations that don't include "we
will silently break your shit". Because my understanding of the sentence
ending in "at any time for any reason" can be paraphrased as "we will silently
break your shit if and when we feel like it".

And, well, we're starting to get repetitive, but I disagree with the premise
of the second paragraph. Building a service like this that relied entirely on
Twitter (and the local internet provider and the local power company and
whatever else) with no way to find out if it stopped working was pretty
stupid. Yes, it's the sort of mistake a lot of people make, but that doesn't
mean it's not-stupid. Pointing out that it was stupid conveys that the
ultimate problem was not in choice of service but in the thought process in
how the problem was considered and how the solution was conceived.

Maybe there are more tactful ways of pointing out "you didn't think this
through very well". But hell if I know what they are (the thing I just said is
not IMO any nicer than "that was dumb") and hell if I'm going to put a bunch
of effort into figuring it out for a throwaway internet comment.

~~~
wpietri
The word "will" is wrong. The TOS is more reasonably interpreted as "we _can_
shut down tomorrow, and _might_ break your shit". They're saying that they
won't assume the user's financial liability if things break. Pretty much every
modern web TOS looks like that.

Unless you have some sort of objective measure, calling something "stupid" is
just your way of feeling smart at somebody's expense. It's a dick move.

The more useful standard is "reasonable", as in the legal term "reasonable
mistake". Here, I think he made a reasonable novice mistake. Being a dick to
reasonable novices does not reduce the number of mistakes; it just decreases
the number of novices who will actually ask. And also the number who will post
articles like this, ones useful in educating other reasonable novices.

I'll add that if you aren't willing to put any effort into a comment, maybe
you just shouldn't make it. You can just say out loud, "Boy, that was stupid,"
enjoy feeling superior, and move on. If you're going to the extra effort of
pissing all over somebody in public, then you might as well go all the way and
make a useful comment. Or even a compassionate one. That those things are hard
for you is only about what you've decided to get good at, not what's actually
easy or hard.

~~~
mikeash
I'm not allowed to make subjective statements? What the hell?

I like how you do everything you criticize. You keep saying I shouldn't say
things are "stupid" because it'll hurt the poor guy's feelings and it's
subjective anyway and people won't learn that way. But you have no qualms
calling my comment "a dick move", as if that was somehow an objective fact,
and then acting like I'm supposed to learn from _your_ comment even though the
guy who wrote the article is not supposed to learn from mine.

Honestly, lead by example or go away. You're not helping by committing every
sin you accuse me of while claiming to be trying to educate me.

~~~
wpietri
I'm not inclined to allow to you make subjective statements as if they were
objective. If you want to say, "I think you're stupid for making a mistake
like that," then say it. Then it will be obviously dickish. And I'm also not
inclined to let dickish statements, sutble or overt, go unchallenged, because
it interferes with one of my goals, which is that novice programmers
experience a helpful and supportive community, so that they grow up to be
accomplished ones.

That an action is dickish is indeed factual. The guy attempted to make a
positive contribution to our collective dialog by offering a story of his
mistake. That took bravery and generosity. You admit you made a lazy negative
comment, one you now seem to realize wouldn't help him. You were, in the words
of Urban Dictionary, being "brash and insensitive", "put[ting] your own
interests" first.

Your last paragraph is especially funny. By your own standards, my behavior is
apparently fine, but still you complain. If you think it's ok to call
somebody's mistakes stupid, surely I can point out your mistakes using
negative tone. Funny how you think negativity is wrong when you're on the
receiving end of it.

~~~
mikeash
I wouldn't care about your negative tone if 1) you weren't talking about how
bad a negative tone is and 2) if you were making any kind of decent argument.

I care about substance. Your comments have none. Thus they are irritating.

You're not inclined to "allow" me to make subjective statements as if they
were objective? What do you think you are, the Internet Police? Go away before
I laugh at you a second time.

~~~
wpietri
You're the one who brought up "allow", so I can't say what you mean by it.
What I mean is that I'm disinclined to let your dickishness go by
unchallenged.

Your "OMG hypocrisy" routine is cheap and lazy rhetoric. I'm not saying a
negative tone is universally bad. I'm saying that replying dickishly to a
positive novice contribution harms the dialog by reducing future novice
contributions. Whereas I think negative tone in reply to a dickish comment can
help the dialog by reducing future frequency of dickish replies.

If you don't like being irritated by people calling you out to being a dick to
noobs, maybe you could try stopping being a dick to noobs. Then we can all be
less irritated.

------
11thEarlOfMar
This is just the beginning. Connecting things to the Internet is a whole new
kettle of fish and there is lots for the novice developer to learn. There will
be disasters of all scales. So we need a new school of thought.

------
sgulseth
So, you let a bird guard a fish?

