
How Twilio screwed us over - feint
http://feint.me/2010/09/how-twilio-screwed-us-over/
======
jeffiel
Just posted this on Anthony's blog, but thought I would post it here as well:

\--

Hi Anthony,

I'm sorry that you feel we were't straight forward, although our goal was to
be completely transparent in our communication with you about the status of
this feature. Since the time of launch, we thought we had clearly messaged
that international SMS has been an unsupported feature, and that the entire
product was in beta status. We had published this fact on our SMS product
FAQs, as well as in the public GetSatisfaction forums. Our apologies that we
weren't clear enough about that fact, which caused you to roll international
SMS features into your app.

We do pride ourselves on putting developers and their applications first, and
in being open in our communications, product capabilities and limitations.
Please feel free to email me at jeff@twilio.com, I'm always open to feedback,
questions, comments or concern.

Sincerely,

Jeff Lawson Co-Founder & CEO Twilio.com

~~~
OoTheNigerian
Is this an apology? "you feel".... " we thought"

You are not really taking responsibility here. You are good guys. Just say "I
am sorry" and ask how you can make him feel better.

[http://37signals.com/svn/posts/1528-the-bullshit-of-
outage-l...](http://37signals.com/svn/posts/1528-the-bullshit-of-outage-
language)

~~~
fredoliveira
Isn't "I'm sorry" the very _first_ thing he says after "Hi Anthony"? And as if
that wasn't enough, he goes on to say "Our apologies" later in the paragraph.

~~~
javery
He said he was sorry that you feel we did x. So he is apologizing for for the
fact that Anthony feels a certain way, not the same as saying "I am sorry.".

~~~
sprout
He has nothing to apologize for. You use features clearly marked
'unsupported,' you are going to get burned.

~~~
gommm
I was also planning on using twilio for a project and when looking at their
faq for the initial research, it wasn't very clear that it was something
completely unsupported that could be disabled at any moments...

------
abraham
From the Google Cache of Twilio's International FAQ:

> You may send SMS from a Twilio number to an international number and find
> that it works. However, this is an unsupported feature and service outside
> the US may change at any time.

Can't get much clearer then that.

[http://webcache.googleusercontent.com/search?sourceid=chrome...](http://webcache.googleusercontent.com/search?sourceid=chrome&ie=UTF-8&q=cache:http://www.twilio.com/faq/international)

~~~
csmeder
Well it would have been clearer if it said: _However, this is an unsupported
feature and service outside the US may change at any time WITH OUT WARNING._

The reason he is upset is that Twilio didn't give him a warning window.

~~~
donohoe
The wording could be better, but the intent is very clear. Twilo is still ok
in my book.

------
mayank
Has anyone noticed an abundance of ranty blog posts (with authors who cannot
distinguish between their/there) on HN lately? Here's the issue in brief:

> The email from twilio informed me they had, without warning switched off all
> international SMS functionality. ...they were able to massively degrade the
> functionality of our app – an app which not only do people pay for, they
> rely on.

Which I suppose is interesting enough, but then:

> But when it comes what really matters, being able to offer my customers a
> reliable service, you need a written agreement.

Which isn't exactly profound wisdom. If you're charging people for a service,
you better make sure that you can hold your backend providers accountable for
their end of the chain.

~~~
redstripe
Well, since they don't provide an SLA you can't really hold them accountable.
Does that mean you shouldn't build services for which you charge money on top
of twilio?

Is twilio better suited to internal use rather than for building SaaS apps?

These are serious matters.

~~~
wccrawford
If you give an SLA for your service, you had better have SLA all the way back
down the chain. So yeah, it does mean that you shouldn't charge money for
services built on Twilio (until they have an SLA) if you intend to provide
reliable service yourself.

------
jacquesm
SMS is the ugly step-child of international communications, a thing that
should cost $0, and should be easy has been made in to a nightmare to deploy
and a patchwork of carrier related troubles.

I can fully understand their decision to go US only for now, but I _really_
hope that at some point they'll be able to roll this out reliably world-wide,
that alone would lay the world literally at their feet.

So many services are now hard to impossible to implement because there is no
easy way of doing this, in my own site I would have several applications for
it right away but _only_ if I can guarantee international delivery.

~~~
points
There are _so_ many other SMS providers, with cool APIs, that send
internationally without any issues.

I used one such service from 2000 or so when I ran a ringtone website.

It's a solved problem. Just search Google for "bulk sms gateway" etc and
you'll find literally hundreds of providers with various APIs and so on.

~~~
jacquesm
I've tried a few over the years, and I've yet to find one that can actually
deliver on a service level agreement, would you mind sharing which companies
you have good experiences with?

~~~
points
I'll have a look back for the main one I used and had success with. They
offered delivery reports, raw data sending etc I never had any trouble with
SLA when I was using them for sending out ringtones. Will see if I can find
the provider I used. It would have been British.

~~~
jacquesm
Cool, thank you very much for looking!

------
mahmud
I invite the author to sample the basket of shit that is SMS delivering
services out there before complaining about Twilio.

We have _paid_ for an "SDK" with one provider, they came to our office and
installed a _hardware_ device which we also had to buy, and they still can't
manage to deliver 10% of our messages. That was in Australia.

After that mess, I personally wrote Twilio a letter asking them if I can help
them establish themselves in Aus and the Middle East; which they declined,
saying they were getting around to it themselves, which is good.

Bottom line: I would chalk this up as a slight oversight by a bunch of very
capable, if very busy people, bless their heart.

They are one decision away from being fucked over by Skype and Google, if
those two ever decide to compete with them. So I am rooting for our kids, even
if they have to make gut-wrenching decisions like this on the spot.

Remember, just a year and a half ago they were nothing but a RESTful URL that
accepted a few XML messages. They have come a long way, imo.

~~~
statictype
Couldn't you just use a gsm modem connected to some computer with a sim card
in it?

~~~
viraptor
That only kind of works... Actually, once when we needed to deliver many
messages, we tried to get 6 usb gsm dongles. Then we discovered that you can
overload 1 gsm transmitter with 6 standard dongles - maybe the equipment was
just weird in some way, but at some point messages simply started to be
rejected. Another problem is the delay - you need ~4s to get an
acknowledgement which seriously limits the throughput. Then there are other
problems with mass-sending... there is one 8-bit counter, but I can't remember
now for which functionality (either replies, or delivery confirmations), which
killed the idea in the end (it was supposed to be mass-notification, so
thousands of texts / hour).

Just find someone who provides smpp gateways and save yourself the pain...

~~~
statictype
Ok right. I wasnt really thinking of bulk mail. Gsm modems work fine though
for less frequent usage like sending notifications when the motion alarm is
triggered etc.. Twilio isn't really for bulk sending either, is it?

------
emcooke
For those wondering about the technical aspects of this decision take a look
at this thread:
[http://getsatisfaction.com/twilio/topics/international_sms-1...](http://getsatisfaction.com/twilio/topics/international_sms-1591m)

When we launched the Twilio SMS Beta we tried hard to support sending SMS
messages to both US and International destinations. When there were problems,
we worked with our customers to collect forensic data on hundreds of carriers
worldwide and pass it to our carriers partners to debug.

At Twilio we are dedicated to working with top quality carriers and
technology. After months of working to fix problems, were not able to deliver
the reliable International SMS service our customers have come to expect.

We apologize for any problems this has caused for our customer and we'll work
to bring back International SMS service after were able to deliver on the
quality we do the rest of Twilio services.

Cheers, -Evan

CTO and Co-Founder

~~~
feint
That's fine. In fact it's a good move. But why email me after the fact. It's
like you brlirve its ok to communicate with paying customers like that.

~~~
emcooke
Hi Anthony, Twilio service status available via our public status page
<http://status.twilio.com/> We communicated degraded international SMS service
to customers on August 16.

You bring up a good point that the information might not be readily
discoverable. We'll work to make the status page more findable and to extend
the API (<http://status.twilio.com/documentation/rest>) with features such as
RSS to let customer subscribe up-to-the-minute status information.

~~~
feint
Or. Just email me before you cut a feature next time.

~~~
davemc500hats
unless automated, not a scalable response pattern.

~~~
ajdecon
Presumably they have a list of all their customers with contact info. Not
saying you should spam them often, but for the message "no more international
support"? I think it's worth telling everyone.

------
nakajima
As a __heavy __user of twilio (I work at groupme.com, we're entirely text
message based, and built entirely on twilio), I'll say that every time we've
broached the topic of international SMS, they've always cautioned us that it's
experimental, and so we've explicitly marked our product as such (USA only for
now).

That's not to say that we don't want international bad, but we pretty much
trust their judgement as to whether or not they say a product is ready or if
it ain't. Twilio does good work, and they're amazingly responsive to developer
requests/feedback.

------
bond
So Twilio as an unsupported service which they charge others for it. They
choose to close the service and don't have the courtesy to inform users before
that and now say the service was beta/unsupported and they are sorry for it?

That's really lame...

They have the obligation to inform users they're going to close a service,
whether it's beta or not. The answers they gave is not the way to do it,
giving an excuse because the service was unsupported and completely ignoring
their fault in warning their users shows that they don't give a damn about the
users.

There's nothing wrong in coming here and apologize for not giving a warning
before closing the service, but they choose not to do it...

~~~
jacquesm
They already did, and 6 hours before you posted this.

<http://news.ycombinator.com/item?id=1726444>

~~~
bond
No they didn't: "I'm sorry that you feel we were't straight forward, although
our goal was to be completely transparent in our communication with you about
the status of this feature. Since the time of launch, we thought we had
clearly messaged that international SMS has been an unsupported feature, and
that the entire product was in beta status. We had published this fact on our
SMS product FAQs, as well as in the public GetSatisfaction forums. Our
apologies that we weren't clear enough about that fact, which caused you to
roll international SMS features into your app."

This is not an apology for not informing the developer before closing the
service...

~~~
jacquesm
Just quoting what you quoted: "Our apologies that we weren't clear enough
about that fact, which caused you to roll international SMS features into your
app."

Reads like an apology to me.

Read on for a bit, they acknowledge that their communications were lacking and
have improved on that by making it easier to find this information.

What do you expect of them? Groveling? A refund?

~~~
bond
Look, what he's doing his apologizing for not making clear it was an
unsupported service.

What he should have done is apologizing for closing the service without
telling developers about it.

They can try to deflect this as they want but the fact remains, he should have
sent a warning the service would be closed at date X, not closing the service
and tell afterwards they're sorry for not making it clear...

------
markkat
That is really too bad all around.

The state of SMS is ridiculous. We go through a gateway. Certification was
painfully slow, and the cost... OMFG the cost of a text. -It's as if the
alphanumerals had to be mined, shipped, refined, and then placed into each
message by hand.

I applaud Twilio for what they are trying to do. IMO, they are playing
interference for an ugly beast.

------
grigory
This is worrying. We're using Twilio in my app - they've been good to us so
far, certainly very friendly and helpful. The service has been spotty at
times, but mostly reliable. Most of our customers are based in U.S./Canada,
and all paying customers are; we also had a backup SMS delivery built
recently. However, providing absolutely no notice (didn't seem like I missed
an email from them) of such a significant change in service certainly isn't a
good way to go about this.

------
alttab
This retort to Twilio is just as unprofessional as their "poor service."

As others have pointed out, SMS is an unsupported feature and service outside
the US can change at any time.

 _So in conclusion, apart from twilio losing one of there early adopters
(task.fm isn’t a big customer, but that doesn’t mean we can be screwed over)
they are also damaging there brand. They used to be all about developers._

I think only getting miffed that your _OWN_ service wasn't perfect and then
posting to HN about how task.fm was "screwed over" is more damaging to twilio
(is it?) than an SMS outage.

The author didn't read Twilio's own SLA and hardened entrepreneurs don't burn
bridges publicly everytime something goes wrong.

I know trying to build an awesome web application is hard and its frustrating
when supporting services go down, but that is the nature of our beast. We
build on top of each other, so we must respect and support each other. Getting
mad and cutting off your own hand because you are angry isn't helping build
anything.

Google's API has bugs that my start-ups web application runs into all the
time. Sometimes they change API behavior, which the visualization screws up
the entire UI in the process. _This effects thousands of our users and we have
to release work-arounds as fast as we can._ But I don't blame Google - it was
our choice to use their APIs.

Generally, this sort of thing will continue to happen with all web service
APIs. As a community of developers, I would ask them why it happened and
provide constructive feedback about how important it was. You might just find
they'll listen and you'll have more pull.

Great start-ups have a short list of early adopters or engaged users that they
trust and continually go back to for product advice. Writing a post like this
pulls the rug out from under your own service because you refuse to use it, it
will cost you more, and twilio will suffer from lack of valuable constructive
feeback.

Really just my POV, but hopefully you'll find why posting inflamatory articles
like this only hurts entrepreneurs and our community at large. At most it sets
an example of how we treat each other.

~~~
rmah
Heh, you can't really hold up Google as a paragon of developer relations or
customer service. They have a pretty poor reputation on both counts.

------
feint
For the record - I should have said this in the original post, I like twilio.
I liked their business model and I really liked the product.

I had multiple email conversations with twilio team members over the past year
and "caution" wasn't something I would say was strongly portrayed

------
rokhayakebe
It's 11:24 in phoenix. I just finished a 6 hour sprint working on an app that
will rely 100% of the Twilio api, so the headline caught my attention. By the
time I came back both the CEO and CTO gave an explanation and apologized. I
can respect that. I hope Task.fm gets back on track as well.

Edit: J, maybe reimbursing him would not hurt.

------
grease
Just curious, what are the alternatives to Twilio in case some one wants to
send out SMS across countries?

~~~
MortenK
www.tropo.com and www.teleku.com does both international voice and SMS I
believe. There's also www.cloudvox.com, but I'm unsure if they also support
international.

~~~
akalsey
Yes, Tropo does international voice and SMS. We've got numbers in 36 countries
and speech recognition in 9 langauges.

If you buy a number from us outside the US, we can't do SMS on that yet, but
we're working on it. In the meantime, feel free to send texts to and from
anywhere from your US number. We won't even charge you any extra for the
international sending.

Teleku is now owned by Tropo and as such can do international SMS as well.
CloudVox doesn't do SMS worldwide, but I was talking to one of their founders
recently and he mentioned they're working on it.

------
ajju
Their current faq states that they don't deliver sms internationally.

<http://www.twilio.com/faq/international>

Based on that I presume they switched off the feature globally and not just
for you. Correct?

That doesn't mitigate your complaint.if anything, doing that without advance
notice is worse. Did they provide any explanation in the email?

Twilio has between a very reliable provider in my experience so far, so this
is worrisome.

------
sahillavingia
I'd like to hear Twilio's side of it; they seem like such a great company this
feels... weird coming from there. I think there's more to this story.

 _waits patiently_

------
teljamou
I have founded Nexmo.com, Low Cost SMS API (in private beta), early this year
after spending 7 years in the Wholesale SMS industry. Delivering awesome SMS
service on a global scale is tricky – but it should not be! We just finalised
building our back-end SMS network.

Here are some challenges the industry faces: ———————————————————-

-Many intermediaries to reach end-users:

Between an App and an end-user you can get up to 4 or 5 intermediaries
operating their own infrastructure. Adding intermediaries not only increases
cost but also exponentially reduces quality – QOS is measured in terms of
availability, delivery ratios, speed, features supported, etc…

Ask your provider about their sourcing strategy? Who do they buy from? Do they
have a strict policy in terms of suppliers’ selection? Can they share delivery
ratios?

-SLA does not really matter:

What if the providers of your provider do not guarantee any SLA? A provider
can potentially guarantee an SLA on its own infrastructure but can’t control
what happens afterwards.

-Non transparent fees:

Customers get charged on SMS sent to networks that are not or poorly covered
but listed as LIVE in the provider’s reach list – even when you sign a written
agreement.

-Tricky to reach all Networks:

Some networks in the world are tricky to reach and sometimes the only reliable
way to deliver SMS is to connect directly. Other are installing SMS filters to
cut certain type of traffic (e.g. Spam). For instance, if your provider
mentions networks such as Comcell Colombia or IAM in Morocco, you’d rather
check again because these networks are impossible to reach with full feature
support/reliable service without direct connections. Currently there are at
least 26 networks that are tricky to reach and 22 that are filtering.

Another rule of thumb is to check whether in Europe their prices are above
$0.055, since European Wireless Carriers (also applicable to other
regions/countries) require a termination cost for reliable SMS routes. If
below that price, it could mean that your provider (or the provider of your
provider!) is using unsolicited routes that are not reliable/has limited life
time.

———————————————————————————————————

Delivering hassle free (and Low Cost!) global SMS termination is not obvious
but we are working on it :)

Tony Jamous twitter: @nexmo e-mail: tony.jamous [@] nexmo.com

------
JonM
I was wanting to use Twilio for UK SMS sending but the system couldn't send to
my own mobile. I emailed again a few weeks back and it was still
"unsupported".

I'm glad it didn't work on my own mobile as I would have definitely integrated
it into my site and no doubt run into the same huge problem here!

------
ajays
Presumably, Twilio knows who their top users of the international SMS feature
are. Why not just look at the logs and email the top users, telling them the
service is going to be discontinued? This isn't rocket science, people.

------
po
Why did they switch off international SMS? What was the explanation they gave
you? That seems like an important part of the story that you're missing.

~~~
rmah
It doesn't matter why they shut it off. What matters is that they shut it off
without any notice.

------
jscore
That blog post should've been summarized with one sentence. "dont trust twilio
because they turned off intl sms without warning"

------
knodi
Check out Moonshado SMS on heroku addons for International SMS, it works great
for and its so damn cheap.

~~~
friendstock
that's good to hear... we will try it

------
lovskogen
Off topic, but the text-shadow 1px #fff really strained my eyes while reading
on a iPhone 3G.

------
nomous
(cross posting my comment from his blog)

Sorry buddy; your post is basically a hit piece about an experimental service
which stopped working. It seems they sent out an email and marked it on their
status page.

So I'm going to categorize you with the other startup people who have pretty
blog and hurt feelings for your own mistakes.

I hope I never do business with people like you.

------
drivebyacct2
s/there/their

------
martinkallstrom
This is an important lesson for any startup. Don't rely on free services,
insist that you pay for them and set up an agreement.

But it is also a lesson from the other end. Twilio would probably not have
done this lightly. By not offering agreements for there free service, they had
the option of choosing to disappointing their users instead of loosing what
might be a lot of money. It's never the easiest decision but it in some cases
it is the best option.

~~~
feint
Free service? This wasn't a free sevice. I was a paying customer

~~~
martinkallstrom
My bad for reading the post wrong in this regard.

