
Twitter ‘smytes’ customers - panarky
https://techcrunch.com/2018/06/21/twitter-smytes-customers/
======
eganist
Does anyone inside Twitter know who the executive was who pulled the trigger
on this decision? There's no way this person knew the gravity of accepting the
risk for blowing this many high-profile customers out of the water all at
once.

I'm pretty accustomed to executives making dreadful decisions without the
approval/acceptance of other stakeholders within said person's firm. I'd
rather know who made this error and not interact with that specific person's
department rather than stop doing business (e.g. large ad-buys) with Twitter
as a whole.

I don't want this to be a witch-hunt so much as I want the person to just
_come forward and own the decision_ , because unless they have an
_exceptionally_ good reason for it, it comes off as absurdly high-risk to both
Twitter-the-business as well as to all the clients who've likely written
serious penalties into their contracts for events like this, which again
brings that business risk back full-circle to... Twitter.

~~~
cosmicexplorer
I work at twitter and wasn’t able to find out immediately, but have been
collecting responses like these + the article and communicating with someone
who knows what to do. I only found out about this over twitter and I am
personally deeply frustrated and working to understand whatever led to this.

~~~
philipov
To be honest, this reads less like an attempt to integrate anti-harassment
measures and more like an attempt to completely destroy a business trying to
offer them. It further solidifies Twitter management as complicit in aiding
and abetting the harassers by using its capital to eliminate a threat from the
market. After the speed with which Twitter went after the ICE employees list,
the evidence is clear that Twitter only has a problem tackling harassment when
it's against marginalized populations.

~~~
dang
That's a pretty extreme leap when virtually no information is available yet.
The HN guidelines ask you to " _assume good faith_ " for a reason: people are
all too ready to take such charges as givens and then pile more on top of
them. It's a behavior that harms the container here, and it's not hard to wait
until actual evidence appears.

~~~
philipov
That's fair, I will suspend judgement until further evidence appears, but the
well of beneficial doubt is running low. I think a long past history of
suspicious behavior is a good reason to update our assumptions.

~~~
dang
I don't disagree. But the real reason for having an "assume good faith" rule
is what it does to this community when people don't. Therefore it needs
practicing even when it feels undeserved.

~~~
tedivm
Here's the whole guideline you're talking about-

> Please respond to the strongest plausible interpretation of what someone
> says, not a weaker one that's easier to criticize. Assume good faith.

It's talking about how we all talk to each other. I think it's main purpose is
not to lessen criticisms for corporations so much as to keep the discussion
constructive.

~~~
pvg
You're probably not going to win an argument with a moderator by explaining to
them the subtle logical error in the interpretation and purpose of their own
guidelines.

~~~
cjbprime
Yet now we're failing to assume good faith in our moderators, by assuming that
they will fail to recognize a conflict of interest arising from questioning
the extent and source of their authority, which they have reason to see
maximized.

The parent comment actually seemed spot on to me. The rule is to assume good
faith of your interlocutors. The parent failed to assume good faith of a
billion dollar corporation. I think it's reasonable to push back against being
asked to tone down the latter, while honoring the rule for the former.

~~~
pvg
I think dang explained it pretty clearly in the comment. It is bad for the
forum when, given some very limited set of facts, you offer the most heated,
ragefest pile-on inducing explanation or theory. It doesn't require some
talmudic reading of the guidelines which are, after all, guidelines and not
every single thing is explicitly spelled out. 'Avoid taking discussions into
flamey or ragey directions' is advice the mods dole out to some person or
another nearly daily.

------
kaeawc
We had 20 minutes notice, and then everyone was kicked out of the Slack
support channel and API responses simply died. What the actual fuck. They have
mobile client SDKs out in the wild that are now just eating up battery life as
they retry an impossible query forever.

~~~
jasontan
That sucks. We (Sift Science) have been building something similar for 7 years
and aren’t going anywhere. If we can help, please ping me - jason at
siftscience dot com

~~~
csomar
> We (Sift Science) have been building something similar for 7 years and
> aren’t going anywhere.

But that doesn't come with any actual guarantees does it?

~~~
glitcher
> and aren’t going anywhere

Yet... that you know of.

Nothing specific against Sift Science whatsoever in my comment, but many, many
times this sentiment has been conveyed by companies and it rings hollow IMO.
There are a lot of different ways a company can change or disappear at some
unforeseen point in the future, and claims that "you can trust us to be here
forever" do not carry much weight for the large group of experienced users,
devs, management, etc who have been burned multiple times.

~~~
cortesoft
So are you implying that you should never enter into a business relationship
with another company?

Clearly, you need to always be prepared for your partner to go away, but you
still have to work with other companies.

~~~
sushid
I think the parent commenter is just saying that it's not something you can
actually believe, just like a company saying they'll never sell your data.

Sure, that's the intention but when someone comes around waving a checkbook,
they can in turn buy you and shut you down, sell your customers' info, etc.
Sure, its not YOU allowing that to happen but by ceding control you
essentially allowed that to happen.

------
shakna
> Indiegogo, GoFundMe, npm, Musical.ly, TaskRabbit, Meetup, OLX, ThredUp,
> YouNow, 99 Designs, Carousell, and Zendesk

Half an hour of warning, to screw over all of these?

They broke npm's user sign-ups, and publishing of packages, with half an hour
of warning.

I can't imagine the havoc over at Zendesk either.

That is not 'winding down'. That's ghosting.

~~~
pkaye
What would be these businesses backup plans if Smyte went bankrupt instead
being acquired?

~~~
tinus_hn
Your supplier going bankrupt is a well known risk that you cover by checking
its credit rating and by paying on or after delivery.

~~~
AznHisoka
How do you check the credit rating of a startup?

and how would their credit rating be affected if they pay all their bills but
just reduce the number of bills gradually over time?

~~~
PeterisP
One way that's sometimes done is that you require them to submit an
independent respectable audit of their financials, or in case of
confidentiality issues, not submit the actual financials but a statement from
a respectable auditor that they have verified the suppliers financials and
their cashflow/"runway"/etc satisfies the criteria that you required.

------
danso
Maybe I'm in the minority, but I take an interest in the latest
software/tech/ideas for dealing with the hard problem of moderation, and I had
never heard of Smyte, until now (which made me think TC was being literal with
its " _Twitter 'smytes' customers"_ headline).

Apparently, they were a dominant player in this space, such that its shutdown
impacted such a wide array of significant-sized companies, from ZenDesk to
TaskRabbit to Meetup. Even more surprising is that they were part of YC (W15),
and yet from what I can tell, have only had one significant mention on HN from
3 years ago (65 upvotes, 11 comments)

[https://news.ycombinator.com/item?id=9758464](https://news.ycombinator.com/item?id=9758464)

I would've guessed that Smyte would've been the kind of Silicon Valley
business that was made for media hype (what with all the attention/criticism
given to fake news and Google/FB/Twitter moderation). Then again, it does seem
that content moderation -- important as it may be -- isn't a field as sexy as
AI/deep-learning and other forms of automation.

~~~
bsimpson
They were also noteworthy for their founder Pete Hunt, who is largely credited
with instigating the open sourcing of React. (As I recall, it was internal
Facebook tech that he extracted into a separate library when he moved from FB
to Instagram after that merger.)

------
lilyball
Given that these businesses had contracts with Smyte, how is this not a
flagrant violation of the contracts? Surely Twitter took on Smyte's
contractual obligations when they acquired the startup?

~~~
DannyBee
There are many ways to accomplish this, for example, doing it as an asset
purchase instead of acquiring all the stock.

In the former, you can take the assets (including employee contracts) without
the liabilities.

There's actually a reasonable quora answer (for once) on this, saving me from
writing it up for you :)

[https://www.quora.com/What-happens-to-legal-contracts-
financ...](https://www.quora.com/What-happens-to-legal-contracts-financial-
obligations-and-employee-agreements-during-an-acquisition)

~~~
larkeith
In a case like this, does that mean that Smyte (the company, even if they no
longer own the trademark) still exists and is liable for all the penalties?
Would they usually ask for the cash to pay for them as part of the asset
purchase?

~~~
voxadam
In a case like this what's left other than an empty sack? Unless you're able
to pierce the corporate veil there isn't anyone/anything with assets left to
be made whole by (e.g. sue).

~~~
Gibbon1
One could argue the transfer was fraudulent. And go after both parties.
Twitter and the founders.

Don't know, not a lawyer.

~~~
civilitty
IANAL but I was a plaintiff in a similar case. We were made whole in a
settlement but before that, the judge tore the defendants a new one over this
tactic leading to a bitter dispute between the acquierer and shareholders (or
the board of officers, don't remember) over who was liable for breach of
contract. Eventually it came out that the auditors had done the math and
included cost of litigation/liabilities on these contracts into their
valuations, basically admitting that they knew about the obligations
beforehand, so the acquierer ended up settling.

At the end of the day, judges (and appellate panels) have to interpret the law
and they don't function like automatons. They take into account the spirit of
laws/contracts as well as the letter and most have enough common sense to get
missed off at these naive tactics.

------
gingernaut
Incredible. Shutting of API access to paying customers after a 30 minute heads
up at 6AM.

Astoundingly unprofessional.

------
jarfil
"trust and safety as a service"

So... what was the Service Level Agreement? and the Terms Of Service?

People get all hyped by -aaS stuff, but services do go down, and if you aren't
the one controlling when and how they will go up, then you better have a
contract with the Service Provider on some terms you like.

Even then, prepare for when they go down, because they will sooner or later,
for shorter or longer periods of time.

I find it silly to rely on some service, any external service, so much that if
it goes down it would cause a prod outage.

~~~
pvg
_I find it silly to rely on some service, any external service_

This is always trotted out and it's, by this point, a completely content-free
thing to say. Short of removing yourself from the economy and society in
general to live off potatoes grown in your own dung, you're going to rely on
some external services and the fact that, for the most part, the parties you
do business with will behave responsibly. When they behave very irresponsibly,
unpleasant things happen and people quite reasonably complain.

~~~
bastawhiz
Exactly. You want reliable CAPTCHAs? CDN service? Malware detection? Customer
support software? Website uptime monitoring? Accounting software? Analytics?
Email delivery? Good luck building and running all that yourself.

~~~
im3w1l
All of those can be bought and then self hosted except for CDN.

~~~
laumars
I don't know why you're getting downvoted because you're absolutely right.
Perhaps most of the people in this thread haven't managed servers for more
than a decade - because it used to be (and still is in some organisations) the
norm to self-host half the stuff that was listed.

    
    
      * Malware detection?
    

Isn't that pretty must how most malware detection works? Sure they use the
cloud to pull periodic updates but the software itself is self hosted

    
    
      * Customer support software?
    

I've lost count of the number of solutions available to self host. Both free
and commercial.

    
    
      * Website uptime monitoring?
    

While there is a strong argument for hosting that in the cloud, there are
umpteen solutions for website monitoring.

    
    
      * Accounting software?
    

I didn't even realise running accounting software as a service was now a
thing!

    
    
      * Email delivery?
    

Fair enough, self hosting email can be a massive headache. Less so if you run
Microsoft Exchange (one of the few things Windows actually makes easier than
it's Linux / UNIX counterparts). You can even go for a hybrid approach and
self host the mail server but have a mail proxy between your server and the
outside world - so you benefit from spam detection as a service while still
hosting your own mail.

I'm not going to argue that one is better than the other when it comes to SaaS
vs self hosting. But people in here seem to have very short memories when
arguing that self-hosting isn't at all viable. In fact I have personally
supported all of the above in self hosted versions in the last couple of
years.

~~~
tomhoward
I don't think anyone is disputing that it's possible, but rather how cost-
effective and time-efficient it is [1].

I also have the experience of managing many of these services, and I therefore
know just how much more productive you can be when you don't need to do it
yourself.

Of course there are trade-offs; business is nothing if not a game of
tradeoffs. We're just talking about what is optimal.

[1] Not to mention how much more stable and secure; Self-hosting can involve
huge security and reliability risks.

~~~
laumars
> We're just talking about what is optimal.

I don't believe what is optimal can be generalised like that. "Optimal" is
going to be specific to the business in question. Which is why I'm a firm
believer of leaving all the options on the table.

> Self-hosting can involve huge security and reliability risks.

In fairness so can the cloud. I personally don't see difference as 'risk' but
more 'responsibility'. Some of the places I've worked have been PCI DSS and
Gambling Commission audited so we had that responsibility already. Self
hosting a few other resources didn't really add much extra in terms of our
security responsibilities. But the case would be very different of lots of
other different types of businesses.

~~~
tomhoward
I think we're agreed that one shouldn't be absolutist about these things and
that different approaches will be optimal depending on the circumstances.

It's worth reminding ourselves that the root comment was an absolutist
assertion about the folly of outsourcing.

So the discussion is easily resolved: don't be absolutist :)

~~~
mmt
> It's worth reminding ourselves that the root comment was an absolutist
> assertion about the folly of outsourcing.

Even that strikes me as a mischaracterization, or, at best, taking just one
statement out of context.

It was an assertion about the folly of relinquishing control, not being
prepared for that lack of control, and thereby subsequently suffering a prod
outage.

------
a_lieb
I feel like there _must_ be more to this story. Maybe they immediately saw
security problems, or legal issues. Not saying it's going to exonerate Twitter
(maybe the truth is even worse than it sounds), but this is such an obvious PR
mess that I can't see them doing it for no reason.

~~~
pishpash
That they didn't notice during due diligence?

~~~
rrcaptain
Perhaps they still wanted the company for some reason? Someone knows. Time
will tell most likely.

------
whatshisface
> _Clients had multi-year contracts in some cases._

I'm calling it now: B2B startups will soon have to sign poison pill contracts
that specifically detail what happens if they're bought out. This might ruin
the "buy them for their people and discard the business" method of hiring - or
at least it will if the customers have any sense.

~~~
frankc
I don't know how it works in Silicon valley, but finance companies I have
worked for have pretty much always required that source code be turned over if
the vendor folds when doing business with small vendors.

~~~
walshemj
Yes I worked for the company doing the .coop registry and ICANN had very
strict rules about code escrow we had to follow.

------
mcintyre1994
YC W15, I wonder if this will have any impact on the ecosystem of startups
like Smyte getting established YC companies as early adopters.

Also I wonder how many customers are staying quiet, because "our fraud
protection provider just stopped working" isn't something they want to reveal.

------
FrozenVoid
This is incredibly fragile ecosystem if one obscure API getting shutdown
causes so much damage. This could be prevented by a simple periodic check that
would determine if API is available and some fail-safe alternative to remain
online. Instead this is like a house of cards that breaks with any of the
cards fold. I'm not excusing the abrupt shutdown, its obviously a wrong way to
end service, but being prepared for it is much better.

~~~
necubi
This is a hard call when the system is question is your anti-abuse provider.
If you "fail open" when they're down, you risk allowing a flood of bad users
during outages. Depending on your business, that may be much worse than having
an outage yourself.

(Disclaimer: I work for a smyte competitor).

~~~
FrozenVoid
You can turn on something like manual verification of new users, an
alternative security service, or just temporary halt new account registration.
All of which don't result in system wide failure. Even a few abusive users is
a small and temporary cost to absorb vs complete outage.

~~~
krainboltgreene
Ah, yes, just the casual manual verification of _checks notes_ a fire hose of
data.

------
busterarm
I think everyone is missing that this is the canary in the coalmine for how
the tech giants are going to behave going forward. Hell, Twitter isn't even
that big!

Every tech company big enough probably sees the same writing on the wall the
rest of us do that it's time to press their advantages. That means shutting
everyone else out.

We probably all have mission-critical parts of our infrastructure that aren't
core competencies. We probably can't afford for them to go away tomorrow, but
now it's time to expect and plan for it.

Get ready to ride the consolidation wave.

~~~
pedalpete
but why? If they have contracts with these external companies, and revenue,
why shut it down. Even if it isn't for a huge amount of $.

The goodwill itself has to be worth a TINY bit.

~~~
Rjevski
Goodwill works for good companies proud of what they do.

Companies like Twitter whose core business is to shit on their users and
customers don’t care about goodwill.

------
Endy
While this should be upsetting, and I'm sure it was to the people who were
affected, I'm actually happy to see it. I'm of the mindset that this will have
an overall positive effect on the state of the Web. Why? Because users and
devs both need to remember that companies are not friends, and contracts are
only as enforceable as the courts will allow.

Imagine if Google decided that tomorrow at 6AM PT they were pulling the plug
on YouTube embedding, or GMail / GDrive integrations. The court's going to
say, "They put in the contract that they can change the terms of service at
any time, and they're not liable for outages or lack of service. And you
signed it. Sucks for you."

Then again, I'm just anti-corp in general and this gives me better reason to
avoid Twitter.

~~~
csomar
> Because users and devs both need to remember that companies are not friends,
> and contracts are only as enforceable as the courts will allow.

I bet their term of services mentioned that they can cease operations on no
notice.

~~~
zaarn
Doubtful. For most long-term corporate contracts there is no "we might cease
operations at any point and leave you hanging before end of the contract".

No company lawyer I know would willingly put their signature on such a
contract unless the other side is using a free tier. If someone actually
signed 3 year contract with such a clause, congrats, you managed to throw a
lot of money into a black hole.

------
rememberlenny
If you're looking for a Smyte replacement, check out Koko. It's AI for content
moderation and classification, developed at the MIT MediaLa.

[https://www.koko.ai](https://www.koko.ai)

~~~
gant
Solid track record. Especially the part about spamming suicidal people on
Reddit.

[https://www.reddit.com/r/SuicideWatch/comments/5jin1q/warnin...](https://www.reddit.com/r/SuicideWatch/comments/5jin1q/warning_if_you_get_a_pm_from_uitskokobot_please/)

[https://www.reddit.com/r/SuicideWatch/comments/5x60f0/update...](https://www.reddit.com/r/SuicideWatch/comments/5x60f0/update_on_the_warning_about_pms_from_koko/)

------
_jal
That's really amazing - I am having great difficulty imagining a situation in
which doing this would make sense. Just for starters, even if you completely
discount goodwill, ethics or even PR, I'm guessing Twitter just bought a bunch
of lawsuits. This just seems nuts.

------
AJRF
Twitter seem to have an institutional blindess towards the importance of APIs.
Their execs need to be sat down and have it drummed into them that breeding
this much bad “karma” (for lack of a better term) is going to come back and
bite them in the ass.

~~~
busterarm
This is the same company with NIH Syndrome severe enough to in-house their own
woefully-under-performing message queue. Twice. And they blamed Rails after
the first go of it.

I'm not sure they know what interop means.

~~~
dickbasedregex
Yup. They started the "rails don't scale" narrative.

Something something, a good craftsman doesn't blame his tools.

------
yborg
[https://joinmastodon.org/](https://joinmastodon.org/)

Twitter will never change how they do business, replacement is the best
strategy.

~~~
wencha
This is not an appropriate place for your Mastodon spam.

~~~
vignesh_m
Seems like it is, considering its a twitter replacement and the op is kinda
anti-twitter

~~~
wencha
TFA has nothing to do with Twitter the product, which is what Mastodon
purports to replace.

------
chris_wot
They had contracts that they took on when they acquired the company. Twitter
are now legally liable for these contracts, I won't be at all surprised if
someone sues them. And frankly, they would be in the right.

What an absolute debacle. This doesn't bode well for the entire Twitter
platform.

~~~
x0x0
Often not.

I went through an acquisition of an unsuccessful company. My employer created
a shell which is what was acquired. The old corporation wasn't acquired and
existed as an independent entity to wind down contracts / deal with the
acquisition hold-back, etc.

If eg smyte was out of money, this is plausible. And all of a sudden no-one
was there to run the old corp.

crunchbase says they raised a $4m A in Mar 2017, but linkedin says they had 23
employees. They could burn that in 1.25 years with 23 employees and have hit a
cash crunch.

I have no knowledge, just speculation.

~~~
glangdale
That is not a good structure for the acquisition, especially if you are on the
hook for the shell. Quite often you have all of the responsibility to deal
with shit hitting the fan but no resources or personnel to actually do it.

~~~
x0x0
That is (obviously) the trick: the "acquirer" doesn't acquire the shell. And
therefore is on the hook for nothing. Customers have contracts with a
corporation with no assets and no employees.

~~~
raverbashing
I wonder if this flies with a judge.

Otherwise it would be a good way of making debts disappear.

~~~
pmontra
Yeah, it could be borderline with fraud.

------
gowld
Twitter bought a company, then found out that is isn't production-ready
because it doesn't handle customer data safely, so immediately shut down the
service.

I wonder if that triggers any clawbacks in the purchase price, as Smyte was
running a substandard service.

~~~
tedivm
This is the best theory I've heard on this so far.

------
ComputerGuru
Twitter continues its insanely tone-deaf approach to dealing with third
parties that interact with / rely on their services. I don’t expect anything
more from them, but am surprised Smyte sold out their users just like that, as
I’m sure their fate was part of the negotiations.

Given there are actual damages coming out of this to paying customers perhaps
we’ll finally see them taken to task.

------
greggarious
So Smyte's customers had a valid contract (article mentions a 3 year contract
with one customer who was cut off).

I find it hard to believe being acquired magically relieves you of your
contractual obligations... could this be cause for litigation?

~~~
ttul
It depends. Was Smyte acquired, or was it merely an acquisition of their
assets?

~~~
asfasgasg
Even if the assets were acquired, I wonder why you can't sue the remaining
shell and get a cut of the money that was exchanged for the assets.

I guess the real issue is that smyte's contract probably said something like
(from GCP's terms of service):

> 13.1 Limitation on Indirect Liability. TO THE MAXIMUM EXTENT PERMITTED BY
> APPLICABLE LAW, NEITHER PARTY, NOR GOOGLE’S SUPPLIERS, WILL BE LIABLE UNDER
> THIS AGREEMENT FOR LOST REVENUES OR INDIRECT, SPECIAL, INCIDENTAL,
> CONSEQUENTIAL, EXEMPLARY, OR PUNITIVE DAMAGES, EVEN IF THE PARTY KNEW OR
> SHOULD HAVE KNOWN THAT SUCH DAMAGES WERE POSSIBLE AND EVEN IF DIRECT DAMAGES
> DO NOT SATISFY A REMEDY.

> 13.2 Limitation on Amount of Liability. TO THE MAXIMUM EXTENT PERMITTED BY
> APPLICABLE LAW, NEITHER PARTY, NOR GOOGLE’S SUPPLIERS, MAY BE HELD LIABLE
> UNDER THIS AGREEMENT FOR MORE THAN THE AMOUNT PAID BY CUSTOMER TO GOOGLE
> UNDER THIS AGREEMENT DURING THE TWELVE MONTHS PRIOR TO THE EVENT GIVING RISE
> TO LIABILITY.

------
astonex
What the hell are npm doing that this caused a prod outage.

Either way, really poor from Smyte. No reason to immediately turn off access

~~~
p1necone
If I was including an anti spam/security service as part of my pipeline I
probably would want my system to stop if it went down, rather than just
skipping the protection. Especially as cutting me off from said service is a
possible attack option for a bad actor.

~~~
RIMR
Seriously. Shutting down a security product without warning is just
reckless...

------
dickbasedregex
After hearing so many stories like this for so many years, any time a company
acquires another, it usually seems advisable to jump ship from the acquired to
other vendors before things start going south.

PR will come up with all sorts of reasons said acquisition will benefit both
company's customers but I've yet to see it.

------
fipple
Oooh.. I'm glad they screwed over well-funded companies with multi-year
contracts. Now at least they're going to get the shit sued out of them unlike
when they go after regular users.

~~~
toephu2
Sued for what? Have you seen the contracts these companies signed w/ Smyte?

~~~
fipple
If these companies signed an agreement with a vendor that allowed them to cut
off service with 15 minutes notice, they deserved everything they got.

------
mrleiter
How's the legal situation in America regarding breach of contract?

At least in Austria we have a law that exactly tries to prevent such things
(§38 et seq. UGB): when you buy a company you enter into all of their existing
(non highly personal) contracts and obligations, unless you specifically don't
want that, in which case you must tell the other side and give them some time.

~~~
Can_Not
I am not lawyer. I don't think there's a valid loop hole that would pass a
judge in a court of law. Either the contract specifies early termination
damages or it was neglected.

------
sergiotapia
lmao this is hilarious. Never seen something like this happen with such a
large company. Let's hope the codebases out there had a simple integration
with Smyte and the change to that "smyte" filter was fast. Crikey!

------
Zarel
Hm, one of the tweets mentions a "3 year contract". A contract means two
parties agree to something, with penalties if they don't do what they agree
to, right? Couldn't they sue Smyte/Twitter or something?

~~~
sbr464
Usually there are terms on acquisitions deeply nested in contracts.

~~~
glangdale
Usually those terms involve "keeping the lights on". I imagine Twitter will
just pay people off, which eases a lot of pain.

------
alliecat
Ah I see the plan here: Twitter knows their platform is a cesspool of abuse
and harassment, but instead of fixing it they're pulling the rest of the
internet down to their level so they stand out less!

Genius. /s

------
polskibus
That will make ALL companies think twice about trusting a startup when
choosing a SaaS provider. VC got its way but at the same time decreased any
future startup success potential by a huge measure.

------
ksajadi
As multi-stepped and prolonged as it is to sell to large enterprises, I don’t
look forward to tomorrow when yet another question to be answered before any
large org signs a contract with a small startup like us is: are you going to
sell and shutdown?

This has happened so many times that it can really affect how small companies
are preceived as too risky too deal with by larger customers. _sigh_

------
rusk
Wow, twitter really are just _" the worst"_ these days.

Say what you like about Facebook but it does at least seem as though somebody
is calling the shots in there.

------
tw1010
Companies can do this, there's no regulation against it and there probably
shouldn't be any. But there are going to be consequences and they're probably
not good for the industry at large. Moves like this fundamentally change how
much trust consumers have in SaaS apps. The risk factor of using new apps
fundamentally changes, if ever so slightly. Unfortunately, this arguably works
in favour of the companies that do this. By reducing trust in startups,
there's fewer users who will take the leap on new actors.

Twitter and Google wins by having fewer competitors, but the customer loses
doubly so. If Facebook acquires every fifth social network, followed by
immediate shutdown, users will eventually stop using them, hence less
competition as a whole to FB.

------
sbr464
Wow, who would do this, especially since all of the brands mentioned are
probably also Twitter customers.

~~~
pedalpete
They're twitter users, unlikely twitter customers.

~~~
sbr464
I assure you they are all paying for promoted tweets, ads, or using expensive
social media tracking tools to monitor usage, respond to requests etc. in one
way or another.

Twitter is a pretty big part of any modern online company’s strategy, whether
they like it or not.

------
snowwolf
One thing that doesn't make sense is surely a service like this greatly
benefits from the network effects of being able to see activity across
multiple sites, so if Twitter wanted to continue using the technology,
wouldn't they have been better off leaving it running?

~~~
make3
I'm not convinced of your premise, spam is adversarial & spam detection is
just pattern recognition. Twitter likely doesn't want to allow people to be
able to test against their spam shields by spamming other people

~~~
snowwolf
As I understood it they were using machine learning models to identify
attacks. Abuse and fraud strategies are constantly evolving, so the benefit of
having many clients is that once you see a new attack strategy at one, you can
instantly protect all your clients from that threat.

In your own example

"Twitter likely doesn't want to allow people to be able to test against their
spam shields by spamming other people"

Yes, the attacker may find a new pattern that can bypass the shield, but as
soon as that pattern is added to the machine learning recognition, they are
then protected against it on their own system.

"spam detection is just pattern recognition"

The more patterns you can analyse, the better your recognition?

------
Rjevski
I hope this ends up litigated. The idiots at Twitter need to understand what
accountability is about.

------
sidcool
The call to wind down business with existing clients is a worrying trend. Why
would I avail services of a startup if there is a risk that upon acquisition,
the startup's new parent company won't service us anymore. It's a risk.

------
mullingitover
I'd really like names to be named here, so everyone knows who never to do
business with in the future.

~~~
zaarn
Basically, don't do business with Twitter.

Don't give them your data, don't sell your ads on twitter, don't develop for
Twitter.

There are alternatives.

~~~
ry_ry
in twitter's case the closest alternative is standing in a crowded street,
barking disjointed sentences at passers by in the hope somebody acknowledges
you.

~~~
Rjevski
Do you _actually_ need to bark at passers-by, or do you think you need just
because every other dog is doing it?

Replacing Twitter is simple. Have a proper support channel (Intercom, etc) so
your customers don’t need Twitter to get support from you, and use your
existing marketing tools (email, push notifications, etc) to keep in touch
with them. Done.

------
tripzilch
I'm pretty sure that "to wind down" is not a proper euphemism for "to shut
down".

"Winding down" literally means that before shutting down, there is an
incremental process with the explicit purpose to smoothly move into the
transition. There wasn't anything like that, not even a little.

What's especially rich is that Engineering VP Mike Montano nonsensically
repeats this phrase _after_ the fact: "we made the difficult decision to wind
things down right away". That's plain bullshitting.

He should have said, "we made the difficult decision to _shut_ things down
right away". But that would have sounded super irresponsible! But it _was_
super irresponsible, and describing it differently is just dishonest.

And that wasn't accidental, this entire situation is exactly about the
difference between "winding down" and "shutting down".

Why do we allow people to communicate in this way? (publicly and/or from a
position of leadership). Twisting words in plain sight. There is no good
motivation behind this, it represents messed up priorities between
appearance/saving-face and responsibility.

I don't know anything else about this Mike Montano. Maybe he's a very good
manager when not talking like this. But leadership comes with responsibility
and when you twist words like that, it's only because you're trying to wriggle
out from underneath the responsibility.

------
jeswin
Lessons:

1\. Trust small, third-party, closed-source SaaS vendors at your own peril.

2\. Do not spend/waste your time integrating with random third party services
of questionable/unknown sustainability.

~~~
dingaling
3\. Have a test environment where you can delete routes to external services
and assess the failure modes of your application.

Does the whole page hang? Do UI elements disappear ot overlap others? Do JS
error messages surface in the UI? Can the user continue with core
functionality or at least receive an explanatory message?

------
fauigerzigerk
It makes me wonder. Does anyone actually read the contracts they sign?

I usually hate it when people blame the victims for being so stupid whenever
something bad happens. But some of the victims in this case are corporations
with lawayers.

So either this really isn't a mission critical service and they just took a
calculated risk, or these lawyers haven't done the job they were hired to do,
or Twitter is in breach of contract.

------
laumars
Is this really that out if character for Twitter as an organisation though?
They have a history of being rather hostile toward developers and their own
APIs. Which is ironic considering the nature of the platform.

I would say we should use this as a sign to migrate away but we've had enough
warnings already. I guess Twitter is too big to fail?

~~~
jsilence
and this is yet another example why we need protocols and not APIs.

~~~
recursive
I treat the two as synonyms. What do they mean?

~~~
X6S1x6Okd1st
You could say that an API implements a protocol, but a protocol is much more
general. For instance IRC is a well known and widely implemented protocol.
Facebook messenger implements a protocol for client server commination, but
there are few or none implementations if either side and the main
implementation can change the protocol arbitrarily and without notice.

~~~
cortesoft
Right, but then wouldn't every API that has documentation also necessarily be
a protocol (that might only be implemented by one provider?) If the idea is
that you could switch to a different provider if you were using a protocol
instead of an API, that only helps you if there is more than one provider of a
protocol. In this case, someone else could take the public API docs and
implement their own service that responds to the same API.

~~~
setr
I would imagine the relationship similar to programming languages and
standards; If there's no official standard, then usually for anyone trying to
re-implement the language, they're "doing whatever the mainline version does".
ie anyone trying to reimplement python does whatever cpython does, bugs and
all, to claim "compatability".

But if a standard exists, that is, we've split the API from the primary
implementation of the API, then we as a re-implementer no longer need to
replicate bugs and all; we just need to do as the standard says, and anyone
relying on cpython bugs is, well, at their own fault. Compatibility claims are
no longer dependent on implementation-specific details.

And of course, it doesn't matter until a second implementation begins to
appear; there's no reason for cpython to follow (or have) the standard unless
it actually wishes to be compatible with jpython. At the same time, it makes a
lot less sense for a second implementation to appear as well, as they have to
put a lot of extra time and effort into matching cpython's implementation
details (as far as they matter; a difficult determination itself), and this
also makes it more difficult to produce an alternatively-optimized
implementation (focusing on say, memory-usage instead of speed or whatever).
So it's also a chicken and an egg problem.

So ideally everything would be an official standard, and we'd put little to no
effort into simply matching whatever the popular implementation is at the
moment, and we could criticize the mainline implementation for failing to
adhere to the standard.

In this particular scenario, to replace smyte everywhere, transparently, you'd
both have to adhere to the API and whatever details incidentally exposed by
the API, because you have no protocol to actually reference (the protocol is
whatever smyte was doing). But thats also particularly difficult here, since
you can't even test it against an implementation of smyte anymore..

------
wmil
Given the speed there might have been a legal reason.

Perhaps they were in violation of either the GDPR or their customer data
agreements.

------
sidcool
This is also a responsibly of the founders of Smyte. They cannot just leave
the customers hanging.

------
jnbiche
Unless it turns out that Twitter blatantly lied to Smyte's founders regarding
how they would shut down the service, I personally will never use any service
or software that these founders create in the future. This is absolutely
unbelievable, even for Twitter.

[I've removed the founders' names, even though I strongly disagree with the
characterization that I was somehow acting out of rage. My intent was rather
consumer advocacy. I've had a service cut off in a similar way, and I view
such behavior as being unethical. Fine if you disagree, but please don't
characterize my emotional state that led to this post (which itself is in
violation of HN's admonition to "Assume good faith")]

~~~
dang
Bringing in people's names in order to attack them is a breach of this site's
civility rule. You needn't stoop to that in order to discuss issues of
substance in this story, so please don't.

All: please keep the internet rage reflex off HN. Same for the online shaming
culture.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

(Edit: I didn't mean that you were feeling rage, but that by posting this way
you were stirring it up. In any case, thanks for removing the names.)

~~~
davedx
So all the constant anti-Tesla articles blaming Musk personally on here are
fine, but these guys are somehow above being named despite their names being
public record? Why is this different?

~~~
dang
Personal attacks on anyone aren't fine on HN, but when there's an entire
industry of articles about them it gets a lot harder to enforce the rule.

~~~
icelancer
I haven't seen a single anti-Musk personal attack deleted or addressed, yet I
saw this. A lot harder != impossible.

------
pgroves
Based on the email from the VP, my wild guess is they figured out that Twitter
operating Smyte would make Twitter itself not GPDR compliant (or some other
big regulation), and they didn't figure it out until after the deal was done.

------
ourcat
Shocking in many ways. But not in others.

I predict some back-pedalling later today and access switched back on and
properly 'wound down'.

If not, customers with contracts should have every right to angrily sue.

------
Zooper
The reason companies shouldn't use each other's proprietary code for basic
operations: it's insecure, there's little-to-none of your interests being
protected.

------
ChuckMcM
Wow. That sort of thing can be the effect of little to now negotiating
leverage by the acquired company. They only raised 6.3M according to
Crunchbase.

------
jakoblorz
Note to self: add "to smyte" to vocabulary

------
ComodoHacker
Was there something so dangerous for Twitter in Smyte's business model that
required such a brutal shutdown?

~~~
Rjevski
The fact they had an actual product they were proud of? Clearly that’s
considered unacceptable by Twitter’s standards.

------
songzme
Playing devil's advocate. Here's a possible scenario:

1\. Smyte had a security flaw that could potentially (or already did) affect
twitter customers.

2\. Leaving Smyte service on could do alot of damage, so turning it off would
be the sensible thing to do.

3\. Telling the world about this security flaw would create a really bad
backlash.

------
throwawayqdhd
I'm always surprised that Twitter has managed to stay in business so long.
Anaemic growth, massive spam and bot accounts, an advertising platform that's
hostile to most small advertisers, a user-base that makes YouTube's comments
look downright gentle, constant hostility to third party developers...and yet
it continues to chug along.

~~~
cpeterso
Buzzfeed posted an article called "How Twitter Made The Tech World's Most
Unlikely Comeback" just today:

[https://www.buzzfeed.com/alexkantrowitz/how-twitter-made-
the...](https://www.buzzfeed.com/alexkantrowitz/how-twitter-made-the-tech-
worlds-most-unlikely-comeback)

 _even as those eulogies were being published, things started changing.
Twitter began beating earnings expectations. Star ex-employees trickled back
in, finding a new, more positive internal culture than the toxic one they’d
left. Advertisers came back too, as did users. The company finally began
addressing its trolling problem. And its stock, once unappealing to analysts
like Nathanson at $14, is now trading above $46.

It’s still somewhat taboo to say it, but it’s no longer possible to deny it:
Twitter is making an unexpected, somewhat miraculous comeback. It is the first
major consumer social company to lose users and start growing again in a
meaningful way._

~~~
jumbopapa
I really think they need to thank President Trump for that. He saved Twitter.

~~~
ceejayoz
They did, in December, when they exempted "government entities" from their
abuse policies. [http://www.businessinsider.com/donald-trump-has-immunity-
in-...](http://www.businessinsider.com/donald-trump-has-immunity-in-twitter-
new-abuse-policy-2017-12)

~~~
vkou
It's not illegal/abuse/threats of violence when the President does it.

------
Hendrikto
This is why I don‘t really like SaaS for important parts of infrastructure.
It‘s not under your control.

------
B1FF_PSUVM
Off-topic, but twitter related: has the site fallen off search engines?

Yesterday I wanted to look at some joke account (by correctly spelled name),
Google yields nothing from the domain, Twitter wants me to sign up to search
there ...

(Not a habitual user, or even reader, as you may guess.)

------
pavlov
It’s fairly common in so-called acquisitions of startups that the the acquirer
never buys the company stock. This leaves annoying liabilities like existing
contracts in the selling company which will be shut down. The buyer might
purchase IP so that the company to be shut down can pay out their investors
and old customers.

However that situation would usually never be described as an acquisition by
the buyer. (The seller might put out an announcement that says “We’re proud to
be joining FooGleZon, it’s been an amazing journey!”)

In Smyte’s case, Twitter PR does use the word “acquire”:
[https://blog.twitter.com/official/en_us/topics/company/2018/...](https://blog.twitter.com/official/en_us/topics/company/2018/CommitmentToHealth.html)

That sounds like something else than an “amazing journey” non-acquisition. But
IANAL, just speculating out loud.

------
ojuara
I don’t know how to run a business. But, I know how not to run a business.
That’s one way.

------
kbyatnal
Funnily enough, they had a HN "Who's Hiring" post less than 3 months ago.

[https://news.ycombinator.com/item?id=16739153](https://news.ycombinator.com/item?id=16739153)

------
TheCapeGreek
Wow, even Smyte's website was gutted. Bold move, to say the least.

------
jrochkind1
> We had a 3 years contract with them and they just disappeared overnight. No
> communication at all, they just turned their servers off

Hmm, is the new owner not bound by that contract?

------
noobermin
Tell me again how the silicon valley system works and works well at what it
does.

Create a product, gather funding.

Attract customers.

Get acquired, founders make bank.

New owners trash the product, stranding customers.

Tell me again how this is good system not for only making money but actually
providing value to customers.

~~~
ta1234567890
Going off on a tangent here, but it's interesting how the first thing on that
list is "create product".

I would argue that most new companies (at least started by and solely run by
technical founders) fail exactly because of that.

Product-oriented people tend to think that the success of a business is due to
its product(s), when the reality is businesses succeed due to their ability to
sell their products. Hence, the most important thing is actually being able to
properly find and target an audience, which should be the first step of
starting a business.

The second step of starting a business should be to test marketing channels to
make sure the selected audience can be effectively/efficiently reached. Only
after this has been properly done should a product be ideated, based on the
knowledge acquired about the target audience and the marketing channel(s) that
work.

Just my 2 cents to the HN community after having learned this lesson the hard
way, failing multiple times and starting several businesses.

~~~
triangleman
Yes, yes, and yes.

What's more, you could be product focused and still produce bad products. I
just left a startup that seemed to have really bad decision making in areas
like "what feature should we build first/second/third", and major areas
needing design and careful thought were instead treated as an afterthought.
But the tech was solid and lots of work was getting done, so... good enough?

------
consultSKI
this is typical Twitter. no surprise.

------
kirRoyale
Remember when the Trending section wasn't advertisements?

------
douglaswlance
Twitter is starting to seem awfully authoritarian...

------
dvtrn
I'm fully prepared for whatever negativity comes with this comment because
it's being asked from a place of genuine curiosity-but yes there some snark
baked in:

Does the rapacious acquisition game of Silicon Valley and seemingly increased
rate of consolidation of startups to established giants feel weirdly like he
days of Ma Bell to anyone else?

It seems like more and more There's tales of these acqui-hires with promises
that the aquirer will present some kind of "polished" alternative but under
the larger brand and I've found quite frequently the promised offer-if ever
shipped at all-is far less superior and far more self serving to the brand
than what was previously offer by the acquired when they had more on the line
by operating as a startup.

Or am I just a cynic?

~~~
dingo_bat
Acquisions have been the principal way big companies have always done
"innovation". Nothing new here.

~~~
odbol
Also the principal way they remove competition.

------
alex_duf
I wonder if twitter uses npm...

~~~
dentemple
They utilize SSR React, so, yes, yes they do.

------
tempodox
If the umpteenth demonstration was needed that you can't rely on any 3rd-party
internet service, here it is.

------
dmead
npm? that sounds way worse than the others

~~~
threeseed
We don't know what happened to the other ones though.

------
tjpnz
What on earth do they mean by "safety"?

------
itronitron
if you aren't the customer then you're the product... no wait, they are the
customers.

------
tomelders
Site's that block users for not allowing cookies and tracking are essentially
paywalled. Communities like HN should do their part to penalise sites that
pull these kinds of tricks. If not us, then who?

So, anyone got a non-paywalled version of this?

------
r3vrse
In the next installment of "How To Make A Successful Exit While Not Give A
Flying Füçk About Your Existing Customer Base Or Industry Rep. That Got You
There 101"

Is anyone else as tired of this schism of material gain over all other
considerations? If the tech industry wants to end up with the same ill repute
as Wall Street this is certainly the right way to go about it.

~~~
iamdave
This is it? I've often considered the abuse of 1099 workers to be the thing
that one day catches up with tech as a precursor to its own "Enron" moment.
Maybe it'll be both.

~~~
r3vrse
How visible is that issue though? How likely to be digested by the masses?

"Tech company takes money and screws over customers" is a headline that writes
itself.

------
flukus
The screwed over themselves as soon as the decided to rely on third party SaaS
solutions for core infrastructure.

How many more examples do we need for this to sink in?

~~~
laumars
You're getting downvoted but if people look past your inflammatory tone there
is a good argument to be made.

Back when it was more traditional to buy a software licences and self-host
rather than pay a rolling subscription to an _X_ aaS, a service going dark
just meant that you were shut out of updates but not the software itself. Sure
you'd still eventually need to migrate away but at least you could then do it
on your own terms.

~~~
bartread
> Sure you'd still eventually need to migrate away but at least you could then
> do it on your own terms.

I think people often underestimate the value of being able to manage their own
destiny. They key point here is "manage" though: you have to actually _do_ it
- it's an active process.

If, on the other hand, you simply stick your head in the sand and ignore the
situation it will eventually get to the point where it becomes extremely
difficult and expensive to deal with due to a variety of factors such as no
clear migration path, loss of institutional knowledge, and so on. I name no
names, but worked for one UK company in particular with form here.

I'm not making an argument against SaaS (we use a few of them), but it's
important to understand the trade-offs you're making in terms of ceding
control.

~~~
x0x0
Sure, but a service like Smyte requires significant expertise to make work
well and run. Pretending you can snap your fingers and implement a Sift
Science or Smyte is ludicrous.

Further, some of the biggest value they will provide is being able to see
transactions between vendors. So eg calculating an IP address reputation score
by seeing bursts of fraudulent purchases. That is impossible to do in house
only.

~~~
bartread
> Pretending you can snap your fingers and implement a Sift Science or Smyte
> is ludicrous.

But that's not what either I or the GP were talking about: we were talking
about hosting a product that you'd purchased from another vendor on your own
infrastructure.

~~~
jamespo
But this sort of systems is particularly suited to being hosted by a third
party as you benefit from a network effect via the other customers.

~~~
laumars
I'm not going to argue that these sort of problems aren't ideally suited for
SaaS however it is still worth remembering that systems like this have existed
before SaaS took off. eg you would host the API yourself but cron a db update
every _n_ hours or days. So if the service went offline you still had a local
database and API.

The obvious downside to that is you don't have real time updates (as well as
the usual other benefits that SaaS brings), which is why SaaS suites this kind
of business well. But it can be done and was sometimes done this way "back in
the old days".

~~~
dmurray
Having the db freely available makes it a lot easier for a bad actor to figure
out how to avoid getting in it, though. So situations like this are suited to
having a centralized owner of the data who can perform rate limiting, restrict
to its paying customers, etc.

Distributing the database to customers is still workable despite this drawback
- antivirus systems are a common example.

~~~
laumars
> Having the db freely available makes it a lot easier for a bad actor to
> figure out how to avoid getting in it

I don't agree with that. Having the database only allows bad actors to see if
they've been added to the database - not the business logic of how they ended
up in that database.

You could argue that knowing you've been added to the database is still an
advantage, which is true. But those same bad actors could still test the
system with SaaS solutions too. eg when one acquires stolen bank cards it is
common to use those details to make small payments for things like food
delivery to test if those card have been blocked or not.

~~~
dmurray
Having a fast, cheap and detailed feedback loop is useful for testing for the
bad guys, like in any programming process. For example, in the credit card
case, you can check thousands of cards without needing to find a merchant who
will take your thousands of small, frequently-declined transactions. Or you
can check the details for cards you haven't fully compromised (maybe the guy
selling you stolen bank cards only gives you a portion of the details until
you've paid him).

~~~
laumars
Oh absolutely. Like many things security related, it's not always about
stopping them but rather slowing them down enough so that it doesn't become
cost effective.

------
tzahola
Daily reminder that every SaaS is a liability out of your control.

------
sonnyblarney
There needs to be a class action suit.

This is BS.

~~~
mortenjorck
Fortunately for Smyte, its former customers probably all signed arbitration
agreements.

------
dmitrygr
And this, boys and girls, is why you never use any service of which you cannot
host a fully working copy yourself.

~~~
nerdbaggy
What is your suggestion for a self hosted client fraud platform?

~~~
dmitrygr
Hire a few competent engineers and make one

~~~
not_kurt_godel
You could do that, and they'd quickly tell you that fraud detection requires
vast datasets that your company almost certainly cannot generate or maintain
in-house. Then what?

~~~
dmitrygr
Find a way to live without it I guess. Certainly that would be better than
having your entire production stop working because somebody else decided to
take a multimillion-dollar check and run with no concern for you or contracts
you had signed

~~~
not_kurt_godel
If you can afford to cut multi-million dollar checks for a fraud/abuse
prevention service, then you can also afford to retain a legal team who will
recoup your losses and then some when those contracts are broken.

A temporary production outage due to a rare situation is far preferable to
being at the perpetual mercy of hackers constantly pwning whatever half-baked
homegrown security system you forced your engineers to implement with
inadequate support & resources. Real businesses don't operate on "I guess"
solutions and they don't just fork over huge chunks of cash willy-nilly
without doing at least some cost-benefit analysis and making sure they're
protected in case of contract breach.

------
ratsimihah
Here's a cookie, I'll take your life in exchange.

------
saudioger
Based on this I assume the Smyte founders were retiring on a tropical island
before the ink even dried.

------
erikb
I want names out there. Who is responsible for this!!!! Who was the CEO of
smyte? Who is the manager at Twitter that gave the comand to shut down
everything?

------
benatkin
"Smyte" would be a good name for something that kicks users off a social
network in a dystopian novel. "Smite" appears dozens of times in the Bible and
means to strike someone, sometimes fatally.

------
nstj
It surprises me that this article has so many votes when nowhere has it
actually provided any details of either the commercial terms of the
acquisition OR whether any Smyte customers actually lost any money from
_prepaid_ service.

Sure, it sucks that people lost an API they were using, that much we know, but
hundreds of votes for a “bad” commercial transaction on which we have no
details?

The way I see it, a number of Smyte users just received a free billing period
of API access?

------
nstj
I may have just stumbled in to the middle of this, but why the brouhaha?

Did customers pre-pay for Smyte service and not receive it?

~~~
danso
FTA:

> _According to reports from those affected, Smyte disabled access to its API
> with very little warning to clients, and without giving them time to
> prepare. Customers got a phone call, and then – boom – the service was gone.
> Clients had multi-year contracts in some cases._

