
Riak and the demise of Basho - bryanrasmussen
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2017-August/019500.html
======
andrew_deane_
Today bet365 have signed the agreement to purchase all Basho IP. We expect the
agreement to be ratified in the US courts next week. Once cleared, our
intention is to open source all code, help rebuild the community and
collaboratively take the development of RIAK forward.

In the coming weeks we will hopefully answer the questions people have and
will be calling on the community to help forge the initial RIAK Roadmap.

One of the initial questions we have for the community is which OSS license
would people like applied to the code? Our thought is the most open and
permissive.

~~~
kuwze
GPLv3 and Apachev2 are the best in regards to dealing with patents.

~~~
atonse
Please don't do GPL – It's a no-go and too restrictive for a lot of
organizations.

Apache 2 or MIT would be the most permissive.

~~~
pron
If your organization uses any of: Linux, Android, emacs, gcc or any GNU
utility, OpenJDK, or MongoDB, then GPL is clearly _not_ a no-go. GPL only
poses issues when the product is a library. For standalone products or ones
with linking/access exceptions like OpenJDK, GPL poses no issues, and
organizations aren't afraid to use it, especially when they don't intend to
modify it.

GPL is a great license that ensures that improvements to the product are
shared back with the community. In fact, for a database like Riak, Affero GPL,
like the MongoDB license may even be better, as organizations that make
improvements to Riak but don't redistribute it but run it internally on their
servers would also be required to contribute back their modifications.

~~~
mirashii
> If your organization uses any of: Linux, Android, emacs, gcc or any GNU
> utility,

Two things wrong with this right off the bat. First, not all of these are
GPLv3, which is the version with the problematic patent clauses that scare
companies away. Second, it is not just in libraries that the GPLv3 is
generally accepted to pose potential issues. You need look no further than
Apple and the great investment they went through to avoid ever shipping GPLv3
software. The GPL kicks in as soon as you distribute the software.

~~~
pron
> First, not all of these are GPLv3, which is the version with the problematic
> patent clauses that scare companies away.

So, if Linux were GPLv3 you're saying that most companies wouldn't use it?

> You need look no further than Apple and the great investment they went
> through to avoid ever shipping GPLv3 software.

Most companies aren't like Apple.

> The GPL kicks in as soon as you distribute the software.

That's right, and that's a _good_ thing. If you distribute GPL software _that
you modify_ , you need to make your modifications public. But as Riak is not a
library, it does not infect any other component. If you distribute software
with (say, Android) Linux but don't modify Linux then the GPL doesn't infect
the other components of your software, as they're not actually linked (or
Linux has some explicit exceptions).

------
wjossey
In 2012, I worked on a nine month project to migrate a fairly large hot
dataset (10s of TBs) off of SimpleDB and onto Riak. At the time, we were
experiencing tens of hours of downtime a quarter, much of which could be
attributed to SDB.

My favorite story about that migration is that four months into the project,
we were moving along nicely and ready to start cutting over to Riak after the
Christmas holiday. My office had decided to go out to lunch our last work day
before everyone headed off for the holidays, and I received a call from a SDB
support team member informing me that we either needed to move off their
platform by Christmas Day or they'd have to shut us off (we'd start degrading
other customers). Fortunately we were nearly ready to pull off the cutover, so
we quite literally cut all of our traffic over on Christmas Eve (one of our
busiest days of the year).

Over the upcoming years, our database continued to grow and grow, and all the
while Riak trucked along. It wasn't always a smooth road, and we certainly had
our challenges from time to time, but I have yet to hear of anyone who has
used a massive 100TB hot database and not had to do work and maintenance.

Throughout the years I used Riak, it served me very well. I'm grateful to the
innovative work so many at Basho created during their tenure, and I'm glad to
see Bet365 attempt to steward the project to a new phase of its life. If I
could toast every last basho employee, I would. Thank you all!

~~~
kstrauser
> I received a call from a SDB support team member informing me that we either
> needed to move off their platform by Christmas Day or they'd have to shut us
> off

Wat?! That's an excellent reason to burn fields and salt them.

~~~
wjossey
I was / am pretty sympathetic to their situation. At that point, SDB was
already deprecated in favor of dynamo, and they had been asking us to move off
of SDB for over a year (before I had joined the company via acquisition). If
someone ever does an oral history of AWS, I'm sure that SDB would be the
"grand mistake" they made in their first few years. It was never ready for
prime time in the way it needed to be.

~~~
kstrauser
They still advertise it
[https://aws.amazon.com/simpledb/](https://aws.amazon.com/simpledb/) :

> Amazon SimpleDB is designed to integrate easily with other AWS services such
> as Amazon S3 and EC2, providing the infrastructure for creating web-scale
> applications.

If I hadn't heard your story, I wouldn't see anything there to steer me away
from it.

~~~
TheDong
It's not listed in the AWS console anywhere. That's enough to scare most users
off I imagine.

~~~
kstrauser
Huh, you're right. They still call it out in the DynamoDB FAQs though
([https://aws.amazon.com/dynamodb/faqs/](https://aws.amazon.com/dynamodb/faqs/)):

> While SimpleDB has scaling limitations, it may be a good fit for smaller
> workloads that require query flexibility.

------
cies
I think Riak properly materialized the NoSQL promise of fully scale-out
architectures. Have not seen an open source product that delivers on this (no
Mongo does not cut it for me). Sad to see they did not manage to make a
business out of it... The bet takers may be a good patron to the project, but
ideally it is governed by an independent shop that focuses on further
development and marketing.

~~~
_asummers
RethinkDB was good from a distributed systems perspective, but a nightmare to
maintain in production. Backups and restores would take 12+ hours on ten
gigabytes or so and slow queries would grind the whole system to a halt, which
isn't possible in Erlang based systems like Riak due to the preemptive
scheduling of the BEAM.

~~~
cies
Riak builds on theoretical foundation, laid out in Amazon's Dynamo paper. The
other NoSQLs did not have this theoretical underpinning, and so they just
offer a "best effort".

~~~
jchrisa
Dynamo is not exactly a performant or efficient model. It's the equivalent of
pulling all the distributed systems guts out and handing them to the user to
deal with. And the resulting toll is quantifiable:
[http://damienkatz.net/2013/05/dynamo_sure_works_hard.html](http://damienkatz.net/2013/05/dynamo_sure_works_hard.html)

~~~
kstrauser
Damien's a very smart guy, but I don't think I agree with him here:

> Within a datacenter, the Mean Time To Failure (MTTF) for a network switch is
> one to two orders of magnitude higher than servers, depending on the quality
> of the switch.

A switch is highly unlikely to fail. They seem to be bulletproof. But having
worked with a datacenter (on the engineering team of an early AWS competitor),
switch _misconfiguration_ was all too common. Maybe a tech accidentally plugs
in the wrong ethernet cable and forms a switching loop. Maybe someone fat-
fingers a tag and a broken VLAN gets automatically deployed to 10,000 nodes.
Either way, the _switch_ is alive, well, and pushing packets - but they're the
_wrong_ packets and the result is indistinguishable from hardware failure to
the end user.

At datacenter scales, these things happen... not infrequently. If you engineer
your database to expect that netsplits are rare, you're going to have a bad
time.

~~~
kyledrake
VLANs were the bane of my existence when I had to figure out how to deal with
them. I don't envy anyone whose job is to manage them on switches for a lot of
servers.

------
michaelbuckbee
I haven't been following this, but from what I can gather:

\- They raised $60mil (the last round being a Series G for $25mil!)

\- They're being sold for what I can only this is a pittance / acquihire by
Bet365

\- Bet365 is buying them because they're heavily invested in Riak internally

\- They're open sourcing the code (likely because they need help)

I'm not trying to disparage anyone here, but this makes my blood run cold.

Is Riak so critical to Bet365 that the right move was to _buy the company_
versus switching to a different storage system?

Is there a NoSQL vendor that's doing well out there?

Should we all just be using Postgres anyway?

~~~
benwilber0
> Is Riak so critical to Bet365 that the right move was to _buy the company_
> versus switching to a different storage system?

Quite possibly, yes. Data has _incredible_ inertia. Not only because of the
storage system used, but also all of the years of ops tooling, application
integration, etc. that surrounds it. I don't know anything about Bet 365, but
if they had any non-trivial amount of their business data locked up in Riak
then from a cost/risk analysis they may have decided that purchasing the
company was the safest and cheapest option to protect their own company.

~~~
nerpderp83
I thought data had gravity.

~~~
wlesieutre
For those not in on the joke, "Data Gravity" is a concept proposed by Basho's
former CTO: [https://datagravity.org/](https://datagravity.org/)

------
i_have_to_speak
Coverage by TheReg has more details:
[https://www.theregister.co.uk/2017/08/25/bet365_to_buy_basho...](https://www.theregister.co.uk/2017/08/25/bet365_to_buy_basho_release_code/)

~~~
carapace
"Bankrupt bit bin biz Basho Technologies" and this gem.

------
ulkesh
This really doesn't surprise me. My experience with Riak (admittedly in
2012-2014) was that it did not live up to the hype. As strictly a key-value
store, it wasn't bad, but toss in 2i, etc, and it quickly became problematic.

Open sourcing the software will surely be a good thing, though. I'm eager to
see where it leads.

~~~
aetherson
I worked with riak pretty extensively in that time period. I'm in more or less
the same boat. It had cool features, but they didn't make up for the overall
kind of slow performance and limited feature-set.

(Admittedly, our use of riak was clearly a case of premature optimization: we
didn't have anything like the scale to require an available data-store.)

~~~
macintux
As a technical evangelist for Basho, I would encourage sales prospects to
think carefully about what they really needed. One of the problems trying to
sell Riak is that, unless you know you need the availability it can offer at
the scale it can support, you're probably better off choosing something with
more features and less scale.

And, although clearly we're swimming in more data than ever, not all that many
people have both massive data sets and a burning need to never lose any of it.
A lot of data can be lossy without any real consequence.

------
menacingly
In my experience with Riak, it absolutely lived up to its promise of scale,
even if single-request speed was never quite as zippy as anyone would have
liked.

However, the Dynamo design is just borderline unpalatable from a developer's
perspective. I enjoy being aware of exactly the trade-offs being made, and
Basho was always really open about it, but I don't blame anyone for not
wanting to dig deep in the weeds to make it work for fairly common use cases.

------
_Codemonkeyism
Still amazed that $66m in funding was not enough.

------
joaodlf
I have no experience with Riak, but I once went to a conference (a very short
while ago) and saw a presentation by a higher up of Basho (I won't disrespect
the person by naming them), in short: It was a very generic, bland and
positively abstract presentation about big data. Other presentations were
highly technical, so I was excited to get to know Basho as well - I came out
of that thinking "wtf was this guy talking about?". It's obviously not a
reflection of what Basho stood for, or even what the individual in question
stood for, but that's the only memory Basho has left in me.

~~~
macintux
A shame you were left with that impression. For quite some time, from top to
bottom, Basho had quite a reputation for solid technical talks. The
distributed systems conference RICON was quite excellent.

(Disclaimer: former Basho technical evangelist & engineer.)

Suggestion for something much more meaty:
[https://youtu.be/3SWSw3mKApM](https://youtu.be/3SWSw3mKApM)

~~~
nerpderp83
I suggest going through all the RICON talks,
[https://www.youtube.com/user/BashoTechnologies/playlists](https://www.youtube.com/user/BashoTechnologies/playlists)

This talk on LVars by Lindsey Kuper blew my mind
[https://www.youtube.com/watch?v=8dFO5Ir0xqY](https://www.youtube.com/watch?v=8dFO5Ir0xqY)

~~~
macintux
Lindsey rocks. She had a recent blog post hit HN.

There's a fair number of RICON videos on Vimeo too. One of my all-time
favorites from Joe Hellerstein:
[https://vimeo.com/53904989](https://vimeo.com/53904989)

------
simonvc
More importantly, they acquired Russel Brown (
[http://basho.com/posts/author/russell-
brown/](http://basho.com/posts/author/russell-brown/) )

~~~
russelldb74
Ah they didn't. I went to work for them after I was let go by Basho in
January, but have since moved on to general contract work on Riak. I am lucky
enough to remain in contact with the people at bet365. My take, for what it is
worth, is that this is a very generous gesture from bet365 and it helps the
whole community. bet365 really hide their light under a bushel in terms of how
big a UK technology success story they are. I'm looking forward to working on
the newly open sourced replication code.

~~~
sitkack
You could make a killing getting MDC to work between Riak and BigTable [0]

[0] [https://cloud.google.com/bigtable/](https://cloud.google.com/bigtable/)

------
rasjani
I've no experience w/ Riak what so ever but years ago I was contacted by hh
for an open position at Basho. They where looking for field support engineer
to a local gaming company. They would hire me and put me directly to client.
Maybe i'd have gotten some training on the Riak, but it was never mentioned
during the rounds. All interviews where done remotely. Eventually I did say no
to the offer.

Some time later, even the guy who interviewed me was out from Basho.

Idk, whole process just fell very weird.

~~~
wlesieutre
I knew a couple of folks in Client Services at Basho, one of whom later moved
over to the engineering side. From what he told me, the client services group
was a great team to work with and had great management.

~~~
wlesieutre
Missed edit window, but also definitely trained on the job. Neither one had
prior experience with Riak.

------
cies
Seems to be a little too much traffic for their database. Here's a link to a
cached version by Google.

[http://webcache.googleusercontent.com/search?q=cache:http://...](http://webcache.googleusercontent.com/search?q=cache:http://www.casinoguardian.co.uk/2017/08/25/bet365-acquires-
nosql-database-developer-basho-technologies-remaining-assets/)

~~~
StavrosK
That loaded very slowly for me as well. Here's a prettified IPFS-hosted
version:

[https://www.eternum.io/ipfs/Qmeh3XePHFqRxFiK56Apy3WdVXNw2YVG...](https://www.eternum.io/ipfs/Qmeh3XePHFqRxFiK56Apy3WdVXNw2YVGXWR1Er3t28fTPE/)

------
cookiecaper
Basho has silently gone dark since around Easter, with no indication of
anything on their site; it still looks like Riak/Basho are an active, living
company. Despite being hosted on basho.com, this is just a post to a mailing
list, still not an official account/announcement of anything.

I have a client with a support contract and while I don't know for certain
that they didn't contact _someone_ within the organization, I know that most
people, including myself, found out only when someone asked if we could
contact Basho support and they were informed that it wasn't there anymore.

~~~
ErlangSolutions
FYI Erlang Solutions now offers Riak Support

------
unkown-unknowns
On a side note that has to be the longest e-mail footer I've seen in a good
while.

Reminded me of a usenet post from years ago but can't find it now, anyone know
which one I mean?

While looking for said usenet post I found something else though, here:
[https://www.slashdot.org/story/18304](https://www.slashdot.org/story/18304)

~~~
nailer
Indeed. Martin: just use a link. Law firms do it so you can too.

------
robotmay
I've always liked Riak, it seems very sensible. Having more open source stuff
related to it will be a good thing.

------
ams6110
Wow that one slipped by me totally. I had no idea Basho was in trouble (though
I'm not a Riak user, I have used some of their other erlang projects including
webmachine and lager).

Glad to hear their code will be open-sourced.

------
arianvanp
Seems like a good fit. Bet365 is deeply embedded into the Erlang community.

~~~
cies
Here a link to their Github:

[https://github.com/bet365](https://github.com/bet365)

Not too much Open Source code released by them. Mainly some forks and two RPC
libs.

~~~
flurdy
That is only one avenue to open source contributions. They may contribute via
individual commits directly to other libraries such as
[https://github.com/basho/riak](https://github.com/basho/riak) etc

------
innocentoldguy
This is great news! I've spent the last two years working with Riak. I love
its admin tools, and Basho engineers were the best to work with. I'm glad to
see Riak live on.

------
EGreg
I thought Riak was the most impressive and useful open source database until
CockroachDB. Would like to see it continue!

------
adityapatadia
I evaluated it in early 2011. At that time all NoSQL dbs were new in market.

For me, their restriction on usage in cluster mode, very less support for all
languages and lack of file storing abilities like GridFS of Mongo made it
unattractive.

Eventually Mongo seemed good choice because it was completely free for any
kind of use with amazing features and lots of libraries. I am glad I made that
decision :)

------
zokier
If you are going to open source Riak, then maybe consider donating it to
Apache Software Foundation? Maybe not immediately, but in the long term it
feels like it would make sense.

------
sitepodmatt
Ironic that the posted link returns a 500 "Error establishing a database
connection"

~~~
jefozabuss
The link points to a news site which probably is not that well provisioned.

~~~
notyourday
State of the internet in 2017: News sites that does not use caching. In 2017.
Let that sink in.

