
RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Open Source (2016) - tosh
http://sagemath.blogspot.com/2016/10/rethinkdb-sagemath-andreessen-horowitz.html
======
cyberferret
Interesting to see RethinkDB being re-posted on here again, even if it is a 3
year old article. We use it extensively in our background worker processing of
our SaaS and it has worked magnificently for years. I even write a small HN
related fun project with it a couple of years back [0].

I haven't kept up with how it is travelling as an open source project, but I
am hoping that community interest is still strong and that the project will be
around for a while yet.

[0]- [https://hackernoon.com/tophn-a-fun-side-project-built-
with-v...](https://hackernoon.com/tophn-a-fun-side-project-built-with-vue-js-
and-rethinkdb-f22159641c1b)

~~~
drdaeman
> it has worked magnificently for years.

It surely did the job, but unfortunately, I have different experience.

\- It's very space-ineffective. One particular thing I've discovered is that
it's very ineffective for storing numbers. A strings are much more space-
efficient
([https://github.com/rethinkdb/rethinkdb/issues/6304](https://github.com/rethinkdb/rethinkdb/issues/6304)).
However, there must be something else. I'm slowly moving to PostgreSQL
(experimenting with CockroachDB as alternative) and maybe that's just me, but
similar datasets with similar indices require significantly less disk space
there.

\- It dies under heavy load. That could be a tolerable issue - I can throttle
updates my app generate, but there's no way to control replication speed.
Trying to add replicas to some of larger tables reliably kills the destination
node for me.

\- There's no way to limit memory, and optimizing for this is hard. A node
with 64GiB RAM lives okay, but the one with only 16GiB gets periodically OOM-
killed (cache size on that node is set to mere 6GiB).

\- Sometimes, it has some weird cluster discovery delays, where restarted
nodes just sit and don't connect to their peers for minutes, even though
they're reachable and there are no network issues. This is not a problem for
large clusters, but I have only 3 nodes (cost constraints), where a loss of
one moves system into partially degraded state as replicating all tables is
too wasteful. Tolerable, but not fun.

> how it is travelling as an open source project

Unfortunately, I'd say it is dead. There is some discussion activity, but
almost no code activity.

~~~
cryptica
I'm launching a scalable open source cryptocurrency payment system using
RethinkDB.

It's amazing how well it works and scales on Kubernetes at the click of a
button. There is a video of it on my project website
[https://crypticle.io/](https://crypticle.io/) I intend to keep using it for
as many future projects as possible.

AFAIK, the business did not survive because the product was too good and
didn't have enough problems to charge support for.

Actually this is a big problem in open source and that's why I'm involved in
cryptocurrency now.

~~~
drdaeman
> It's amazing how well it works and scales on Kubernetes at the click of a
> button

Oh, just try to scale it down ;)
[https://github.com/rethinkdb/docs/issues/958](https://github.com/rethinkdb/docs/issues/958)

Deleting from system table is okay, but messing with server tags (when the
setup is heterogeneous) is not exactly fun.

To be fair, that's a very minor issue. Perfectly manageable, just a slight
inconvenience when you figure things out.

~~~
cryptica
It's not that difficult, you can use the reconfigure query to put all the
shards on one server, then you can scale down the other servers.

------
jrochkind1
Anyone wanna tell us what's happened to RethinkDB since then, and what's
happened to SageMath since then?

~~~
williamstein
RETHINKDB:

\- There is a recent discussion about whether or not RethinkDB is dead here:
[https://github.com/rethinkdb/rethinkdb/issues/6747](https://github.com/rethinkdb/rethinkdb/issues/6747)

\- Judging by new PR's and commits, Rethinkdb development slowed down
substantially in Jan 2016.

SAGEMATH:

\- SageMath development is still very active; maybe more so, due probably to
strong Europe grant support (OpenDreamKit):
[https://github.com/sagemath/sage/graphs/contributors](https://github.com/sagemath/sage/graphs/contributors)

SAGEMATHCLOUD: What this blog post is "really" about.

\- We renamed it "CoCalc" \-- [https://cocalc.com](https://cocalc.com)

\- I rewrote everything that depended on RethinkDB using PostgreSQL. Though it
was months of work, this was obviously a VERY good decision in retrospect!

\- I rewrote all the realtime sync functionality to not depend on the database
at all (just use a custom protocol over websockets), which is obviously a much
more scalable approach to this problem. RethinkDB (and anything like it) is
totally the wrong tool for implementing realtime sync for file editing.

\- Development on CoCalc is very active:
[https://github.com/sagemathinc/cocalc/graphs/contributors](https://github.com/sagemathinc/cocalc/graphs/contributors)

Disclaimer: I wrote this blog post and started Sage and CoCalc. I didn't
repost this here, and just happened to notice it while scanning HN.

~~~
xrd
I found your blog post fascinating.

Are there any decisions you wrote about here that you now feel differently
about (beyond rethinkdb, obviously).

Your take on open source and debt and lifestyle businesses is so interesting
and I'm curious if almost four years has changed anything for you.

~~~
xrd
More specifically, has any of the business advice you got turned out to be
surprisingly wrong?

------
williamstein
Discussed before
[https://news.ycombinator.com/item?id=12663599](https://news.ycombinator.com/item?id=12663599)

------
truth_seeker
RethinkDB was truly an inspiration in a reactive way of thinking and a
simplified solution for distributed sharding.

Change Streams in MongoDB are inspired by RethinkDB.

