
InfiniDB goes out of business - karavelov
http://www.infinidb.co/forum/important-announcement-infinidb
======
samcrawford
Sad to see. We were one of their early paying customers in 2009/2010 (we got a
great deal). Performance was fantastic over very large datasets, but the bugs,
storage requirements (very expensive SANs) and query limitations became big
problems for us. We moved away after a year.

Just recently one of our team has been looking at them again (following a
strong benchmark being posted at mysqlperformanceblog). So I went and checked
out the forums and saw it was pretty much in a similar state to when I last
used it four years ago.

Sad to see they couldn't make it work. The team was always really friendly and
quick to help with issues - good luck in the future.

~~~
imaginenore
There are so many scalable (and often free) databases out there. What do you
that can't be done in Cassandra or Mongo or Vertica?

~~~
swartkrans
Honest question, is MongoDB actually scalable? I keep reading about how it is
not for serious business[1][2][3][4]. Who is using Mongo at scale and what
sort of loads are they seeing and are the level of problems on par with other
solutions? As a developer I like Mongo's API, but I would worry about using it
given what has been written about it.

[1] [http://aphyr.com/posts/284-call-me-maybe-
mongodb](http://aphyr.com/posts/284-call-me-maybe-mongodb)

[2] [http://svs.io/post/31724990463/why-i-migrated-away-from-
mong...](http://svs.io/post/31724990463/why-i-migrated-away-from-mongodb)

[3] [http://use-the-index-luke.com/blog/2013-10/mysql-is-to-
sql-l...](http://use-the-index-luke.com/blog/2013-10/mysql-is-to-sql-like-
mongodb-to-nosql)

[4]
[https://news.ycombinator.com/item?id=8353228](https://news.ycombinator.com/item?id=8353228)

~~~
deepsun
I agree with you, per our experience Mongo is not scalable. We use Mongo in
our company (60GB per db.stats().fileSize), both for relational data, and for
what we call "fat data" (just to avoid calling these mere gigabytes "big data"
\-- those are just lots of statistical numbers, more like OLAP cubes).

Over many problems and sad surprises, we came to the point that we'd better
have used PostgreSQL (or any other SQL) for relational data, and some NoSQL
for "fat data". Or just single PostgreSQL would do better, because Mongo's
feature set is a subset of PostgreSQL's feature set. Just Mongo renamed them
and pretends it's something new.

~~~
nl
_Or just single PostgreSQL would do better, because Mongo 's feature set is a
subset of PostgreSQL's feature set._

Just a slight warning on this, because I was mildly burnt by hearing this and
assuming it was true.

For background, I'm a long time happy Postgres user.

Recently I decided to use it for a new project instead of Mongo and to utilise
the new JSON support.

Turns out it is great for CRD apps, not so much for CRUD.

The current release version of Postgresql (9.3) has no capability to do
updates to parts of JSON stored as the JSON datatype (ie, you have to read the
entire JSON blob, change it and write it back)[1].

Updates within JSON fields are coming in 9.4.

[1] [http://dba.stackexchange.com/questions/54663/how-can-i-
updat...](http://dba.stackexchange.com/questions/54663/how-can-i-update-a-
json-element-in-postgresql-json-datatype)

~~~
jbellis
How sure are you that mongodb doesn't rewrite the entire document with each
update? I couldn't find a definitive answer without source diving.

~~~
calpaterson
Some updates are done in place, some are not. To give a specific example -
integer updates are always done in place. The most common reason that a
document has to be rewritten is when it is updated size is too big to fit into
its previous space. Then it has to be moved.

[http://blog.mongodb.org/post/248614779/fast-updates-with-
mon...](http://blog.mongodb.org/post/248614779/fast-updates-with-mongodb-
update-in-place)

I have found "source diving" a scary experience with Mongodb in the past.

~~~
jbellis
Could be, but that article just says that the update is "in place." Which may
mean that just the changed int is written, or it may be the entire document.

------
scottkrager
Dang....they just raised $7.5m in February?
[http://venturebeat.com/2014/02/10/calpont-renames-itself-
inf...](http://venturebeat.com/2014/02/10/calpont-renames-itself-infinidb-
takes-7-5m-to-suit-up-for-database-wars/)

Curious if they are returning money to VC's or just really burned through that
much in 7 months...

~~~
psaintla
I once worked at a startup that burned through 3x that in about six months.
7.5m in six months is entirely possible.

~~~
rdtsc
On what? Did they hire all their friends and payed them huge salaries and
signing bonuses. Fancy hardware?

~~~
Arcanum-XIII
For actually being in a company like that it's mostly that and bad overall
management - at a time we did have a ratio of two (useless...) PM for on dev.
Dev underpaid, PM overpaid (as expert freelance, of course, or with bonus),
some huge expense made for stuff that make no sense (branding), negotiating
badly some contract so that they cost the company more than they provide...

Of course after a while this stop since all these mismanagement corrupt the
products too, and pretty fast you're left with a huge black hole for money
that doesn't produce anything meaningful.

------
NamTaf
I've been playing with Infobright community edition but also evaluated
InfiniDB. I found InfiniDB was not compressing my data nearly at all whereas
Infobright was utterly jamming it down - factors of over 300x for even small
datasets of ~2m rows.

I don't know what the differences are to produce that, but when it comes to
storing as much crap as I was looking at, I was willing to design around the
limitations that Infobright CE has (ie: no insert/update queries) rathre than
deal with the massive extra disk cost. I have currently got 223m rows sitting
in Infobright and it's taking about 38MB.

I really hope that the OSS project takes off and that InfiniDB sees some
better compression implemented, similar to Infobright. The extra features that
InifiniDB has over Infobright CE (not only insert/update but also a multi-
threaded infile loader, for example) would convert me if only the compression
were better.

I'm all new to this though so if there's some good reason why they differ so
greatly I'm all ears and would love to know. Maybe I screwed something up in
the configuration? I'm not sure.

Either way, it's sad to see them go. Columnstore databases fill a really
useful application that I can only see growing over time as more and more
operational data is collected by industry.

~~~
eis
223 million rows in 38MB would mean less than a fifth of a byte per row. Is
your data really that extremely repetetive?

~~~
NamTaf
Phenomenal. Some is binary (sparse, too - 99% false sort of thing), one other
is an integer from -1 to 8, some are floats (decimal degrees for GPS) and one
is a datetime. 38MB is what is reported when I use the following query, which
I found somewhere:

SELECT table_schema, sum (data_length + index_length) / 1024 / 1024 'Data Base
Size in MB', TABLE_COMMENT FROM information_schema.TABLES GROUP BY
table_schema

You're right though - the cost per row sounds _really_ low, I might try to
find it via other means tomorrow and report back.

~~~
NamTaf
Yup, just confirmed that the raw folder size of the database is 39.8MB and
reports 39MB (not 38) in my query. It is also 222.78M rows. Note that you need
to remove the space between the sum and the bracket if you want to run that
query.

I can only assume that IB does some sort of differential compression on the
data to get such a small filesize on that data and that it's an artifact of
the machine data I'm using. But that's what I think the next big wave of data
will be - stuff generated by data loggers on machines and equipment and
analysed to scrape out incremental improvements on efficiency, reliability,
etc. from previously relatively low-tech industrial corporations.

------
bradhe
InfiniDB is/was a great idea. The unfortunate bit was just what a Rube
Goldberg machine of a data store it was. I spent a good few days just getting
everything _provisioned_ from our automation.

------
karavelov
It looks most of the successful MPP analytic databases are based on PostgreSQL
(e.g. Greenplum, Redshift). It's sad to see that InfiniDB could not make MySQL
work for them reliably.

~~~
samcrawford
I don't think it's fair to characterise this as a MySQL vs Postgres situation.
Having used InfiniDB quite a bit I am familiar with how it related to MySQL.
It basically only used MySQL as a frontend (meaning as an application you
could use the mysql libs and query protocol); everything beyond that was
InfiniDB's own stuff.

~~~
karavelov
Yes, you are right. This is true also for the "PostgreSQL" databases I
mentioned - they use only the frontend of Postgres.

------
mhoglan
Clear up a few things here for all the speculation. I was an architect at
InfiniDB that came on in Nov 2013 to build out the Enterprise Manager which
was coming to alleviate many of the provisioning, management and monitoring
woes that customers were experiencing and help modernize those aspects. The
first beta offering of which was early July, unfortunately the ship had sailed
so to speak. So I know first hand how and why things did not work here. As
with all things, take your lessons learned, move on. Success is not the path
of learning, failure is (see survivorship bias)

Some notes: Labeling InfiniDB as MySQL+ is a gross underestimation of what it
does. MySQL is used as the front end query parser, and that is about it.
Everything else behind it was custom written, and that is where the power is.

As with all DB technologies, your use case is the primary thing that
determines your mileage. Comparing InfiniDB to MongoDB is one of the first
signs to me that you don't fully comprehend the differences between database
architectures. For the use cases that InfiniDB was made for, we routinely were
faster performing on a smaller footprint. Using InfiniDB as a document store
can be done, but that is not what it was made for.

What people call 'big datasets' is relative. Some think 500GB is alot, some
think 5TB alot. Coming from telecommunications monitoring background, I will
appreciate your dataset when we are talking TBs a day of churn per monitoring
point with hundreds and thousands of monitoring points. The size of dataset
you are working with, along with your use case for analysis is the two most
important things in determining the technology stack. InfiniDB operated at
these higher end scales very efficiently. There is a reason why Impala was a
primary comparison, and we would usually operate on fraction of the hardware
they needed.

Best technology does not always win. See InfiniDB.

Decisions made by previous executive teams years in the past can set a course
that cannot be corrected sometimes (not efficiently or without alot of money)

Patents are worth their weight in gold.

Being open source is great for the community, but is a challenge to a business
to build consistent revenue. There were many big projects running InfiniDB
with the open source version, but not contributing to revenue. Even if they
did sign up with support, you need custom feature development and other big
ticket items to make impact. Or you have to build a large customer base paying
for support, and that takes time. With the multiple iterations of adapting the
technology to different architectures over the years, that was hard to retain
those customers consistently. Also many customers will pay for support for
their rollout or initial deployment, but when the project is done, they feel
they are adequate enough to live with open source only.

Just because a company raised $X in a month, does not mean all that money is
slated for going forward from that month. On top of that, payroll is not
cheap, and you would be surprised how quickly you can burn through money
keeping the lights on. For those of you who think people at startups are
working for pennies on the dollars, I would advise you it is not the case. And
if you are one of those people, I wish you the best, and odds are there are
other reasons why you are doing so. Why would good engineers work at discount?
Equity? There is not enough of the pie to go around to make that sustainable.
Most startups pay competitive market salaries.

InfiniDB was at a junction where it was time to go for it, or go home, and
that is exactly what happened in 2014. The marketplace for data solutions with
Hadoop rising, other MPP vendors consolidating, and bigger players entering
the field, made it very competitive, and the time to swing for the fence was
now, versus treading water and hoping.

Even with stars aligned and everything else, all you have done in a company is
weight the opportunity of it succeeding, not guaranteed.

I really enjoyed my time with InfiniDB and the team there. I really do feel
its a missed opportunity with some decisions that could have been changed
several years ago. Not securing patents and probably choosing MySQL as a
frontend are some of those.

Side note, core group of us at InfiniDB have landed at Tune, a company that
has appreciated the technology of InfiniDB and what it can offer for their
solutions. Look forward to this new opportunity and what we can provide to the
ad and mobile analytics space.

~~~
tptacek
_Patents are worth their weight in gold._

I guess I agree, in that $1220/oz x 0 is 0.

I have my name on a couple patents. I've never seen a situation in which
patents actually helped a viable business.

~~~
mhoglan
Try selling a company and being asked, what patents or protections do you have
for someone not doing exactly what you have done.

Also try defending yourself against billion dollar companies the space.

but glad you took analogy literally

~~~
tptacek
I've been a party to the 8-figure sales of more than one company and stand by
what I said. I wasn't at Arbor when it sold (for what I assume to be mid-high
9-figures), but I know a lot about how unserious patents were there as well:
not a real factor.

Most importantly: the "door-to-door" time from initiating a patent application
to bringing it to bear in a legal dispute is something on the order of a
decade.

Incidentally: I didn't take your analogy seriously; I just used it as a hook
to disagree with you. I'm not an anti-patent zealot. I've just worked in
startups for ~20 years and have come to the conclusion that they are a total
waste of time for software companies.

~~~
mhoglan
I agree that there are a lot of patents that are not factors and generally
frivolous. I would argue though that at Arbor the patents / algorithms around
classification of flows and DoS detection are a cornerstone of the business
being competitive and protected, and should someone else step on their space,
they were positioned well for staying competitive. Before InfiniDB I was a
Principal Engineer at Tektronix Communications who acquired Arbor and know
them very well too (nice company and sounded nice to work at too)

btw, dont get me wrong in saying that if they had their patents they would
have been successful. Just one cog in the whole machine. And the timeline I am
referring to is there are things back 5-6 years ago that could have been filed
before others were doing it, and it would have provided a nice differentiator
in the market. It would have helped, but it was not the sole reason.

~~~
tptacek
They were not.

------
mp99e99
Sad, wish they made it.

