

Show HN: BigObject – Run your data analysis faster - chenyuanjen
http://bigobject.io

======
buremba
It seems like a stream processing engine that uses a custom query language
similar to SQL (well, even though the syntax doesn't look like SQL, the
documentation says so). The website is not helpful enough considering that
your target audience is actually developers. "Run queries 100x to 1000x
faster" doesn't mean anything.

~~~
chenyuanjen
Hey thanks buremba. We're working on a more elaborate introduction with clear
definition and position in system diagram, use cases, etc.

------
lootsauce
I quickly scanned the white-paper on linked on their site [1]

So they are doing some interesting things with a virtual memory space using a
tree structure of the data that reflects various relations in the data in a
star schema. Those objects are persisted in memory mapped files, mapped into
an effectively infinite single shared memory address space in the virtual
memory model. They argue that you can then do new more efficient algorithms on
these memory mapped files.

Sounds like they are seeing performance we are getting in Google BigQuery with
queries against 100s of millions of rows taking less than 10 seconds however
they did not show how it scales beyond that.

Another interesting thing is this is designed to sit on top of existing
systems to serve not as the source of truth but the layer to do analytics with
realtime data, a very compelling approach IMHO.

They liken their data-structure to that of RDD (Resilient Distributed Dataset)
used in Spark but unlike RDD it is not temporary and is writeable.

The other interesting thing about this is as far as I understand it, this can
be seen as a virtualized implementation of the computing model that is coming
with new memory hardware [2] where ram == storage

[1] [http://www.slideshare.net/BigObject/big-object-store-in-
plac...](http://www.slideshare.net/BigObject/big-object-store-in-place-
computing-for-interactive-analysis?ref=http://bigobject.io/)

[2] [http://www.zdnet.com/article/nanteros-carbon-nanotube-
memory...](http://www.zdnet.com/article/nanteros-carbon-nanotube-memory-
breakthrough/) "The race for a workable NVRAM is in its final stages. HP's
memristor, Crossbar's RRAM and now, Nantero's NRAM, are all technically sound,
backed by tens of millions of dollars in R&D, and close to broad market
release.”

~~~
mdaniel
I thought the slideshare.net link was odd, but it seems that is the only non-
paywall-ed version of the file. As a friendly FYI to other interested parties,
[http://bugmenot.com/view/slideshare.net](http://bugmenot.com/view/slideshare.net)
is a mechanism through which one can use the download button, resulting in a
much nicer on the eyes PDF.

~~~
chenyuanjen
Hey mdaniel that is an IEEE paper we published.

------
MasterScrat
How exactly is that supposed to be faster?

What will happen when I "Launch" it, and why should I give it my email address
beforehand?

~~~
chenyuanjen
Hey yes~ you can also run BigObject with docker container. In that case, you
won't need to send us your email address. Email address is for use to send the
link for those who chose web service. And also stay up-to-date with our latest
release.

~~~
ahazred8ta
I hate to say it, but your people really need to redesign the entire website.
[https://docs.bigobject.io](https://docs.bigobject.io) is a better starting
point than your main website. Your "Learn" page only has a video, but most
developers would skip your "Quick Start Tour" video and assume it was a waste
of time if they can't first spend time reading a detailed text explanation of
how BigObject works. No link leads from "Learn" to your docs site.
Technically, a link to docs does appear in the sidebar of the Product page,
but that page contains so much uninformative text that many readers would lose
interest before examining the sidebar. Please consider moving the information
at [https://docs.bigobject.io/docs/why-bigobject-
analytics](https://docs.bigobject.io/docs/why-bigobject-analytics) to your
main page.

~~~
chenyuanjen
Hey your comments are really helpful. We just updated the content and it
should be clearer now. We will make the design prettier later too.

