
Ask HN: Next Generation Database - craxel
If you had the next generation database that may make existing databases obsolete, how would you go to market?<p>What do I mean by next generation database?  How about a high performance, encrypted database that never has the encryption keys on the servers? How about an encrypted database where every single tuple&#x2F;record can be encrypted by the application with a different encryption key? How about an encrypted database that supports spatial, time series, graphs, and much of SQL while minimizing inference risk?  How about a distributed database with massively parallel consensus for encrypted, ACID transactions (million+ per second) and replication for high availability?  How about a distributed database that supports trustless&#x2F;immutable transactions - in effect is also a distributed ledger?<p>You probably don&#x27;t believe me yet - but our probabilistic spatial hash&#x2F;filter with some cryptography turns out to be the huge breakthrough for searchable encryption (more specifically securely indexing encrypted records) -- not fully homomorphic encryption.<p>How would you go to market?  Developer adoption?  Enterprises?  Government?
======
oldandtired
I have seen all sorts of different database technologies being touted as the
"next generation" database technology over 4 decades. Each has not superseded
any of the others. What ends up happening is that a "new" database technology
finds an application area for which it is the "best fit" and is then used
accordingly. Some of theses application areas can be broad and some can be
very specialised.

If you can build a usable system and can make a market for it, do so. If you
think it has broad application and want it to be ubiquitous with a variety of
players (including FOSS systems) providing implementations then release all
the relevant implementation and research information.

But you first have to decide if you want to give the basic ideas away or
whether you want to hold them close to your chest. Until you make that
decision, a lot of the questions you have asked are not going to be so
answerable. There are consequences for giving the information away and there
are consequences for keeping it close to your chest (including patents, trade
secrets, etc.)

The application fields where you think you technology may be benefited from
would require you actually building full-blown demonstrations systems against
which you can compare existing technologies. I have seen many different
systems over the decades which were simply "flash in the pan" technologies and
as such take your comments with a couple of tonnes of sea-salt.

If your technology is both viable and worth the price and convincing, you
might make it a successful venture.

~~~
craxel
Thank you for taking the time to reply. We have so far kept things proprietary
and we do have patents. We've done several proof of concepts with very large
companies and they've been successful and moving forward. We have a
"partnership" with a very recognizable IT company that reviewed our
technology. However, the sales cycle is really long and a lot of "that's too
good to be true" \- so thinking about the ubiquitous path.

I started programming in 1982 - so I've seen a lot and understand the
skepticism. However, the choices made in the 70's about indexing (BTrees)
aren't the best choice for a highly parallel, distributed world. Plus BTrees
order the data, so they'll likely never work for "secret" indexing. For
instance, it is quite amusing - all of the interest in fully homomorphic
encryption and it being the holy grail if they can just make it fast enough.
FHE is a secure computation technology, not a scalable search technology.
Let's say you have 1 billion records - with FHE, you'd have to evaluate all 1
billion records for every query (which would take days at best current rate of
1 million evals per hour). As soon as you index those 1 billion records to
avoid that with a BTree, you are subject to substantial inference risk because
the records would be ordered. So, without an index that has limited inference
risk, FHE is DOA for indexing/search (it is too slow regardless). If you have
that index with limited inference risk, you don't need FHE for
indexing/search.

I would argue that our data structures for indexing data are ready for an
update - in a world with exploding volume and velocity of data and huge
security and privacy risk. Think about it, one hacker breaches a database
server today and they get everything. If we keep concentrating valuable info
like this, it'll keep getting stolen.

Again, appreciate your reply.

~~~
oldandtired
In regards to "gackers", "crackers" and "mackers" devising ways to access the
data, these kinds of people will always be a hazard to deal with. There are
all sorts of ways to deal with this, including not actually collecting all
such data in the first place, but that's a discussion for another time and
place.

As far as having proprietary model and the use of patents, these already
dictate how you will move forward from here and as such, you now have to have
the do the hard yards to bring everything to fruition. You have to ask
yourself this question: do you have the necessary nouse to follow the business
model through to completion? Just remember that when you bring in business
"partners", they will have their own priorities that will concern them and
yours will be secondary or even tertiary to them. You have to stay ahead of
the games that you are involved with (there will be many of these) and it is
very easy to lose sight of this and drop the ball.

Finally, just remember that there are no guarantees that even if you have the
best technology and all the patents that surround this new technology that
there will be anyone who will be interested in what you have to offer at the
price you want to sell it at.

From a personal perspective, I no longer have any interest in making anything
proprietary or ever getting in the ugly world of patents. So take my comments
above in that light. I have run my own business and it is a lot of hard work
to keep it going and growing. I no longer have any such interest as I find
myself now in a place of having far more important things to interested in and
devote my time to.

