Hacker News new | past | comments | ask | show | jobs | submit login
Amazon Quantum Ledger Database (amazon.com)
283 points by mcrute 7 months ago | hide | past | web | favorite | 170 comments



I'm pretty sure this is what most people actually want when they say they want a blockchain, they just don't know it yet. It's got all the useful database features with none of the "well uh let's distribute it over people's computers and let people mine coins to support it" complexity, which adds nothing to 99% of usecases besides silly ICO potential.


This.

When I see something like this:

https://techcrunch.com/2018/09/24/walmart-is-betting-on-the-...

I just don't get it.

I understand the motivation behind them wanting this sort of broad supply-chain tracking, but I don't see what "blockchain" actually solves for them that can't be more readily/efficiently solved in other ways.

A trusted third party, with an immutable transaction ledger, and a simple to develop to set of APIs seems far superior from a product perspective in a case like this rather than those solutions that will simply build this same thing on "blockchain" software. The only gap is an API which is domain specific (like for food supply chain).

(And even this won't solve many of the trust issues in food supply chain, but it looks like it will offer the same useful traits of a blockchain only more simply).

[edit for clarity]


> A trusted third party

What does an industry with thousands of different major players have to gain by coming together to give a single company a monopoly over their entire industry? And in fact, one of the biggest strengths of blockchain is that it provides a viable alternative to monopolies for various industries.


I don't think the parent was suggesting there be a single trusted third party.


> A trusted third party, with an immutable transaction ledger

This is overall great. QLDB is necessary for a lot of things, and it will be useful for a lot of the stupid 'blockchain' projects to be displaced by a better technical solution.

Now cryptocurrencies and blockchains are exactly what they are: a decentralized version of this. I believe that decentralization has some use cases that are extremely important. Now it can stand on its own, without being conflated with verifiable ledgers.


Using a private blockchain has the same issue of trusting a third party. As soon as someone has control of who can write to the ledger, you need to trust them, period.


A lot of people would assert that "private blockchain" is an oxymoron. The semantics have not resolved yet.

I expect that they will resolve to exclude such things.


Well AWS is already the trusted third party for so much stuff. What's another db ?


Ask the financial industry e.g. Swift.


The idea is immutable micro-transactions to track operations in a complex inventory system. Consider a situation where a customer finds a nail in their banana. This 'blockchain' db could be used to recognize where the banana was sold, what truck delivered it, where that specific banana and was sourced.

It can be an interesting solution or vaporware depending on your level of interest. I personally think micro-transactions and blockchain do not go well together but reading debates about the tech is still interesting.


Assuming no bad actors what’s the difference between this and a single DB that simply tracks the banana?

A blockchain won’t solve the problem of creating a unique identifier for each banana and ensuring it was correctly scanned.

Also this isn’t a good example fundamentally because the banana is not immutable as in anyone can swap the barcode from one banana with another or it can fall off.

Blockchains really only work when everything is immutable.


> Assuming no bad actors

Blockchain gives you security against bad actors. Here's a similar use case for a marijuana supply chain: https://www.nasdaq.com/article/leveraging-blockchains-to-tra...


Unsurprisingly, the article provides zero details as to how a blockchain actually solves any problems here. Frankly, it's very obviously useless for all "supply chain" related ideas. If you're going to do something illegal why would you record it in a blockchain?


What’s the advantage over a centralized system?


You have many vendors that handle the product. The farm, multiple distributors, and transportation companies.

Having a consolidated/consistent view of transactions from the full supply chain relative to tangible product could be genuinely helpful for reducing errors that could be consequential from a food safety perspective. It's unlikely to stop cheating since many of these food vendors will almost certain being humans entering transactions at keyboards (and then when you can get compliance with that transaction entry) and at those points all you can do is make cheating harder.

The challenge is how to get that consolidated/consistent view of the world across disparate parties, some of which don't know all of the other parties involved, in a secure enough manner, and in a way that the supply chain participants will be willing to take part in.

Larger players, like Walmart, can set the stage for some of that cooperation to happen just because of their relevance to the market. The government agencies that buy food also want that and also are pushing for it (that's where my clients are getting some pushes in that direction). None of that means the end goal will be achieved for any number of reasons, including many that are not technical in nature... "blockchain" just complicates those odds.


So you’re saying blockchain won’t help too much?


Pretty much.

There are attributes of blockchain implementations which are desirable attributes to solve this problem, but they can exist outside of block chains.

In fact, it's part of why I find this Quantum product interesting for this... they seem to have those qualities that would be helpful without bring much of the other baggage.

The only thing that's not clear is how you handle authority (who says who can participate, for example). So I don't think this Amazon approach is the complete answer, but this sort of approach seems like a path which might bear more fruit than trying to implement blockchains for the problem.


Inmutability of the transaction history.


You don’t need a blockchain for that also if the “transcation data” itself is not immutable such as in the case of a banana so how would this work exactly?


The transaction data should be immutable; many of the actual facts concerning the Banana are not mutable: it was picked when it was picked, it was moved when it was moved. Assuming the transactions reflect this accurately, they need not be mutable. What the transaction data should not be is irreversible. So, for example, if there's a data entry error discovered, I need to show that I'm changing the current knowledge of that banana without losing the prior erroneous transaction. It's the same sort of thing we'd do for a financial ledger: we don't change existing erroneous records in the database... we book new correcting records.

In a number of your comments you talk about errors: the transaction processing technology being discussed will not address those issues be they mistakes, non-compliance, or anything other that rather naive cheating.

I think people hear about some of the attributes of blockchain, such as being distributed, and say, "hey, that solves some of the problem of getting disparate, unrelated parties transacting in a common data set."... along with a misunderstanding of trust model and its relationship to data integrity across that shared ledger... and they get all excited about blockchain.

You're right you don't need blockchain and a central database could do it. The harder parts are administrative around the many parties and then getting the parties to participate at all. I think the transnational immutability aspects are beneficial for food safety, I don't think the raw complexity of implementing a new blockchain to do it makes sense.

I do think with the right kind of industry focused solution on top of something like this Amazon product, can be that sort of single database, with some of the helpful parts that have appeal in regards to blockchain and without the complexity. There are still issues which I don't believe something like this Amazon product solves, but I do think it moves the needle.


The Walmart thing is just PR. I doubt it's anything beyond a pilot and it's probably just a permissioned ledger, much like QLDB.


I work with a couple of food industry businesses which are also in business with some of the other members of the Walmart initiative.

The specific technology the Walmart group is working with is IBM's implementation of: https://www.hyperledger.org/about

There is a concerted effort from what we can see, and are getting some subtle hints that we need to be going down this path from some shared government customers. That is far from saying that this will be successful or that it will get past early stages, but they're definitely trying.

Having said all that, my understanding is this is more in line with a true blockchain effort rather than something like QLDB (though I hope I've misread on that count in case I do get stuck on dealing with this).


Yeah in particular I think one of the things people are really going to appreciate about QLDB is that you can query it with SQL like expressions. Instantly familiar and almost drop-in replacement for many apps vs trying to build your own Blockchain ledger


How can you make something like MakerDAO on a centralized ledger?


I don't think anyone is arguing that you can. But most companies who want a ledger don't need a blockchain to accomplish it.

In your case, you're talking about no trusted actors in a very decentralized system. We're talking about walmart tracking supply chain and whatnot.

For a centralized ledger where you do have trusted actors, blockchain is a hilariously shitty solution.


I completely agree. It's almost like blockchains are only useful for trustless decentralized systems.


I have no need for what MakerDAO offers. Do you?


Yes, I do. I trade derivatives without the need of a broker. I can also lend DAI to anyone in the world through a smart contract.


It's still too early for most people to understand how amazing the concept around MakerDAO and DAI actually is. They will, but it will take a bit of time.


Been hearing about it but must admit I don't fully understand it. Would you mind a sentence or two on why the concept is so amazing?


MakerDAO is basically a community run central bank on the Ethereum blockchain. You can lock away assets you own (currently only ETH but eventually even your house will work as a collateral). By locking away your collateral you can withdraw DAI which will be generated based on your collateral. This is comparable to how banks worked in the past where they only printed money for which they had gold as collateral.

You need to at least have a collateralization of 150 %. If you fall below that, your collateral will be seized, the same amount of DAI that you own will be bought from the market and burned. The rest of your collateral minus 13 % liquidation fee will be send back to you.

At any time you can pay back your loan in DAI or add more collateral.

This mechanism keeps DAI stable at 1 USD.

The entire mechanism runs entirely on the Ethereum blockchain.

Know why is that cool?

1. You have a 100 % decentralized token that you can keep in any crypto wallet that supports ERC-20 token (Be your own bank)

2. You have a token that isn't volatile so it is safe to pay out salaries or buy groceries with it.

Stablecoins that are pegged to Fiat money like the USD still have one centralized aspect though. The currency that they are pegged to is still controlled by centralized organs (e.g FED, European Central Bank etc).

The cool thing is, this stuff is just getting started. MakerDAO is just a few month old and pegging to the USD is really just the beginning of a new era.

In the long term, imagine you have a stablecoin that is pegged to the purchasing power of North America or Europe or any other region of the world.

It's totally possible. The data feeds are the hardest part but there are solutions for that.

Imagine having a currency that is pegged to the purchasing power of your region without any governmental control. This is where we are heading.


Great explanation. I heard about it too but didn't understand it this way. May be I don't have finance background. Any resources you think are great to learn more about this ?


How does it monitor the peg in a decentralized, sibyl-resistant, manner?


Price feeds provide the data. They get normalized (removing extreme values) and I think there's some parameters in regard to sensitivity that they can tweak. The trusted accounts for these feeds are voted by all MKR holders. The overall system is very much like a community run central bank.


Hey, I only said 99%.


True, and I agree that most ICOs are just money grabs trying to sell 'blockchain' for services that don't need one. But decentralized ledgers offer possibilities that weren't possible before, and I suspect that in the next ten years these applications will start to surface.


The biggest problem I see with it is putting all your eggs in one basket. One company could easily become untrustworthy without you realizing it. A lot of people trusted Enron before the scandal came out.

That problem could potentially be mitigated by doing best-of-three or best-of-five with multiple trusted parties.

adeptus 7 months ago [flagged]

Actually, I'm pretty sure most people don't have a clue when it comes to blockchain. Amazon can't claim it is both centralized and immutable. That's complete nonsense. Also WTF is the point of a blockchain if you need to entrust your ledger data to a third party? Amazon puts out some spectacular products and innovation, but this one is laugh out loud material.


> Amazon can't claim it is both centralized and immutable.

Amazon can claim it is both centralized and immutable, provided you trust Amazon.

> Also WTF is the point of a blockchain if you need to entrust your ledger data to a third party?

Maybe none? I might be willing to bet that 99% of blockchains in use today could in fact just use AQLD or similar.

And this is what it comes down to: most of us live in a world where trust can, and does, have to root out somewhere. So much of our society is built around this. Blockchains feel sort of alien in this world.


>Amazon can claim it is both centralized and immutable, provided you trust Amazon.

No it can't. The second it is hacked and somebody alters the ledger, it is no longer immutable. Centralized security models eventually all get hacked. What you could however say is that the application you need to have immutable isn't super mission critical, and thus you can afford a very low probability - but not 0% of getting hacked.

>And this is what it comes down to: most of us live in a world where trust can, and does, have to root out somewhere. So much of our society is built around this.

And this is where you might be a bit out of touch. From China hacking US IP, from Banks world wide manipulating markets, gold, Libor, frequent global election fraud, government mis-spending of funds, corporate & government accounting fraud, Fukushima lies, Fakenews etc. The trust people have in institutions and governments world wide is in an accelerated decline. We are constantly asked to trust, and we are constantly let down. Properly decentralized blockchains are 'Trust machines'. They aren't alone going to solve all the issues I mentioned, but they do have the potential to increase trust in many processes related to these various issues. The key requirement is decentralization though. A blockchain managed by a single entity is nothing more than a dumbed down over-marketed database with limited use cases.

Google: Decline in trust: https://www.google.com/search?ei=WEv_W6DfIe2x0PEPxe6AmA4&q=D...


>Centralized security models eventually all get hacked.

This is an incredibly strong assertion and not one that you have proven. Nor do I believe it is something that you, or anyone, could prove.


You can certainly claim something is centralized and tamper-evident. ie demonstrate proof that something has not been mutated over time.

See RFC6962 Certificate Transparency logs and their consistency proofs for a widely used example.


AWS is spot on here. Stop using blockchain as a database if you just want the cryptographic verification and immutability of assets et al. That’s lieterally all 95% of corporate use cases for blockchain wanted all along. Stop the hype and insanity and focus on a tool that actually does what you need.


Huge props to AWS for being honest and not blockchain-washing.


Meet Amazon Managed Blockchain: https://aws.amazon.com/managed-blockchain/


Ha yeah it looks like Amazon is taking the grapeshot approach and doing it all.


Is it possible to comply with GDPR while using this to store data? Given that it operates like an append-only log, is it possible to actually remove data to comply with a GDPR request?


You can use cryptoshredding: have an encryption key for each user (stored outside of this ledger) and encrypt all PII with that key. Throw away the key if the user wants you to delete their data.


But then you must also plan for what happens when that encryption is broken. So I think you also need to control and protect your storage in order to make that a safe strategy.

The more I think about these things, the more I distrust cloud providers, and want my own hardware.

Do you really trust these companies enough to hand them the keys to all your data? Is there really any way to provide secrets to your app without trusting the hosting provider?


If you only care about legal liability, cryptoshredding is generally recognized as an effective measure for secure deletion.


Fuck players who operate like that. Slater Systems will always protect its users at all cost.


What if the key leaked before you have thrown it away?


That's a good question!

If your keys leaked, you'd probably have to assume you lost all of the data up to that point. To secure the data going forward, you'd need to generate a second key per user for all of the future data. Well, and hopefully shore up the security problems!

I agree, though, that an immutable ledger like this complicates things in a way that you-shouldn't-mutate-but-can datastores do not.


I think it's worse than just losing the data. If you operate a public cryptography ledger with users data in EU and do it under some company name, you won't be able to comply with the "right to be forgotten" or how it's called.

I'm currently working on this problem in application to blockchains. The plan ATM is to implement cryptographic snapshots of the data, where the old transactions are erased but their proof is available.


It's almost like regulations on remembering are a bad idea...


Probably the same as when the actual data is leaked.


key rotation, disclosure, generation, storage, escrow, regulatory jurisdictions - there are a lot more issues than what you mention.


By not storing that type of data. It's you need to store that type of data you can also anonymize or turn into keys where you keep the answers in a separate (mutable) database.

Also, for some purposes (legal) you are allowed to store the data regardless because you have to for other reasons.


But what about successors of GDPR? What if it becomes illegal to harbor certain information that already exists in the ledger?


Seeing as it's immutable, it would appear not.

Which is why you use it for specialised use cases and keep any PII out of there.

It would be possible to replay into a new ledger, filtering out the pieces of data to be deleted, but that goes against having an immutable log in the first place.


It's easy enough to store sensitive data externally (e.g., in a key value store) and simply store a reference to the data along with its hash in the ledger. When data needs to be removed, delete the data from your KV store and add an entry to the ledger noting that it was removed.

But you probably wouldn't store sensitive user data in this kind of database anyway. Not ever use case is well-suited for a ledger like this. In most applications, this would be pointless overhead.


Your data modelling has to be GDPR compliant not your database.


How do you delete user data from an immutable store? You get into cryptography at that point and then some edge cases make it not so simple.


There are two parts of every PII storing system. The actual PII store which is super small, "mutable" with your terminology, locked down so nobody can access it without raising an alarm and usually not accessed at all except for some very limited use cases, including GDPR ones. The rest of the store just uses references to the entities sitting in the GDPR store, like a numeric id (foreign key in SQL terminology). This way any data store, SQL, datalake, etc. can be easily GDPR compliant without needing to delete data in the large data stores and this also increases security because in case of a security breach to the data stores the GDPR data cannot be accessed.


If you tie a user to a uuid separately from where you are logging the transactions, you can nullify the existing UUID link to the given user and be in full compliance with GDPR.


The point is that you never put it in an immutable store in the first place if there's a chance that it would need to be removed later.


GDPR "Right to Forget" has an exception clause that defers to regional Accounting compliance laws, such as retaining a credit card transaction for 5 years.


you probably would want to store the reserved data in a separate store, and just reference them from the immutable one.


Ha! Somebody had the same doubt I had!


So is “Quantum“ pure buzzword hype? Or is there a reasonable connection to the product?


Quantum just means something that can be counted or measured, usually in discrete amounts like whole number. Assuming the audits and entries are discrete and transactional, the word makes sense. But yeah, it just looks like they are trying to go for marketing buzzwords.


In the context of computing technology, “quantum” is not the appropriate term, and I only agree it makes sense with the most generous interpretation of the term, whose meaning you describe finds itself mostly in certain physical theories. They could have used the word “discrete”, which is overwhelmingly more contextually accurate, if that’s what they mean.


I don't see why it wouldn't be appropriate. Within the context of computing, there is no one set meaning of "quantum". In fact, there are at least three:

1. Quantum computing, as in quantum physics.

2. Time quantum, as in the time slices a multitasking scheduler gives to threads.

3. Quantum Corporation, the disk/tape storage company.

Seems like there is room for one more meaning. They probably mean "quantum" as in "quantum leap" since this is a different paradigm for ledgers. It makes sense to me.


Eh, it's a branding thing. I doubt many customers are confused.


I predict many confused/rushed tech journalists will write articles that mention "Amazon already has a Quantum Computing Product" after this.


Wouldn't "Quantized" be more appropriate then?


The first time I clicked on the link I got a 404. The second time it worked. I guess it's this indeterminacy which makes it quantum. /s




The color joke is a throwback: https://dilbert.com/strip/1995-11-17


Uncanny timing


Why are they using the word "quantum" here? With the advent of actual quantum computing this will just become increasingly confusing.


Came here to make the same complaint. Quantum science / computing is already confusing enough without things arbitrarily named "Quantum X". I wouldn't really be complaining if I didn't think Quantum Databases will probably be (are?) a thing.


Under the guise of "immutable ledger" (basically just a honest scout promise by Amazon) Amazon gets into industrial provenance business. Genius. No sarcasm. Like that satellite ground station business yesterday. Basically "own the platform of the future" strategy the same way like spinning up the AWS 10 years ago. While companies like Google struggle to find any new business, Bezos just "printing" it non-stop (does he have a separate area in his brain labeled "Bezos X" ?).


Newbie here. Questions: 1) Can somebody go over use cases and target customers, where one would prefer this over more traditional DBs. 2) Since this is centralized, it is in theory possible to go back to a previous state of the ledger. Is that blocked simply because the product doesn't support rolling back to a previous state?


1: Use case. Traditional bookkeeping. Asset account balances start at zero initially, then increase with every debit and decrease with every credit. If someone mis-reads a check and enters a deposit as $200 instead of as $20, we do not change the $200 to $20. Instead, we do one of two things: reverse the difference with a $180 credit, or reverse the deposit with a $200 credit followed by a new $20 debit. Traditional DBs support all three approaches: reverse and repost, reverse differences, change the original. If you need to know why the original was changed, in a traditional DB you need an additional audit trail. Implementers can neglect to implement a n audit table. With QLDB, there is no risk that the implementers will choose to update the original deposit, because the database is write-once.


> build ecommerce site

> have an inventory list and supply chain

> want good audits of what happened when built into your database


Am I right in believing this looks like a competitor to Datomic?


Datomic doesn't have cryptographically-guaranteed immutability -- its logical data model is immutable and so you can query the past versions of data etc., but there isn't anything stopping one from altering the history. In a ledger like this, history cannot be modified (assuming reasonable computational limits). Datomic also has much richer data model and query languages.


I too am getting that vibe


Uhm... If some of my data is stored in one instance of that database and I then ask the owning company to delete such data in accordance to , say, GDPR... What happens ?


You delete it. You think there won't be a "I'm pretty sure I want to delete this?" option? you probably have to explicitly go out of your way to do it since the point is not to delete data.

There's probably also an option for not keeping history for data that it doesn't make sense to have history for.


If you can delete things then it's not an immutable database.


Just as a serverless networked applications are not really serverless.


Amazon QLDB does solves some of the QDPR compliance shortfalls of blockchain, for example the physical location of the data could be guaranteed. But it doesn't solve the big one: the right to full erasure, also known as the right to be forgotten. The tactics suggested by other users here, such as 'losing' the key, do not fully qualify as full erasure, just pseudo-erasure. So far, to the best of my knowledge, nobody has as yet developed a GDPR-complaint, public-facing DAO application. It's a grey area. I am working on a blockchain wallet that allows contractors and gig economy workers to request, own and administer their own employment references. The main problem we're facing is to guarantee full erasure, and to guarantee the physical domicility of data (for example,in the EU region). I was excited about QLDB but I see many problems in its current state.


This reminds me of temporal database [1] concepts. I've been thinking that besides the hashing aspect of it, otherwise, it's actually basically the same as a temporal database, isn't it?

I do like that they referred to it as a database rather than blockchain, as it more accurately captures the fact that it is a data store and I always felt like blockchain was more a reference to the way that the data is cryptographically linked together in the data store.

[1] https://en.wikipedia.org/wiki/Temporal_database


I'm no expert, but there definitely are similarities. You could design a temporal table where a change in a customer's email always generates a new row versus an update, or you could just be capturing a snapshot of time-series data at a point in time. A ledger database is much more like double-entry bookkeeping, or git, where you are capturing each change.


This is what I use protected git repo with a pgp signature for.


What throughput do you get with that?


Is there a need for high throughput in some domain?


Yes? Name any domain and there's somebody who wants high-throughput auditable transactions in it.


From a centralized source?

So sure. Voting. Wouldn't it be great to have an auditable trail?

But why would you then go and put all of that in one spot?

Wouldn't a better approach be reconfiguring the tech to work in a faster fashion?


I also don’t get it. Auditable trail? Use a Merkle DAG. Or something like this Permissionless Timestamping Network: https://intercoin.pdf/whitepaper.pdf

I mean how hard is it? What are the real challenges?


Glad it's not just me.

Elsewhere they say something like "But it works like a database! You do queries and everything!"

Ok. So sign a database.

It's like they're expecting developers to not know a damned thing about blockchain, but know they want a secure database and something with cool buzzwords in the name and AWS has one for them as long as they have a credit card.

AWS needs to provide clear and compelling places where AWS is not the answer to everything. I am not hearing that, and it indicates to me a lack of appropriate executive direction. Yeah, sure, build a cult. But build a cult with guardrails.

I'm not sure that this is the "Make stuff people want" that I would be comfortable pumping to others. I don't know. I must be missing it.


Why wouldn't you put it all in one spot? The technical requirement of needing an audit table is ubiquitous and I bet almost every SQL database in history eventually ends up having a history table of some sort. This technical requirement is unrelated to any regulatory requirements about (not) storing data in the cloud, and certainly shouldn't be coupled to it.


Can you (have you?) say anything more publicly? Would be interested to know more.


Git and QLDB both use the Merkle tree concept to prove that history hasn't been edited. The GPG signing proves new events are coming from a trusted person or computer.


Thanks. (I'm actually aware of the mechanisms involved and have implemented something similar in the past, although with a rather different set of tradeoffs.)

My main interested was in use cases and the details of the integration with git.


Interesting, I actually implemented a simple immutable ledger on top of mysql this year to track balance mutations of our InToken coin. The coin exists on Stellar as well but we track federated stellar accounts in our database for most of our users. This means that we represent them with a single stellar account. We allow users that clear our aml procedure to send/receive tokens to stellar.

Having our own ledger means we can cheaply do internal transfers, micro rewards, and other incentives. Doing the bookkeeping correctly and in a tamper proof way is of course important for us, which is why we built our own ledger database to ensure we don't end up with corrupted data (either through bugs or malicious activity).

I use content hashes as the id that include the id of the previous ledger entry and the key data stored in each row. The core design is pretty easy (it's basically a linked list) but the devil is in the details with this stuff since you indeed need to worry about auditing and making sure you don't end up losing transactions.

Additional headaches include dealing with concurrent transactions and the fact that mysql does not do serializable transactions (at least not in sane way). Each row should only have 1 successor meaning that every new row involves looking up the previous row, and using it's id as a parent id. So we have a select and an insert happening in a transaction. We have a simple db constraint enforcing the parent is referred only once. We retry transactions when this constraint gets violated. This does actually happen when two concurrent transactions decide to use the same parent id. If transactions were serializable, this would not happen and the second transaction would end up using the id of the first.

Another of the gotchas is that data migrations are kind of hard/impossible in an immutable data store. The only way to do it would be to effectively recreate a new database with new content hashes. So there are some things that I'd like to change that I can't actually change because it would break the content hashes. But by and large, this design is working quite well for us so far.

So, in short, it's not rocket science but hard enough that having a well supported product that does this is worth having. We're actually considering open sourcing it at some point since it seems there are quite many projects out there that use a ledger primarily to have some tamper resistant immutable and auditable log of transactions.


Given your problem with serializability, why not use a database which supports it better than MySQL?


By the time we had to debug this as a thing, we already had our implementation ready. Also cost for a small mysql RDS db in amazon and the convenience of not having to administer that were a factor.

But you are right, I have considered switching to cockroachdb since they advertise their support for serializable transactions.


Why not PostgreSQL? It's also on RDS.


I wonder if this is just based on Kafka with some native tamper-proof functionality added to it?


I had a similar thought - this is probably built on Kinesis or Kafka with additional crypto layer


Not sure why there's so much hate on here. The article explicitly mentions that the database is Owned by a central trusted authority. This is distributed ledger technology within the AWS ecosystem, not decentralized blockchain technology


The promise and all the overhead of real distributed blockchains is that you should avoid any need to trust anyone, and if you're really paranoid then you should not trust even your own database copy. All this could be hacked just as Amazon can. Power of real distributed ledger is that as long as less than 50% of the data copies (consensus) are not hacked, then database as whole can be trusted. Now if you can hack also consensus, then even blockchain is hackable. With the "centralized ledger" you don't have the consensus protection level, therefore I see only marginal (if any) security improvement on top of just plain database with properly applied access control and auditing. You still have way more parties you need to blindly trust.


One thing I havent seen mentioned in the comments is talk about auditing challenges, how would journal auditing go? are there already tools for this? is it just another "table"?


This doesn’t explain how amazon has removed capability for themselves to mitm the whole process. Or perhaps I missed that part somewhere?


Did they just ship Trillian, or is this an AWS-exclusive design?

The webpage says they use SHA256, but it doesn't go much more into detail than that.


AWS-Exclusive from my understanding. Not in AWS but from reading other comments, it sounds like this has been widely used internally for some time.



I was at Amazon during early development and watched several of the PoA talks about it. On the one hand, the concept is surprisingly simple, on the other hand, it's such a useful general purpose concept that it's surprising it wasn't done earlier. I believe it's first public use case was Database Migration Service. It's neat to see it offered as it's own product.


Do you know if the QLDB write API is more like an event store or more like a relational database? It sounds like the read API provides both models (read the journal, or read the snapshot).


It should work for both cases. It's an append-only ledger that captures any mutations on records previously on the ledger. It uses a recent snapshot primarily for efficiency to avoid having to read the entire ledger to enforce any transactional or schema constraints.

So it's an event store in that each write transaction is an event. But it's a relational DB in that it (at least the internal version did) schemas, multi-record updates, and transactions.


That's an understandable implementation, but a bit disappointing. It means that there's not going to be a simple way to process the journal since that log will consists of differently sized writes to the database. It would have been more interesting if QLDB was a CQRS+EventStore with built-in snapshots.


Wow, this is exactly the sort of database I was dreaming about for a logging solution. I need to deep dive into pricing though.


QLDB = Quantum Ledger Database

Seeing 'quantum' in that acronym is so cringeworthy. I'd prefer Immutable Ledger Database, or a word that shares more with the technology than media hyperbole.


> Adjective - quantum

> Of a change, sudden or discrete, without intermediate stages.

- https://en.wiktionary.org/wiki/quantum#Adjective

It makes perfect sense if you know what the word means.


Cool, but I don't think I could get away with marketing a raspberry pi as a "quantum computer" because its bits change suddenly, discretely, and without intermediate steps.

Like it or not, "quantum" has accumulated a) more connotations that make it less likely to be interpreted as "sudden or discrete" by most people, and b) an association with other pseudosciency buzzwords that don't have a very good reputation in product names.


Yes, but given the trend of quantum computers, it's confusing using that word freely.

Case in point, the whole reason I clicked through on this was becausei thought it might be some earlier quantum database.


https://cacm.acm.org/news/232751-quantum-clouds-leap-ahead/f...

If Quantum computers become a thing, then coining a new database 'QLDB', is a bad idea, despite what 'quantum' may mean in the dictionary.


... it also makes perfect sense as a ruse if you figure all those buzzword-chasing QuantumComputinAIDeepLearningBlockchainOfThings will be flocking around something that is not just useful but also buzzword compatible.

IBM has spent a bit of money drumming up all things Quantum (while being at most moderately more successful at actual useful QC implementation), it would be ironic if some of the buzz (in people who think Quantum is the "next big thing" without knowing how actual current QC would benefit them) now transfers to something useful and real but not-QC


Given the existence of quantum computing it makes sense as false advertising.

I accept that it may just a terrible choice of name that is not deliberately intended to confuse though.


Does the name suggest it has Quantum Computing + Distributed Ledger?


Looks like these guys were on a very similar path. https://m.youtube.com/watch?feature=youtu.be&v=3T2H4qO9_b0


Do any immutable open source databases exist?


Not good ones because everyone wants to solve the distributed store, proof-of-work & contract-as-software problems rather than the primary thing that makes blockchain useful for normal businesses.


onecool 7 months ago [flagged]

Why all of the Amazon spam? Three advertisements in 6 hours.


Their big reinvent conference is this week, they're making a series of product announcements.


New feature announcements is not spam.

Especially when it's the biggest cloud provider that many of us depend on.


[flagged]


The AWS conference is going on:

https://reinvent.awsevents.com/


So?


Please stop now.


So a lot of people who work in that ecosystem are interested in the new announcements and as can be seen from a lot of the stories on HN, a lot of people are nerding out over them. This is kind of what HN is for.


Lots of announcements of cloud-related tech, which HN users find interesting.


If centralized, why not just restrict the API to make history immutable? Lol


When the largest cloud provider takes time to create and market a new product it's probably worth thinking about for more than a "lol".


AWS employee here. QLDB has been used internally at AWS for quite some time. We decided to turn our internal service into an external facing product because we saw a demand for a ledger system with central authority. Many people were trying to force Blockchain to accomplish this, while we already had an internal ledger solution that is far more scalable while still being cryptographically verifiable.


What's it used for? (Can you say?)


A lot of stuff. See the two tweets I referenced in this reply: https://news.ycombinator.com/item?id=18553937


Thank you!


This I’m that’s why I’m asking


That doesn't give you the ability to cryptographically verify integrity of the database. Think git repository with pgp signature on every commit.


I think once centralized, there is inherently something wrong with blockchains. There is always chance of bait and switch, even though you trust amazon.


Sure but QLDB isn't a blockchain. Blockchain is just one of many things that can be implemented on top of it. You could implement the same thing with an SQL database and signed history tables.


QLDB != Blockchain. It does not use blockchain at all.


Does quantum mean its thread safe?


The word quantum doesn't mean thread-safe. But it is thread-safe, since it's fundamentally based on a durable ACID Transaction journal. All the threads in your application can agree on state.


As they pointed out, this requires me to trust Amazon. But since that's the case, what's the point of it being auditable? Amazon can easily drop the last N transactions on your ledger and tell you the digest you provided was "not found".


Amazon could leak your secrets stored in Secrets Manager, or KMS.

ACM could leak your private keys.

EC2 hypervisor could have a 0day that allows China to steal all your data.

CloudTrail could drop audit logs.

At some point, you have to trust the company who's software you're using. Sure Amazon could easily drop the last N transactions, but what motivation do they have to do so? What behavior in the past makes you think this wuold happen?


Completely agree - what I'm wondering is why they are telling their customers that they can verify the data for themselves, when that is entirely pointless: "and using cryptography, you can easily verify that there have been no unintended modifications to your application’s data"


How is it useless?

If the digest you provide is found, then you know the data hasn't been changed. Sure, that's the only bit of information you get, but in a lot of cases that's the only one you need.

It doesn't guarantee your data can't be changed, no. What guarantees that is your contact with Amazon.


Your other comment puts it nicely - "trust but verify" - I guess it's more about auditing yourself, than Amazon.


Maybe they just want to quell the blockchainia.


They could, but why would they?


Indeed, they wouldn't - but the product offering sounds like they are telling customers that they can audit their data and that it is cryptographically verifiable - but why does the customer care about this, unless they don't trust amazon to preserve data integrity?


"Trust, but verify."


It seems to me the main reason people don't go for such a Blockchain is because they don't 'trust' Amazon. Fair enough - why not then have a blockchain owned by several different competing companies with huge data centers: Google, Amazon, IBM etc. This solves the computation and scalability issue with blockchain while also making it less likely one company gets too powerful (any new transaction will need to be confirmed by all the parties (Amazon, Google) providing a self regulating mechanism).


Multi-company operation is the holy grail of private blockchain projects, but it's also too complex for people to manage.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: