Hacker Newsnew | comments | show | ask | jobs | submit | electrotype's comments login

What scares me about microservices is the case where some operations must be transactional.

What if, in a given use case, multiple microservices are involved but the operations must be transactional : if one of the services fails, all previous operations must rollback. What are the recommended ways of implementing this kind of transactional behavior in a modern HTTP/REST microservices architecture?

I know the pattern is called "distributed transactions" and is often related to two-phase commit protocol. But there doesn't seem to be a lot of practical information available about this topic!

I found this recent presentation[1] that talks about it, but I'd like to learn more on the subject. Also, I'm looking for practical tutorials, not highly academic ones! I'd really love to see code samples, for instance.

Any links, suggestions?

[1] http://www.infoq.com/presentations/microservices-docker-cqrs

-----


We've moved down this path from a massively complicated distributed transaction environment on top of MSMQ, SQL Server etc and you know what? With some careful design and thought about ordering operations and atomic service endpoints, we didn't need them at all after all.

Transactions can be cleanly replaced with reservations in most cases i.e. "I'll reserve this stock for 10 minutes" after which point the reservation is invalid. So a typical flow for a order pipeline payment failure would be:

1. Client places order to order service.

2. Order service calls ERP service and places reservation on stuff for 10 minutes.

3. Order service calls payment service (which is sloooow and takes 2-3 mins for a callback) and issues payment.

4. Payment service fails or payment fails.

5. Order service correlation times out.

6. Order service calls notification service and tells buyer that their transaction timed out and cancels the order.

7. ERP service doesn't hear back from the order service and kills reservation.

etc etc.

At step (4) you have an option to just chuck the message back on the bus to try again after say 2 minutes. If everything times out, meh.

-----


Thanks for this! It seems like a very interesting pattern and I was completely unfamiliar with it prior to reading your comment. Looks like a search for "reservation pattern" gives lots of good places to start digging, but I'm wondering if you have any favorite resources on the subject. Is there a good treatment of it in some particular book? Or maybe presentations you've found particularly enlightening?

-----


It's an old SOA pattern. Most of those patterns are still valid if you take away the WS-* cack and stuff.

-----


What happens if the payment succeeds, but the subsequent call to the ERP fails? Wouldn't the reserved items be released?

-----


ERP call is atomic and happens first. All that says Is "can I have 2kg of potatoes for 10mins please?" If that returns a success code then you can process the payment.

Also if the ERP says "in ten days you can have that amount of potatoes" you can ask for a longer reservation and issue the payment later.

Its all about careful ordering and atomicity at the service level and determining what must be done synchronously and what can be done asynchronously.

-----


I was interpreting the parent poster's question to mean:

1. reservation is placed.

2. Payment succeeds, but either success is not known, the process requesting payment crashes before the response, etc.

3. ???

Since we never got to telling ERP "hey, that reservation will be permanent because the payment succeeded", but the payment succeeded… what do you do? Does the reservation expire (but my potatoes!)? How do you even know that the payment succeeded, if perhaps a network connection goes dark and requires 2h to fix?

-----


In this case it's not really any different than other distributed transaction systems... another process (potentially manually) has to review, and correct things...

What happens when your payment processor succeeds in processing the transaction, but you don't get the success code? You either retry/confirm/correct... One would assume you would, upon not getting confirmation that your reservation was made permanent, retry the commitment, if it was already committed, then the erp service can return the appropriate response.

-----


I missed the confirmation step above. That would happen once the payment has been correlated.

At step 3 in your list above the payment would time out and a refund would be issued. Usually payments time out as well so you can usually reserve cash (pre-auth in banking terms). So we end up with stacks of reservations.

If something breaks you can retry within a reasonable limit or wait for everything to drop all the reservations.

-----


The concept of Aggregates in Domain-Driven Design is based around the need for business invariants that must be maintained with transactional consistency in a system that is generally eventually consistent.

Overall, you have to learn to love eventual consistency, but small portions of the domain should absolutely be clustered together around transactional consistency needs that are absolutely necessary.

Check out "Implementing Domain Driven Design" by Vaughn Vernon; chapter 10 in particular talks about this.

-----


Caitie McCaffrey gave a talk on this subject at GOTO Chicago 2015 [1], refreshing the 'Saga Pattern' for a distributed environment. Short summary, IIRC:

1) Use a linearizable data store to store transaction metadata

2) Each step must have a complementary rollback step

3) Rollback steps must be idempotent. Depending on the type transaction, sometimes 'rollback' is too strong and you instead implement other types of recovery (e.g. 'roll-forward')

[1] http://gotocon.com/chicago-2015/presentation/Applying%20the%...

-----


Asked a similar question on Stackoverflow recently: http://stackoverflow.com/questions/30213456/transactions-acr...

Didn't get many useful replies. http://ws-rest.org/2014/sites/default/files/wsrest2014_submi... looked promising.

-----


OASIS has got you covered: https://en.m.wikipedia.org/wiki/WS-Atomic_Transaction

Though it probably needs some JSON and more use of HTTP-specific features in order to be acceptable today.

-----


Funny, I made a very similar comment recently:

A concrete example we've faced. A certain operation requires writing data to N flaky services. You successfully write to N-1 of them, but the Nth fails. Now what do you do? If these N things were just database writes to the same DB, transactions would save you, as you could just rollback. Without that, the answer has to be handled in code -- do you reverse the previous changes (if possible) by sending delete events, or leave the system in some sort of half-baked state and rectify things later via some other process? (I'm interested in hearing of other options...)

The answers I got were:

1) apologetic computing (Amazon)

2) consensus algorithms / paxos

The problem I see is that these may be non-trivial to implement and/or not fully understood or standardized.

-----


3) Reservation pattern.

This one really works well.

-----


Here is the example code for the talk: https://github.com/cer/event-sourcing-examples and a corresponding blog post with some links: http://plainoldobjects.com/2015/01/04/example-application-fo...

-----


We have one instance where we update an account, that goes out to potentially 11 service calls which are not transactional. We are having to maintain state in our app because the micro services have split this up so much.

-----


Postgres's documentation is one of the best I've seen. Thanks to everybody working on it!

-----


Why not Dart?

-----


Because Dart isn't a viable option for writing libraries or frameworks which are meant to be used from JavaScript.

You can interact with JS libraries, but exposing some API to JS-land is a completely different matter.

There are some vague plans to improve this, but there hasn't been much interest, because if you use Dart anyways, you do of course want to use it as the "host" language. It has the tooling and all that jazz. Of course you'd want to use it for as much as you can.

-----


There is angular Dart, I'm sure there will be angular Dart 2. The problem of Dart , well have you ever seen some Dart code ? it's miles away from javascript. Have you ever seen some Dart2Js code ? It's heavy , megabyte heavy. Who's going to work with a framework that weight that much ,developped in a third party language ? Good luck with that.

-----


It's only megabyte heavy if you use all of Dart's reflection capabilities. If you don't use dart:mirrors, Dart is competitive in size when compiled to JavaScript, especially when you consider that a modern web app today contains a bunch of support libraries like jQuery (which itself is ~30kb). With Dart, you don't need jQuery because Dart's dart:html and dart:core libraries are pretty good out of the box. Of course, do your own measurements as each app is different.

-----


Do you really think writing a framework for a language in a different language is a good idea?

-----


Angular.ts.js.at.dart.wtf - http://work.j832.com/2015/03/angulartsjsatdartwtf.html . Also see the comment from Brad Green - https://plus.google.com/+KasperLund/posts/JtWmoEs5EjC

-----


There is a Dart version:

https://pub.dartlang.org/packages/angular2

-----


Yeah. Seriously, why not dart. Any clue, anybody?

-----


I guess as TypeScript is less likely to break existing JavaScript code base, as all JavaScript code are already valid TypeScript code.

-----


Dart requires loading a dart runtime js file on to the page, as far as I know. And I think they would claim Typescript is javascript + some extra stuff on top, whereas dart is a whole new language to learn.

-----


No Dart compiles to JS. Obviously there is some overhead with transpiling languages, but it's actually pretty small. They have done a good job with it.

-----


Dart can be run in both modes. Chrome (dunno if enabled by default and/or in stable) contains a Dart VM, so you "only" need to transpile into JS for non-Chrome.

The main problem I had with Dart was [a year ago] that using existing JS libraries like jQuery wasn't as seemless as I would liked. So that would be a reason against Dart, if Angular wants to make that aspect easier.

Haven't looked at TypeScript's interop with JS. I got frightened by the name Microsoft and the fact that they only had a huge language spec PDF and not a single tutorial or something easier to digest.

-----


> Dart can be run in both modes.

In production, it's only ever transpiled to JavaScript.

> Chrome (dunno if enabled by default and/or in stable) contains a Dart VM, so you "only" need to transpile into JS for non-Chrome.

Not the real Chrome. A version of Chromium that you can use to make the development cycle smoother. When you deploy, you still transpile to JavaScript for all browsers.

-----


> Haven't looked at TypeScript's interop with JS.

All JavaScript is valid TypeScript. This was one of the founding tenants for TypeScript. So interop is "just use the JS".

You can optionally also provide a binding file that tells TypeScript about the JS code so that it can perform compile-time checking on calls. There is a repository of these bindings for common libraries called DefinitivelyTyped [0].

> ... They only had a huge language spec PDF and not a single tutorial or something easier to digest.

Not sure when you originally looked, but there is now a tutorial [1], handbook [2], and samples [3] in addition to the language spec [4].

[0] https://github.com/borisyankov/DefinitelyTyped

[1] http://www.typescriptlang.org/Tutorial

[2] http://www.typescriptlang.org/Handbook

[3] http://www.typescriptlang.org/Samples

[4] http://www.typescriptlang.org/Content/TypeScript%20Language%...

-----


Yes Dart compiles to javascript, but it has pretty different semantics from javascript and needs to insert extra javacript to get your transpiled javascript to run. For example, noSuchMethod isn't available in all js engines so using https://www.dartlang.org/articles/emulating-functions/#inter... in dart would require a non-trivial amount of wrapping code to be generated.

http://stackoverflow.com/questions/8815437/why-is-javascript...

> And the size of helloworld.dart.app.js is 102k

> When ran in optimize mode, it generated the following javascript - helloworld.dart.js which is of size 20k

-----


> And the size of helloworld.dart.app.js is 102k

No, it's not.

  dart2js --minify hello.dart -o hello.dart.js
Results in a 5 KB JS file.

This game is 35 KB (procedural graphics included):

http://mbtic.com/games/fuzetsu

-----


I imagine they've improved the runtime size since that stack overflow post was posted a few years ago. However I've never used dart and just tried it out using a basic html sample from the tutorial page and I see:

    264 Mar  5 12:21 a.dart
    237K Mar  5 12:21 a.js
    101K Mar  5 12:22 a.min.js

It was also slow to compile, 6 secs.

Anything I'm missing?

https://gist.github.com/ratbeard/70902d6347ad448de0a4

-----


If you add gzip compression, it's still smaller than doing the same thing with jQuery. "dart:html" provides an idiomatic DOM API where all list-like things are actually Lists, events are streams, Futures instead of callbacks, and so forth. It's pretty nice to use, but it's also one of the bulkier libraries.

Also note that you could now add a thousand lines of code and the file size wouldn't increase much. You already paid that one-time cost.

You can further improve the size by using a better minifier on top of --minify.

-----


Ah, that makes sense re:same size as if you added jquery, and the API abstractions do sound better.

Though as a counterpoint (and relevant to angular 2 :) I read they're going to just use the raw dom api's instead of a jquery/jquery-lite abstraction layer as the dom api's don't need the smoothing over in modern browsers like they used to.

-----


> "No Dart compiles to JS."

The question is not here, the relevant question is

"Does JS compiles in Dart ?"

Because, JS Compiles fine in TypeScript, which means Typescript benefits from all JS ecosystem and vice versa.

-----


Because Microsoft wants to add tooling in Visual Studio to use this and they obviously would want to use Typescript for that.

-----


Why Dart?

-----


A master password, no?

-----


No.

See the acres of discussion about Google Chrome not having a master password. The fact that they caved in and no provide a master password does not mean it's a good idea.

-----


The argument that I recall for Chrome not having an optional master password was that it was often less secure than using the system's encrypted data store for their account, if available.

Requiring a master password to decrypt the network passwords is a perfectly fine idea if you want to maintain portability and reduce the chance that your network passwords are accidentally exposed. An attacker has to both have the password file and either figure out the master password or have code execution privileges on the user's account to gain the network passwords. This is more secure than trying to ensure the password file doesn't get "misplaced" (e.g. on an unencrypted drive, in unencrypted backups, unintentionally through a fileserver, etc).

-----


Java / Guice / PostgreSQL and.... a custom framework!

-----


Is there a way to launch a specific run/debug configuration using hotkeys now? Someone knows?

-----


Wow, very cool!

Can someone explain what's going on at 20:24 : http://youtu.be/7YWYS209ydE?t=20m20s ?

He forks the king and the queen, then his opponent moved his queen!?! This is not a legal move right?

-----


Had to re-watch a couple times: on that move, white had already moved pawn from g2 to g3, the knight from d6 to e8 is a pre-move that happens very quickly (you can see the red square); while white was setting up that pre-move, black moved the queen.

Does that clarify or did I miss what you are asking?

-----


Got it, thanks!

-----


One mini complaint about the new customization : It's not possible anymore to place the URL input on the menu bar.

Because of that, I have to display both bars (I like to see the menu).

Otherwise : good job Firefox team!

-----


http://www.reddit.com/r/ProgrammerHumor/comments/22xb60/i_ha...

-----


Last time when their servers were 2,99$ or something I bought 3 and they were all cancelled without real reason.

Why should I trust them now?

-----


That 2.99 euro deal was a disaster for them. Instead of new customers flocking to them, a large amount of their current customers were cancelling their existing servers that were more expensive and ordering the cheaper ones. And if I remember correctly it was in the tens of thousands of order backlog for that deal.

-----


That offer was actually 2,99 euros and it was for European only.

-----

More

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: