Hacker Newsnew | comments | show | ask | jobs | submit | saurabhnanda's comments login

I am increasingly getting irritated with REST for our app. It takes too much thought to figure out the "correct RESTful approach" when building out an API endpoint. And then again, there are no right answers. And all the JSON serialization/de-serialization logic, for every single endpoint. Too. Much. Work.

I'm wondering why Thrift/Proto-buffers aren't popular? Even for browser-based JS clients. What're the pitfalls?

-----


Most of the things you describe have nothing to do with REST. That might be part of your problem :)

-----


Is the person getting a lower salary for personal reasons being compensated with higher stock? Or is everyone being altruistic about it -- "I won't drain cash if I don't need it"?

-----


The purpose of our salaries is to keep us alive while we build a great company; what arbitrary number of dollars is required to do that isn't important to me. We have an equal number of shares, because he's an equal partner.

If I take an extra 20k of salary, that's 20k less we can use to hire someone better/smarter. I'd much rather work with smarter people than drive a nicer car. I don't really care for "stuff" anyway.

If you offered to let me stay at this level for the rest of my life and promised I'd always be able to work on something I believed in, I'd take that deal in a heartbeat, even if it's far below market.

http://austenallred.com/everything-you-need/

-----


I just wish there was a way to fast forward 2 years into a 10k LOC project to find out what new problems this creates. After running after the new hotness in technology for the last 5 years, I've realised it's never about the problems it solves today. It's the shit that you have to maintain a couple of years down the line.

(Not saying that this is bad -- just a random rant)

-----


FWIW, the framework I wrote (Mithril) is a direct product of working on a multiple-man-year Angular codebase and the pains that arise from such a beast. I wrote about that here ( http://lhorie.github.io/mithril-blog/lessons-learned-from-an... )

On a related topic, I talk about complexity walls (the idea of code that outgrows a framework's zone of comfort) here ( http://lhorie.github.io/mithril-blog/decreasing-cognitive-lo... ) and there are slides for a presentation I gave a while back that talks about the design decisions that went into Mithril to reduce learning curves. (here: http://lhorie.github.io/mithril-presentation-oct-js-tech-nig... )

-----


Mithril is a tremendous little framework, and Leo has done an exemplary job of writing it, documenting it and blogging about usage patterns and tricks. While declining frequent requests that he cram more functionality into its core, he takes the time to explain the trade-offs, showing a deep understanding of the problem space which gives me confidence to build upon his work. So thanks Leo!

Hopefully, in time, more people will realise the "zen" of mithril's minimal approach. In the meantime, interested readers can check out lichess.org, which recently rewrote its impressive UI using mithril.

-----


I work at a coding bootcamp and we recently used Mithril to teach one of our JS units. I was skeptical at first that it would be properly understood by novice developers... boy was I wrong.

Mithril is an awesome framework. A breath of fresh air really.

-----


Mithril "HTML" syntax is not for my taste:

return m("html", [

    m("body", [

        m("input"),

        m("button", "Add"),

        m("table", [

            m("tr", [

                m("td", [

                    m("input[type=checkbox]")

                ]),

                m("td", "task description"),

            ])

        ])

    ])
]);

-----


I did a quick tweak of React's JSX transpiler to output mithril-compatible code [1], as the above was the sort of syntax I was using JSX to get away from!

    return <html>
      <body>
        <input>
        <button>Add</button>
        <table>
          <tbody>
            <tr>
              <td><input type="checkbox"/></td>
              <td>task description</td>
            </tr>
          </tbody>
        </table>
      </body>
    </html>
[1] https://github.com/insin/msx/

-----


Definitely more readable than Mithril

-----


I think it starts to look like HAML / Slim / Jade once you use CoffeeScript. Understandably not for everyone, but I quite like it, and it doesn't mess with your syntax highlighting like JSX does.

  m "body",
    m "input"
    m "button", "Add"
    m "table",
      m "tr",
        m "td",
          m "input[type=checkbox]"
        m "td", "task description"

-----


Agreed, it's not the best. Maybe you can help come up with something more palatable?

-----


yes man, this abstraction is horror, I hope mithril adheres to something else for declarative tasks a the one above

-----


Where does Mithril fit in this stack: RequireJs, Knockout (with components), SammyJS (for routing/browser history and events)?

--After doing some reading it looks like it replaces some of knockout and sammy but I'd lose data binding and dependency tracking of knockout.

-----


Mithril is roughly equal in scope to Knockout+Sammy. The small gap between what most frameworks call "bidirectional data binding" and what it actually is in terms of vanilla js can be bridged with simple functions (I talk about that in a slightly more advanced context here: http://lhorie.github.io/mithril-blog/asymmetrical-data-bindi... ).

Dependency tracking can refer to two things: tracking changes for the purpose of updating the DOM (which Mithril does handle, albeit in a different way), or computed properties (which I personally think are not a good idea, but that you could do w/ an observable library from microjs)

-----


I tried GWT years ago. It worked amazingly well, and let me write web apps in a real language. But it was significantly slower to develop a new site in GWT vs. plain vanilla HTML + CSS + JavaScript, and the speed to get up and running eventually seduced me away.

Now I'm considering going back and giving it another try. It takes more time to get up and running with GWT, sure, but the stuff you do write with it is written in... well... a real language with types and good tooling and real design patterns. The result is an application you can maintain and build upon for a long period of time rather than a heap of web cruft that collapses quickly under its own weight.

I know it's not sexy, but I've had the same experience you've had with "sexy." I'm becoming generally disillusioned with weakly typed and even dynamically typed languages -- they're fast to write but hard to maintain over long periods of time.

http://www.gwtproject.org

-----


You should also look at Dart. It has a very fast iteration cycle and is a much nicer language that Java (in my very biased opinion - I worked on Dart).

-----


I echo your sentiments about disillusionment with weakly typed/dynamically typed languages and I still haven't really reached a situation where I was glad I didn't have a type system backing me.

-----


I used GWT for a few years on two projects, but I would not use it on any future projects. The first was using GXT, which you should probably avoid for public facing sites.

We ran into some GWT compilation issues mainly due to code size a couple of times with some browsers, like Safari, that left us dead in the water until Google released an update.

The second project used vanilla GWT and we were very happy with how performant it was. But we avoided lots of GWT features which seem over-engineered and complicated. We used JSON overlay types instead of RPC calls (which made our compilation / or dev mode much faster - I forget honestly). We avoided MVP, because I never met anyone who actually fully understood MVP with GWT. Instead, we had a very simple MVC structure that worked great for us (but lacked unit tests..). We avoided most of the CSS features because the complexity/value wasn't favorable for us.

Moved on to a JS project afterwards and it was a breath of fresh air.

-----


"real language"...

-----


Not saying Java is the best language ever, just that it passes a certain minimum bar that JavaScript does not.

-----


I'm a Java developer (and it might shock people, but I like it), where I work we're still stuck with good ol' Struts, it works, and a big international corporation uses this system.

But comparing a functional scripting language with a object-oriented statically-typed pre-compiled language, arguing one is better than the other, does not make sense to me.

You can easily revert the situation and say that static typing is a weakness of java, and it's poor fucntional (until before v8) capability is a drawback and a strong point for Javascript.

I'm not into the JS world so I can't say anything about the tooling, but yes, Java's tooling and ecosystem is an extremely strong point.

My complaint is about dismissing Javascript as not a "real language" while it's got A LOT going for it.

EDIT: I also worked with GWT (GXT from ExtJs, actually) in 2011-2012, and it was really nice. I miss it.

-----


The other advantage with GWT I see is.. as the target changes, your app doesn't have to, only the transpiler.

GWT can output to ExtJS, to <insert next HTML5/JS framework>, to <future platform>. And barring minor changes, your code stays the same.

Now that is worth the investment.

-----


If you want a "real" language with types and whatnot, there's js_of_ocaml.

-----


This is so true.

On the one hand I wish that the docs for these projects could speak to those concerns, but on the other hand I'm not even sure how they would do that successfully. When something is this young it just takes time to figure out how it's going to survive and be maintained.

I remember the days of using prototype.js over jquery. If only someone could have saved me the time.

-----


> I remember the days of using prototype.js over jquery. If only someone could have saved me the time.

There was plenty of criticism from the JS world when Prototype came out, and the main issue everyone had was how Prototype attempted to shoehorn classical OOP into JavaScript (hence, the abuse of prototypes and fake class inheritance), which is a mistake amateur JS developers make to this day. jQuery has at least made it so using the most popular DOM abstraction framework didn't also mean you were running less-than-ideal code inherently, but developers still manage to mess this one up.

I think it's because when you learn how to program, you only learn the classical style, so you're not ready for the pitfalls of prototypal inheritance. So a lot of developers go through the "phase" of attempting to retro-fit their own ideas of how programming should work into a language that is designed in a totally different way, inevitably lead to problems, and have "the revelation" of understanding the prototypal inheritance idea. It's kind-of like a growing pain, which is why I'm really happy JS is being taught more heavily in schools and especially to entry-level programmers. Unlike Java, JavaScript is more accessible (just pop open the REPL in any browser) and its results are easier to observe. While it's not a great tool to teach people the concepts of OOP, I think it's very useful to teach kids at a young age that there isn't just one way to do things. They're different enough to screw developers up on a day-to-day basis, but not different enough that spending a few weeks with the language isn't enough to grasp its power.

As others have said, you really didn't waste your time there. If anything, it was simply a matter of JS developers collectively and simultaneously going through the same growing pains and eventual realizations about how the language works.

-----


    While it's not a great tool to teach people the concepts of OOP...
I actually think it's a very good tool for teaching OOP. It doesn't have types to confuse the issue, it favours composition over inheritance by design, and it fully encapsulates state.

You could do much worse than JS as a teaching language. In fact, I'd say that it was only through practical application in JS that I fully appreciated the concepts extolled by Smalltalk, Self, and Lisp. Had I worked in a Java environment I may not have had those insights until much later.

-----


I'm sorry for coming across as a dick, but the idea that you "wasted" time learning a framework that became obsolete is silly to me, and it seems to be a common sentiment.

Ask yourself: at the time you used prototype.js, did it save time on the project?

If you answered Yes, then it was never a waste of time. Knowing prototype.js AND jQuery makes you a better developer: you learned the hard way that abusing prototypes can lead to hard-to-understand code. That can only be a good thing!

-----


No, you're not being a dick. You're right — it definitely made me a better developer. To my memory I didn't switch because of a particular lack of a feature or features, but because jquery was being maintained more consistently.

Community support makes such a big difference, and now that I'm further down the line I'm averse to having to basically make a bet up front.

-----


Well, there's opportunity cost to consider.

-----


Agreed.

-----


Those problems are best solved with Engineering best practices and culture, in my opinion. Each tough / innovative problem is probably somewhat unique for your startup, and picking a solution (a front-end framework) before you even know the problem limits your ability to solve it creativily.

Most programmers are good enough that with a good refactoring culture, they can evolve the equivalent of an in-house framework. Although, for some reason, I gather that programmers are scared of in-house "frameworks". I think that attitude is short-sighted since the app you build on top of the framework will end up being more complex than the framework itself.

-----


It goes both ways. By rolling your own framework, you inevitably end up reinventing the wheel and solving problems that have already been solved. For each feature you need, you either have to create your own solution, or manually integrate a bunch of smaller libraries. On the other hand, committing to an established framework means you have to work around issues that the framework was not designed to solve.

I wouldn't dismiss using an established framework as "short-sighted". It's a tradeoff: the more complex and unique your problems are, the more it makes sense to roll your own.

-----


I agree there. Deciding on tooling for a long term project is a very tough balancing act.

Although I am a bit afraid that people overestimate the costs of rolling your own code, or "re-inventing the wheel". In most cases you aren't reinventing the wheel, because there are well documented bodies of reference for the design of almost any wheel you could need. Building (writing) a wheel (code) from scratch against a spec is much, much less complicated than inventing it.

Likewise: assembling your own set of design patterns and writing code from scratch is not "re-inventing", and is a lot easier than we give it credit for.

-----


Yeah, that's completely fair. I generally work on projects with constantly evolving requirements, so I tend to roll my own framework(s) by gluing together existing libraries that each solve a specific problem very well. That approach works well for me because most of the time I simply don't know the long-term implications of using an existing framework for any given project, so it's easier for me to evolve my own as I go. But I think there are a lot of projects out there that benefit greatly from the ecosystem behind certain frameworks (Rails comes to mind) and don't run into many bottlenecks due to said frameworks. For them, assembling a foundation is totally unnecessary because there's an open source framework that provides exactly what they need.

I don't have enough experience in different types of environments to say which approach is most suitable in most cases, but I'll definitely say that using an existing framework is the safer path (you have a community to lean back on), and is also advantageous for hiring. So I think you're correct when you say that many developers are afraid of rolling their own frameworks, but I think there are good reasons for that, especially for quickly-growing startups.

-----


I couldn't have said it better myself :)

One hard lesson I learned is that you can't bet on a front end framework having the same mindshare for very long. The churn can get pretty crazy, and in my mind this nudges the needle a bit towards rolling your own for long term projects. Especially if you can offload the complex parts of the arch to the lower-churn backend world.

-----


> the more complex and unique your problems are, the more it makes sense to roll your own.

Of course, everyone thinks their problems are complex and unique.

-----


How many data points do you need to uniquely represent every person in earth? I bet its a smaller number then the number of constraints and requirements in your system.

-----


By rolling your own framework, you inevitably end up reinventing the wheel and solving problems that have already been solved.

You don't have to roll your own framework. You could always just use the micro-libraries that are ubiquitous in JS and pick an architecture that best fits your application. shrug to each their own. :)

-----


Isn't that basically rolling your own framework? :) A "framework" doesn't have to be a huge 100k-LOC library--it can just be a set of conventions and design patterns with some code to enforce them--but you always need some kind of consistent structure in your application if you want it to be at all maintainable.

-----


No, because a framework tells you where to put your code. It will say "put a handlebars file in application.hbs, this is the default, or you can override the default and load it manually" or something to this effect.

So a framework has that "convention over configuration" flavor, while libraries are explicit. You actually have to load the application.hbs file manually with a handlebars parser. Then you use another library for the router, etc.

-----


Maybe we have different definitions for what a framework is, but I strongly disagree with the notion that frameworks have to be implicit and magical. Libraries solve specific problems; frameworks help you structure your code. That doesn't mean that your framework needs to automatically load files named a certain way, or magically call certain methods; it can just be a set of conventions that are optionally enforced by code.

I can't imagine the spaghetti that would result from not using any framework (even a tiny handmade one) and just throwing a bunch of libraries together.

-----


What? By your definition object orientation is a "framework" because it "helps you structure code" and is "a set of conventions that are optionally enforced by code". That's not a framework, that's a paradigm!

If a framework doesn't do something implicitly it's just a large library. If it's a set of conventions not backed by baked-in logic, it's a style guide.

A framework must CALL YOU. It usually gives you a piece of code that loads itself and lets you customize what it does by passing your code/configs to it. Then you tell it to run with what you gave it. The parts of the framework that you call yourself are actually "plug-ins" or basically framework-specific libraries.

If the framework never calls your code and you only call into the framework, that's always just a library. I would argue that actually it's easier to conflate a very full-featured library with a microframework because both really kind of call your code (especially when it's in the form of closures or a DSL).

You would never accidentally call a framework a library, though, because it's obvious that it's handling things for you. It's running everything behind the scenes and you just kind of advise it to do the things you want.

-----


A framework must CALL YOU. It usually gives you a piece of code that loads itself and lets you customize what it does by passing your code/configs to it. Then you tell it to run with what you gave it. The parts of the framework that you call yourself are actually "plug-ins" or basically framework-specific libraries.

I really like this description. I've been trying to come up with a better description of what a framework is and isn't and I kept falling short. This one works well. Thanks! :)

-----


Isn't that basically rolling your own framework? :)

shrug Personally, I don't see it that way. Frameworks are more generalized and reusable. They tend to be so large because they have to take into account a wider degree of problems. Applications with custom architectures and some external libraries are very specific and not typically reusable. Maybe it's just a matter of degrees.

-----


Ha ha ha, this is awesome. Thank you for writing this. You have to admit, benefits that are provided sound amazing, hard to look over.

It is interesting times for sure.

-----


Nanda? Haha funny to run into you here. Kaisa chal raha hai?

-----


I couldn't find out more about you from your HN username. Care to ping me on twitter @saurabhnanda ?

-----


New to this entire brouhaha, but I'm concerned because my entire production stack runs on Linux. At some point devops is not going to be able to do things the old way.

Is there a link where one can read about the rationale of why systemd was needed? Perhaps by the person who started off the initial project?

-----


is this a joke? - "my entire production..." you're a devops engineer? - "link where one can read about the rational.." with good grammar? - "why systemd was needed... " ok you lost me there - "the person who started off the initial project?" and you've been hidden under a rock for the past 10 years??

http://0pointer.net/blog/ - is his blog

are you seriously trying to tell me that someone is going to have to explain what a "container" is by comparing with chroots? :D how have you been running your production stack all these years if you didn't have an internet connection under your rock :)

-----


Is penalising early exit through contractual terms illegal? If not, then isn't this just a case of who has more leverage when negotiating the employment contract? That, and the fact that Indians are so desperate to go to the US that they're willing to sign any contract.

PS: I'm an Indian working in India.

-----


Penalizing early exit with loss of immigration status could be illegal...penalizing with money is probably not.

-----


Having an employee pay visa fees for an H1-B is illegal: http://www.hackinglawpractice.com/video/h1b-employees-cannot...

Therefore if the penalty for "early exit" is there to cover the visa and legal fees, that penalty is illegal.

http://blog.laborlawcenter.com/news/virginia-company-pays-17...

> A firm specializing in information technology has been ordered to pay nearly $1.7 million in back wages to H-1B non-immigrant workers following an investigation by the U.S. Department of Labor in a case that should sound a warning for every employer.

> Investigators also found that the Virginia company charged new H-1 B workers fees for training ranging from $1.000 to $2,500. Such fees are in violation of the law.

-----


Is it possible to use this as a drop-in replacement for Angular POJOs that hold $scope data? We're hitting a whole class of bugs which are basically due to mutability of native JS objects, and we could use something like this for the most complex ng-controllers.

-----


Object.freeze, maybe?

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

-----


I've never tried this myself, but anything is possible.

If you get this working, and want to write a blog post about it, let me know!

-----


Thank you for your thoughtful reply. It matches my thinking, except one part -- I'm constantly wondering if implementing the MQ over an RDBMS is a more pragmatic approach for us at this point.

Conceptually it's still a queue and still affords us the benefits of decoupling. Downside is probably locking issues in the DB. Upside is that is uses components in our stack that we already understand really well.

Thoughts?

-----


Your welcome, hope it helps even if just a little.

There are a number of companies/people that use the RDBMS as a queue, usually they utilize their same database server as the primary datastore. And a few will separate it onto a new server, but either way it is generally not the best idea. Ironically we just had a client do this, we built them a system using a message queue (RabbitMQ) and then they removed the queue and put an RDBMS in its place, because of a lack of familiarity. When we delivered the system we had to prove the solution scaled which it did, easily processing over 20 million messages a day with a single small application server. However, they replaced RabbitMQ and put SQL server in the middle to act as the queue. Their reasons were similar in that they know SQL Server very well, but with that move their throughput went to around a little less than 5 million messages per day using the same small application server. Additionally, they found they had to write a huge amount of code to make up for the queue processing, failure states, dead letter queue etc. Sadly, they put it on their primary SQL Server box as well, which has degraded the performance of their other applications as well, and increased the SQL maintenance drastically. Also, moving between environments, dev, staging and production became a bigger chore too.

But even aside from pure performance, or deployment pains, a RDBMS is really a poor queue. You point out the locking issues, which you are absolutely right they can be a pain, but it is more than just that. Messages by their nature are highly volatile and transitional, which when ran through a database increases the pressure on the tempdb, the transaction log and overall general maintenance is increased many times over. It is tempting to use a RDBMS, because you have a strong background and team with a known skill set and you likely do not see all the differences between storing some messages in a table and a queue (yet). There is a world of difference though when you really start writing code and having to maintain it.

By no means am I knocking RDBMS systems, as they are very good for what they do, but queuing systems are superior for what they do. But to make one a queuing system, you need to write a large amount of robust code to handle dead letters, dropped messages, failure states, acknowledgements etc. All of which sound pretty easy on the surface but are really complex to do correctly. Queues have already figured this out, and been proven.

For example, lets say you have a service that picks up a message off an Amazon SQS queue, and then your service crashes before the message is processed. Based on a timeout value, SQS will automatically place that message back in the queue for the next service to pick up. If you do this in an RDBMS you will have to write all that logic into the database layer, or have other services monitoring and polling the database to figure it all out etc. This is actually quite a tough problem to solve correctly, so it is usually better left to one of the queuing technologies out there.

As for what queues to use, that is dependent on your environment and needs, but RabbitMQ is very good, very scalable and flexible. But there are other systems too, even Amazon SQS for AWS users. The only ones I would typically avoid or I would try to persuade most people to avoid are MSMQ and SQL Server Service Broker, as they are not, in my opinion, robust queuing platforms (Although SSSB isn't horrible).

Also, I believe this link was on HN a week or two ago, I didn't write it, nor do I know the author, but I believe the points are pretty valid and may help you: https://functionwhatwhat.com/why-you-should-not-use-an-rdbms...

-----


Wow. This is super exciting. What does this mean for multi tenant applications? Does this mean we can push our multi-tenancy logic down to the DB (if not logic, at least the last line of "data defence"). Can we create one DB user for each app-level account, ensure that all tables have the account identifier, and then enforce (at a DB level), that one account should be unable to access another account's data?

What does this mean for DB pooling? Is it possible to maintain a set of open DB connections where the account credentials can be applied, query executed, and the account credentials removed, with the connection going back into the pool?

-----


> Is it possible to maintain a set of open DB connections where the account credentials can be applied, query executed, and the account credentials removed, with the connection going back into the pool?

SET ROLE/RESET ROLE can be used for that, however for role A to be able to switch to role B, you need to "GRANT B TO A" which will lead to a combinatory explosion.

It'd probably be better to have a user with all roles granted as NOINHERIT (so the only thing it can do is SET ROLE) and all connections defaulting to that, then when you get a connection from the pool you "SET ROLE current_user" and you "RESET ROLE" the connection before storing it back.

I have not tested it.

-----


i do just that and it works like a charm.

-----


Also, what kind of hit does this have on performance?

-----


Based purely on the syntax they are showing, it seems like in the worst case scenario it should just be the equivalent of having a couple extra where clauses for each potentially matched row, so the impact should be nominal. The real question is whether the optimizer can actually make the queries faster than they would have been before if there are a huge number of rows, but only a small number that the user has permission to access. E.g. can it create an indexed column on the permissions.

-----


This is a topic that I'm actively researching on for my startup (www.vacationlabs.com) and would love to know what other experienced people think.

Our Rails app has grown over time and we ended up doing everything in a monolithic way to get stuff out faster. However, now the app is so big and tightly coupled that newer members of the team find it very hard to contribute. Therefore we are thinking of breaking it down into smaller apps which talk to each other using some sort of APIs -- either JSON/HTTP or MQ.

The MQ use case being considered is this -- the core transactional part of the system handles just that - maintaining tour availability and keeping track of payments. Everything else will be hived off to a mini-app which is notified of booking/payment related changes via a pub/sub model. So the email notification system will subscribe to the new-booking, booking-cancelled, payment-received, etc events and will send out emails appropriately. The billing system will subscribe to another set of events and will bill the customer appropriately. And so on.

Is this the right use case for an MQ? Is using an MQ worth the additional dev-ops related complexity that it brings along? in our case message delivery needs to be guaranteed, else extremely important business functions will not work. How do we deal the MQ unavailability? In normal cases if the DB is unavailable your system is down. Is this how MQ should also be treated? If not, how do you deal with the situation where the core DB transaction is complete, but for some reason you're unable to publish an event to the MQ? If the pub/sub system were built on top of the DB itself this problem would not arise because publishing an event would be part of your DB transaction itself.

Is there a sane way to build a pub/sub architecture on top of a DB, especially Postgres, which is what we're using. If not, any recommendations for which MQ we should be using for a guaranteed delivery pub/sub model?

-----


While it sounds like it could be a case for a queue, the fact that your workflow is mission-critical is itself a reason not to use one. In particular, a reason not to use RabbitMQ.

Some message queues are more reliable than others. RabbitMQ is designed to be clustered, and its handling of partition tolerance has been shown [1] to be very bad, something that I have experienced personally in a production system. Don't ever use it if losing messages will be a problem; and never use it with automatic acking (you'll want messages to be retried if your workers die mid-stream). RabbitMQ can be reliable if your boxes are all on a native (not cloud VM-based) LAN that is stable, and your machines don't occasionally get so overloaded that it impacts network connectivity.

One possibility is to use a message queue purely for signaling, not for state -- instead, use databases and APIs to transmit actual state. For example, let's say you want to shoot off an email every time there is a new booking. The "emailer" app listens to events published by the "booking" app. But the event doesn't contain any information about the booking; instead, the event simply says that there was a booking. When the "emailer" app receives this event, it asks the "booking" app for new bookings that it doesn't know about; it processes each booking, first recording that a (booking_id, email_id) row, then firing off the email, then committing that row.

This makes every participant in the workflow idempotent, because they can run the same piece of logic many times and still produce the same result. If you ever have a problem with the queue going down, you can simply execute the exact same code: You don't need to replay any missing events. You only need to worry about multiple listeners (multiple "emailer" workers) waking up from the same notification and doing the emailing for the same bookings. This is why you must transactionally update your email log table using database locks. You don't necessarily need to use database locks, but such a system needs some kind of locking to be absolutely atomic.

The nice thing about this workflow is that you can make it extra bulletproof by making the "check for bookings to email about" logic run, say, every ten minutes -- in addition to responding to the message queue events. If the queue isn't working, your app will still be doing the emailing, just a little slower. In other words, the queue simply becomes a trigger mechanism.

[1] http://aphyr.com/posts/315-call-me-maybe-rabbitmq

-----


Link doesn't work for me. Has it been moved?

-----

More

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: