
Amazon Simple Workflow - Cloud-Based Workflow Management - jeffbarr
http://aws.typepad.com/aws/2012/02/amazon-simple-workflow-cloud-based-workflow-management.html
======
j45
First thought: D'oh, why didn't I think of this?

New resolution: consider everything I do from a cloud based perspective.

My Experience: A few years ago I considered windows workflow and built my own
simplified version for a project. WF tooling at that time was still evolving.
The requirements were all over the place and I thought it might be prudent in
this case to build something to expect the worst.

As Murphy usually does, the worst never came almost 5 years later. The
workflow engine works well, and simply enough for a simple event based system.
It's more of a routing system and I'm fine with it.

Where ASW catches my eye is the admin tools. Like most apps the admin tools
are among the most complex parts of the app. In ASW, so far I'm recognizing a
certain simplicity.. in a generalized sense for a workflow system.

If I tried to do the same in another workflow technology, I would almost
certainly be running into more configuration upfront, and the admin tooling to
create and manage workflows from a UI perspective would likely be either too
basic or to overkill for what I needed.

That's the crux of generalizing a solution to a problem, it risks being so
much bigger than we need that it adds overhead to what we were liking to
simplify.

ASW initially strikes me as being much simpler. It could be useful to managing
state simply across distributed applications, specifically in the cloud. Not
sure how the pricing would work though, that seems like a lot of money to pay
for logic gates and a state machine, but I could be trivializing.

Now.. just to think up a simple fun utility or project to try this out with..
:)

------
dbecker
Using (and even researching) new AWS services has always been surprisingly
frustrating to me.

You have to sift through buzzword-filled ad copy and corporate-speak to find
any explanation of what the product is or how to use it.

------
bwarp
At the risk of sounding like a Microsoft shill (I'm not), you can already have
your cake and eat it if you use Windows Workflow on top of AppFabric.

It has an arguably better feature set (including extensive transaction,
correlation, failover, load balancing support), is extremely robust and can be
deployed on EC2 or your own infrastructure (or both at the same time - you can
scale your workers up on EC2 for example). The learning curve to do something
useful is about 2 days.

It's pretty cool actually.

~~~
dbattaglia
_> At the risk of sounding like a Microsoft shill (I'm not), you can already
have your cake and eat it if you use Windows Workflow on top of AppFabric._

I've been researching/demoing App Fabric at my job for a few weeks now to
possibly use for our new SOA layer, and that was the first thing I thought
when I saw the SWF email from Amazon this morning. Of course anyone not
working in IIS or Azure (ie Linux or AWS) would get no benefit out of App
Fabric, and this could be an excellent alternative regardless (judging by the
high quality of Amazon's other services I mean).

------
redguava
I really love this new tool. I think Amazon has some amazing technologies and
services.

Why don't they hire a designer and awesome UX guy and fix their terrible
interfaces?

I know most people use the API's, and I realise it's not really stifling their
business, but I still don't see why a little more effort can't go there. I am
sure there are a lot of people around the world that would be very happy for
the console to have some more features and be a bit easier to use.

~~~
spydum
This has puzzled me for quite a while as well. The AWS Console isn't just
awkward, it's lacking some of the features the API exposes if I recall.

~~~
ceejayoz
Yep. It seems to be impossible to delete an SSL certificate for an ELB via the
console, for example. You can set one up, but not remove it - for that you
need the CLI tools.

------
pieter
This really sounds too enterprisey. It's unclear to me what the benefit is of
using SWF, and lost my interest when it started listing business definitions.

~~~
flyt
"too enterprisey" = a service that large businesses can integrate deeply into
their existing applications, Amazon can bill them for, and yet another
compelling reason for companies with obscene amounts of money to dip their
toes into AWS.

Look at this feature next to the AWS Storage Gateway, recently announced:
these are features to get businesses using portions of AWS so that they
demonstrate their reliability and cost effectiveness at a small scale. Once a
business is comfortable dealing with Amazon then they provide compelling
reasons to expand into more of their services (EBS booting + attaching Storage
Gateway volumes to EC2 instances for braindead simple cloud compute migrations
away from expensive, inflexible on-premises datacenters, SES, SQS, SNS
integration with SWF, etc)

Never forget that Amazon is playing the long game.

------
meanguy
Depends on your volume, of course, but seems expensive. 1 million jobs
scheduled, 4 steps per job? $200. And you're locked into their API.

You can have 8 workers each polling an Amazon SQS queue once a second for
$19/month.

Having had huge problems with their queues (3 minute lags, spurious
duplicates) I'm wary of Amazon's polling-based API. SWF's long-poll feature is
smart versus SQS, however.

------
newhouseb
I think this is meant to be a parallel to things like celery and gearman, i.e.
a task queue. A table mapping between AWS services and what components they
replace in a typical DIY stack would be helpful for a lot of people (i.e. SQS
= RabbitMQ, SWF = celery, ELB = nginx, DynamoDB = Cassandra(?)).

~~~
JeffJenkins
Cerery doesn't do workflows with dependencies, just tasks. This is more like
Storm: <https://github.com/nathanmarz/storm>

~~~
newhouseb
Well celery does have subtasks that it can wait and join on (see
<http://ask.github.com/celery/userguide/tasksets.html>), but I suppose that
the logic to do such things would have to occur within a running task as
opposed to SWF where such things are handled by the platform itself.

------
joshwa
I'm actually in the middle of designing a Workflow-as-a-service for my current
employer (based on jBPM 5).

However, this isn't Workflow-with-a-capital-W. It's not even a state machine.
SWF is actually a distributed-processing task queue with persistent state and
execution history. Which is pretty cool.

------
verelo
This is a great idea! Actually this may have just saved me a bunch of coding.

Amazon you're starting to rule the world and its a bit scary...but for now i'm
ok with it.

~~~
jeffbarr
Have fun with it, and let me know what you come up with.

------
Krish123456
This is Amazon's attempt at next gen PaaS. You can find more details here
[http://www.cloudave.com/17315/remember-next-gen-paas-and-
aws...](http://www.cloudave.com/17315/remember-next-gen-paas-and-aws-here-is-
the-second-piece-to-the-puzzle/)

~~~
ridruejo
Yep, your analysis is spot on. Good article

------
Skywing
This looks very applicable to what I'm working on right now. I've built my own
naive implementation of a distributed worker queue, using redis as the basic
centralized queue. There are many edge cases where redis lpush and rpop will
let me down, so as I build my tool, I know that I'll eventually have to move
to an actual message queue.

I've stayed away from RabbitMQ, or even something like ASW, while developing
this project of mine because they're a lot of work to integrate, on their own.
ASW looks awesome but it looks as though it'd take all of my focus to
integrate it. It's something I wish I could use right now but don't think I'll
be able to until I've finished most of my development.

I guess I'm just saying that ASW looks complex.

------
nubela
I don't get it, can someone explain what problem it is trying to solve? By
means of an example?

------
amalag
This is very cool. We are working on a website for a complex workflow. This
could take all of the business logic out of the web application (done via
rails state plugins) and move it to the amazon service. Then just make semi-
dumb forms to accomplish each state. I think that is the goal of this, you
just have to code individual pages which coordinate with your database models
and hit the API as the business layer.

------
chubot
Looks cool, but the link to the API reference is missing. I'd like to see some
code examples and how you write non-java workflows.

~~~
zootm
From the product page: <http://aws.amazon.com/documentation/swf/>

------
troels
It sounds scary to me to have something that core to your business be running
on a rented platform.

~~~
jwilliams
Have you mapped out what services your business consumes and the
relationships? e.g. Your bank accounts, payment systems, deliveries,
suppliers. I think you'd be surprised.

------
davidw
Interesting. I just implemented a workflow thing for LiberWriter. It's a
little unclear how the Amazon system might integrate with, say, an open ended
comment system. Definitely something I'll be keeping an eye on though.

------
jbarham
I didn't see it linked from the blog post, so here's the official SWF page:
<http://aws.amazon.com/swf/>

