
Ask HN: Automating large Enterprise business operations - automator
Hi HN.
I&#x27;d like some advice about automating business operations - what to consider, what technologies or framework to look out for etc. For background, I&#x27;m working for an ICT service provider, and just got promoted to look into enterprise-wide automation and system. My team is small, but I report directly to the C-level. Frankly, I&#x27;m a hobbyist programmer, and was only promoted to this post due to my enthusiasm and small track records i.e. handling CSV files for analysis etc.<p>The ultimate goals are to improve lead time in service delivery, reduce aging tickets, and reduce cost. I currently see gap in performance management, where the bosses do not have clear visibility or simply do not trust the performance metrics.<p>Simultaneously, there are lean internal consultants doing studies and making recommendations. Also, middle managers are being trained in Six Sigma, acquiring &quot;belts&quot;.<p>To reiterate, how do I start assessing the situation and what kind of technologies or methodologies associated with this scenario? Is it SOA, workflow engine, ERP etc? How feasible is using standard web framework i.e. Django or Flask for business automation?<p>Thanks.
======
SEJeff
This is a LOT to read up on, but seriously consider mesos + aurora. It is
really incredible stuff and Twitter uses it for huge production environment.
The Mesosphere guys have better documentation with Marathon, but it has a lot
of catching up to get to the feature set of Aurora.

I happen to be the tech lead of a team that builds internal tools for an
"enterprise" company primarily using Django and Flask. Build it and then show
the business how it is useful. Sadly, that means you might need to write a lot
of high quality docs or high quality clients (if you write an api). I tend to
favor "api first" development.

~~~
automator
who ended up maintaining the internal tools? do you hand it off to the IT
team?

~~~
SEJeff
My team, who writes those tools for the 'IT Team', but really the entire
company ends up consuming them.

Our company's bread and butter is low latency stuff and we're a small team of
webdevs who exist in the IT side of the company, but code fulltime. I quite
enjoy it actually.

------
mindcrime
That's a pretty big, open question. There's a ton of literature out there that
you could (and probably should) read, and no simple HN post can do justice to
the scope of process automation / BPM / workflow, etc.

For some background on business process / workflow automation, a few good
books to review would include:

[http://www.amazon.com/Essential-Business-Process-Modeling-
Mi...](http://www.amazon.com/Essential-Business-Process-Modeling-
Michael/dp/0596008430/ref=sr_1_31?s=books&ie=UTF8&qid=1419303981&sr=1-31&keywords=bpm)

[http://www.amazon.com/Fundamentals-Business-Process-
Manageme...](http://www.amazon.com/Fundamentals-Business-Process-Management-
Marlon/dp/3642331424/ref=sr_1_34?s=books&ie=UTF8&qid=1419304137&sr=1-34&keywords=workflow+automation)

[http://www.amazon.com/Smart-Enough-Systems-Competitive-
Autom...](http://www.amazon.com/Smart-Enough-Systems-Competitive-
Automating/dp/0132347962/ref=sr_1_1?s=books&ie=UTF8&qid=1419304165&sr=1-1&keywords=smart+enough+systems)

[http://www.amazon.com/Social-BPM-Bpm-Workflow-
Handbook/dp/14...](http://www.amazon.com/Social-BPM-Bpm-Workflow-
Handbook/dp/1461146305/ref=sr_1_1?s=books&ie=UTF8&qid=1419300569&sr=1-1&keywords=social+bpm)

[http://www.amazon.com/iBPMS-Intelligent-Systems-Workflow-
Han...](http://www.amazon.com/iBPMS-Intelligent-Systems-Workflow-
Handbook/dp/0984976469/ref=sr_1_2?s=books&ie=UTF8&qid=1419300569&sr=1-2&keywords=social+bpm)

[http://www.amazon.com/Real-Life-BPMN-Analyze-Automate-
Proces...](http://www.amazon.com/Real-Life-BPMN-Analyze-Automate-
Processes/dp/1480034983/ref=pd_sim_b_8?ie=UTF8&refRID=1YW0HA39FP97W6RBE47T)

That said... for starters, I'd back up to the issue of truly understanding the
business need you're trying to address, and figure out a methodology for
mapping the technological capabilities you will implement, to the business
strategy. To that end, I'm a fan of a technique called "capability cases", and
actually wrote a series of blog posts on that topic a while back. I'll refer
you there for more on that topic:

[http://fogbeam.blogspot.com/2013/01/why-capability-cases-
are...](http://fogbeam.blogspot.com/2013/01/why-capability-cases-are-must-
when.html)

[http://fogbeam.blogspot.com/2013/02/so-what-is-capability-
ca...](http://fogbeam.blogspot.com/2013/02/so-what-is-capability-case-
anyway.html)

[http://fogbeam.blogspot.com/2013/05/capability-cases-part-
th...](http://fogbeam.blogspot.com/2013/05/capability-cases-part-three-
sample.html)

The other positive thing about the Capability Cases methodology is that it
helps IT people learn to "speak the language" of LOB executives (and, to some
extent, vice versa, if everybody buys in), and helps bridge that gap between
"the business side of things" and "the IT side of things".

Anyway, if you get through all that and decide that some sort of BPM system is
for you, there are a handful of high quality, popular, Open Source BPM engines
out there that you could consider. If nothing else, they could enable you to
get to a good POC quickly and without spending a bazillion dollars. One
popular option is Activiti:

[http://www.activiti.org](http://www.activiti.org)

It uses the industry standing BPMN2 notation. Stay away from anything that
mentions BPEL, and stay away from Oracle products. Especially Oracle products
that involve BPEL.

Depending on how complex your needs are, you might not actually need a
dedicated BPM / workflow engine. In that case, you could certainly code up
automation code in any general purpose language of choice. But be careful of
going down the path where you wind up rebuilding the wheel.

I'd also advocate taking a careful look at the idea of "Social BPM", which
focuses on making BPM systems more flexible and less rigid, and introducing
aspects involving social connections between humans.

I'm not trying to be overly self-promotional here, but I will refer you again
to a couple of posts I have previously written on this subject, which may be
of use to you:

[http://fogbeam.blogspot.com/2013/05/social-events-bpm-oh-
my-...](http://fogbeam.blogspot.com/2013/05/social-events-bpm-oh-my-but-what-
about.html)

[http://fogbeam.blogspot.com/2014/02/on-solving-social-
aspect...](http://fogbeam.blogspot.com/2014/02/on-solving-social-aspect-of-
bpm.html)

Like I said, this is a big topic. I could talk about this stuff for days on
end, or until you died from boredom. :-)

Seriously though, if I can be of any further assistance, feel free to drop me
an email.

~~~
phonon
What do you think about [http://viewflow.io/](http://viewflow.io/)
[https://github.com/viewflow/viewflow](https://github.com/viewflow/viewflow) ?

~~~
automator
Looks interesting. Any testimonials? Or community where it is discussed about?

~~~
kmmbvnr_
Let me give a few notes here as i'm the author of django-viewflow. I stumbled
with workflow problems in 2004, wrote simple django-fsm library in 2010, and
decide that I could do more in 2013 with django-viewflow.

Common web frameworks like django or flask solves only technical problem
related to HTTP/Client/Server functionality and nothing related to business
logic organization except simple CRUD support.

It works pretty well if you going to create data entry applications for your
business. Data entry applications allows end users to enter the data and do
some queries and reporting. Basically you are offer for end user a set of
forms for data entry, grid for end user queries and set of reports for things
that couldn’t be achieved with simple grid filtering.

The usage pattern for worker is to enter the data, run the query/report and
decide what to do next. This style of applications allows unlimited ways for
interaction for end users, and adopts many scenarios.

But very fast it became not enough. The breakpoint happens when you came with
idea, that you need something like `STATE` or `STATUS` field for one of you
entity. This probably happens when you try to organize workers pipeline and
collaboration.

And this ruins your well designed CRUD application. Mix of hidden state
changes and crazy querying randomly scattered over the code makes hard to have
a whole picture what really happens. After a half of the year, team of 5
developers could produce working but cluttered application that really hard to
change to adopt new business process requirements.

The core problem that common web MVC pattern have nothing related to people
collaboration. Fortunately there was a lot of research in this field, starting
from 198x.

And the answer is - we need an explicit workflow layer (yeah, so pythonic)

You can take a look to
[http://www.workflowpatterns.com/](http://www.workflowpatterns.com/) It’s
pretty formal and hard to read, but it contains set of common patterns that
happens when you trying to describe people workflow.

From historic point of view, there was discussion - what is the best way to
represent people collaboration visually, what building blocks should be used
and etc. In the 2004 there was several popular approaches for this like ebXML,
BPEL, and many other formats.

At the 2014 we have the answer - only BPMN products became successful with the
time. I think you will not very surprised with this fact, if you will know
that BPMN is the only format designed for understanding by humans first. Other
formats was designed like bad visual programming language and they are failed.

The core difference of BPMN that you have a gate nodes explicitly separated
out of task nodes. This give you unlimited amount of flexibility, and most
systems allows to skilled programmer develop very custom gate logic to
accommodate all process requirements, embedded into reusable flow building
block.

And the last part. The rise of workflow systems happens in the 200x. There was
the time when UI-only programs declared as the way to happy user experience.
Missing UI tools for system administration was declared as main problem of
Linux operation system. Nowadays, UI for administration mostly failed. Today
is the rise of script languages like Chef, Salt, Puppet, and DevOps as
profession for system administration.

This sort of things became happens even with existing workflow tools, like
jPBM based [http://camunda.org/](http://camunda.org/)

I believe that should happens with any existing workflow software. With good
designed library or language for workflow specification, developing
application would be faster than any ui programming.

So, as the result of thoughts described above, Viewflow is the library that
offers an additional layer for django web framework allows explicitly specify
people workflow and takes out the workflow logic from django views.

You can take a look for shipment views example, when I start i didn’t expect
that so complex workflow could be exality extracted out of the views -
[https://github.com/viewflow/viewflow/blob/master/tests/examp...](https://github.com/viewflow/viewflow/blob/master/tests/examples/shipment/views.py)

The future of workflow looks like we will have not only well specified
business process but mix of process and cases to accommodate best things from
data entry and workflow application styles. There is even existing standard
for this - CMMN. And it looks like it would be very soon implemented in many
workflow tools.

