
Infrastructure as code - bimil
http://www.alexgallego.org/random/thoughts/2015/03/28/infrastructure-as-code.html
======
pquerna
The crew at Mailgun are kinda doing this, and its been enabled by vulcan
proxy:

[https://github.com/mailgun/vulcand](https://github.com/mailgun/vulcand)

At first, sure, Vulcand is just another reverse proxy, but what they have been
doing is registering individual HTTP handlers with the proxy -- into etcd:

[https://github.com/mailgun/scroll](https://github.com/mailgun/scroll)

(there is a python implementation somehwere too, just can't find it)

What's neat about exposing these individual HTTP handlers, is now your reverse
proxy can produce metrics, apply circuit breakers, etc, all in a central way,
but your "micro" service just has to register with etcd:

[http://blog.vulcanproxy.com/vulcand-news-circuit-breakers-
be...](http://blog.vulcanproxy.com/vulcand-news-circuit-breakers-better-
metrics-and-anomaly-api/)

So, you end up with an architectural style where you can deploy a single HTTP
handler as its own service, similar to what the article was pointing to, but
in a multi-lanaguage approach, where HTTP is the communication method.

~~~
agallego
Very similar :)

We do use a proxy/router. The main difference is that instead of using etcd
for service discovery we are deeply integrated with Mesos.

Our framework scheduler does the registration/deregistration, resource
allocation/colocation, etc.

The main reason for a proxy at this point is to have unified metrics
collection, distributed routing (each reverse proxy knows only its immediately
connected downstream operators), tracing (think dapper), and facilitating the
communication with other languages (c++, java, go, js - node, ruby, etc).

------
beat
The problem is that there's a certain essential minimum complexity to every
interesting piece of software. You can't eliminate essential complexity - you
can only move it around.

Monolithic architecture turns a configuration management problem into a coding
problem. Eventually, coupling within the monolith makes it hard to develop.

Service-oriented architecture turns a coding problem into a configuration
management problem. Eventually, the potential combinations of small services
become unmanageable and untestable, making it hard to run operationally.

You have two kneecaps. Which one gets the bullet? Because you're gonna get
kneecapped either way.

~~~
tekacs
Unless you have a framework sitting over the lot, defining that some set of
(service, interface, version) tuples collectively makes up an 'application'.

This way, you get most-every benefit of SOA whilst being able to reason about
an application as a whole.

That's the approach we're taking with our microservices framework, wym:
[http://wym.io/](http://wym.io/)

~~~
beat
You add a layer of complexity to try to manage the complexity. Which works,
but it ain't a free lunch.

~~~
tekacs
Yes, absolutely. Much in the same way that web frameworks do this. :)

The complexity is handed off to the framework authors/community and (web)
development is revolutionised for everyone else.

------
chriswarbo
It's a nice line of thought, but I think there's a lot of existing work that's
should be investigated. Essentially, it seems like another "wouldn't it be
nice if everything were written in $LANG, even the OS?".

Erlang is mentioned as an example, since its VM certainly offers many features
we'd like from an OS. Maybe comparisons to Smalltalk, LISP machines and even
regular UNIX shells should be offered.

It's likely that all this has been done before, and I'm tempted to think that
its major problems would stem from having too much power in the config
language. For example, I'd imagine you could do this with SysV init's
runlevels and shell scripts, but it would be pretty horrible. What would make
the language du jour any less horrible?

Also related is, of course, the Nix ecosystem: Nix package management, NixOS
configuration management, NixOps provisioning, DisNix distributed services,
etc.

~~~
skrebbel
> _Essentially, it seems like another "wouldn't it be nice if everything were
> written in $LANG, even the OS?"._

In all honesty, I'm not sure you read the article. The entire point is the
idea to deploy tiny chunks of code, independently, in _any_ language.

~~~
chriswarbo
> The entire point is the idea to deploy tiny chunks of code, independently,
> in any language.

What does "tiny chunk of code" mean? According to the article:

> Literally, a simple interface that did one (or very few) things well.

Hmm, sounds like the UNIX philosophy to me. Operating on code in a language-
agnostic way? Sounds like scripting to me...

> What if instead you could deploy classes.

Classes are a language feature, so this looks to be asking for an alternative
scripting language.

> What if your operating system was in a way an API to deploy services (it
> is!) > but the size of the code deployed was so small that it would in turn
> be hard > to make mistakes.

That's the point of UNIX: do one thing and do it well; compose the pieces
using scripts.

Of course, UNIX is not perfect, it's just an obvious comparison. I mentioned a
few others above. That's what I got from the article, after squinting through
the jargon ("deploy" instead of execute, "services" instead of processes,
etc.)

------
dugmartin
This sounds a lot like CORBA
([http://en.wikipedia.org/wiki/Common_Object_Request_Broker_Ar...](http://en.wikipedia.org/wiki/Common_Object_Request_Broker_Architecture)).

The big problem with distributed classes is that network communication links
are fragile and you need to bake how to deal with failures into your
methodology. Erlang handles this in a way that Joe Armstrong explains here:

[http://armstrongonsoftware.blogspot.com/2008/05/road-we-
didn...](http://armstrongonsoftware.blogspot.com/2008/05/road-we-didnt-go-
down.html)

~~~
kyllo
It also brings up Fowler's First Law of Distributed Object Design: "Don't
distribute your objects."

------
marcosdumay
I don't get it. Is that yet another CORBA clone?

I think the CORBA clone currently in fashion is SOAP (or did we get something
newer?). There probably are SOAP tools for Earlang, and you'll really get the
possibility of using an heterogeneous system, like you said you'll want.

Yet, those things never work so well as people think they should. Keeping
interoperability within a code-base is hard work, distributing it just makes
it harder.

~~~
agallego
> I don't get it. Is that yet another CORBA clone?

Happy to explain what you don't get. Definitely not a CORBA clone.

~~~
marcosdumay
What I don't get is exaclty this. What's the difference of this to remote
procedure calls?

------
jkot
For this I would go with some J2EE or clustered framework. Erlang is nice, but
is outdated and does not handle clustering that well. It is about migrating
clients and threads, while multiple versions of the same library exist within
system.

> _The size of the code deployed matters_

And I disagree with this. The hardest deployment I ever seen was 2KB of code.
Sim toolkit is impossible to modify once it is flashed into 2 million SIM
cards. :-)

~~~
hassy
In what sense does Erlang not handle clustering well? It's true that you
wouldn't run a "native" (ie EPMD-based) cluster of hundreds of nodes, but what
would stop you from implementing a clustering architecture you'd build with,
say, J2EE in Erlang?

------
macca321
Sounds like you're looking for a distributed actor framework like AKKA

~~~
agallego
I would agree that on the surface the problem does seem to be solved by an
actor framework.

The issue and main point is that I want that cross language, cross platform
with isolation, tracing and debugging hooks.

------
kirab
Maybe D-Bus fulfils the requirements of OP?

