
The Erlang Shell - strmpnk
https://medium.com/p/ab8d8bec3972
======
rdtsc
> When systems have faults due to concurrency and distribution, debuggers will
> not work.

Those are very tricky indeed, mix in threads with pointers and a system
becomes haunted. "This one customer noticed a crash, on this hardware, after
running for 10 weeks, but ... we couldn't reproduce it". "Oh I suspect there
is a memory overwrite or use after free that corrupted the data". People start
doubting their sanity -- "Oh I swear I saw it crash, that one time, or ...did
I, maybe I was tired...".

Someone (could have even been Jesper Andersen, the author) said that the
biggest performance gain an application gets is when it goes from not working
or being crashed to working. And the biggest slowdown is also when it goes
from working to crashing unexpectedly.

There was talk of 60 hour weeks here before, one of the things that happens at
8pm at night is people huddled over a keyboard debugging some of these hard to
track bugs. Managers and some programmers see it as great heroism, pats on the
back for everyone, but, others see it as reaping the fruits of previous bad
decisions and it is a rather sad sight for some. It all depends on the
perspective.

I guess the point is one of the main qualities of Erlang is not concurrency
but fault tolerance. Many systems copy the actor concurrency patterns and
claim "we have Erlang now but it is also faster!", and that is a good thing,
but I haven't heard too many claim "We copied Erlang's fault tolerance, and it
is even better than Erlang's!". [+]

[+] you can do the same pattern up to a point using OS processes, LXC
containers, separate servers, having watchdog processes restart workers, for
example.

~~~
chubot
Nitpick: mixing threads with pointers isn't really the problem; it's mixing
threads with shared mutable state.

A concurrency idiom I've gotten a lot of mileage out of is to use threads,
_without_ shared state, but _with_ pointers. Instead of shared state, you use
a message passing style where you pass pointers over queues. Threads only
mutate data that they are explicitly handed via a queue. After putting an
object on a queue, they NULL out their local reference.

It's basically like Go's goroutine idiom, but with C/C++/Python, real OS
threads, and a thread safe queue (which is all a channel is). There is also
some relation to the SEDA architecture.

The logical structure of such a program is not really that different than an
Erlang program, I imagine. It just works a little better with existing C/C++
codebases, and is potentially faster in some cases.

It has a completely different character than spaghetti written using threads
and shared state, and can be made extremely reliable. Especially because the
explicit channels provide a hook for testing black box behavior.

~~~
tbrownaw
_Instead of shared state, you use a message passing style where you pass
pointers over queues._

If you don't have a queue library and don't want to risk getting the locking
wrong, you can get one from _pipe(2)_. While simultaneous writes/reads are in
theory permitted to be interleaved, the implementation would have to be
actively insane to interleave anywhere other than page boundaries.

...I actually did this when I was at university, for the first class that
covered locking and concurrency. Amazingly enough, I didn't fail that
assignment for delegating what we were supposed to be learning to the OS.

 _The logical structure of such a program is not really that different than an
Erlang program, I imagine. It just works a little better with existing C /C++
codebases, and is potentially faster in some cases.

It has a completely different character than spaghetti written using threads
and shared state, and can be made extremely reliable._

Maybe this is why I don't get all the C++-hate? The things that make code
unreliable or thread-unsafe tend to also make it hard to read and think about
(even without threads), so I just find some other way that doesn't give me a
headache.

Interestingly code that doesn't give me a headache tends to have a strong
resemblance to functional-style code, even tho actual pure functional code
also tends to be mildly headache-inducing.

~~~
chubot
Yup, pipe() is actually a great, minimal solution. POSIX guarantees PIPE_BUF
to be at least 512 bytes, so you are guaranteed atomicity for any size
pointer. I think on Linux the unit of atomicity is 4096 bytes, which is a
page.

Yeah, there are a million ways to use C++. To be fair, most people have their
architecture forced upon them, rather than creating it from scratch. And the
C++ world does have a culture of spaghetti with shared mutable state and
threads. In Erlang you have the opposite culture. So I can understand the
association that people make with a language, even though it's more a matter
of culture.

In addition, C++ doesn't even have a thread safe queue in the STL (although I
was pointed to a draft for one in the next version of C++ by a coworker).
pipe() is a good solution I would use, but not portable to other OSes.

------
phamilton
I'm another recent Erlang convert (though I'm still trying to get used to the
paradigm and ecosystem). There are a few concepts that haven't quite clicked
for me.

I come from a Rails background. While Erlang's hot code reloading and the
ability to attach to running nodes is cool, I am not quite able to connect the
dots on how this is useful. In Rails, a process is short-lived by design (the
lifecycle of a request). Attaching to a process is useless, but the Rails
console essentially gives you an isolated process to inspect production code
and data. I don't see how extending that to an existing process would be
useful.

So, challenging my assumptions, perhaps short-lived processes aren't the best
approach for a web service. Perhaps that's the paradigm in Rails because Ruby
lacks the tooling to make a long-lived process easy to maintain. If a long-
lived process is as easy to maintain as short-lived processes, then suddenly
new architectures open up. How are these architectures different? What are the
benefits? What are some examples?

~~~
Jacqued
Since you recently started out, may I ask you what good resources you found to
get started with Erlang ?

~~~
Veratyr
[http://learnyousomeerlang.com/](http://learnyousomeerlang.com/) seems to be
the standard reference. I find it most amusing in both style and name (Learn
You Some Erlang -> LYSE -> Lies) as well as informative.

------
brickcap
> As an Erlang programmer I often claim that “You can’t pick parts of Erlang
> and then claim you have the same functionality. It all comes together as a
> whole”.

I am guilty of this. I am relatively new to erlang programming and I have
skipped on some of the features that make the language great. For instance I
have neglected learning both supervision and hot code reloading.

~~~
lostcolony
Hot code reloading I'd personally grant you as an exception; it's a
maintenance tool, not a development tool, and you can largely ignore it until
you want it, if you're adhering to OTP principles.

Ignoring supervision, however, is...not wise, if you have -any- sort of
reliability considerations (even ones as weak as "I'd prefer it to remain up";
if it's just a one off thing that you'll manually execute again if it fails,
meh, fine).

Supervision is really, really easy to use, at least for the base use case
(creating a good hierarchy, so that processes restart the other processes that
should be restarted, is an architectural consideration that can be a bit
hairy), and it's truly heartening to grep a log for something you've been
running a while, find crashes that indicate issues you want to fix (even
simple stuff, like sanitizing inputs), but the system is still running as
though nothing happened.

------
tel
I find it funny that the author willingly gave up static typing to have Erlang
process control—I made the opposite switch as I grew tired of the overhead and
attrition of modeling a complex domain in Erlang. I think the Erlang VM is
easily my favorite place to live as a programmer, but I do with Erlang-the-
language gave more.

~~~
lostcolony
What were your thoughts re: Dialyzer?

~~~
tel
It's nice and once I learned how to use it I tried to be very consistent with
it. The success types provide good documentation, but I found that the types
still felt very flat. The fundamental component of Erlang abstraction is the
process and the process tree, so your types reflect that and don't end up
composing well.

~~~
tel
(To clarify: rather your types end up composing much like messages between
actors. That's a very fine way to compose things, but it's still just _one_
way)

------
_pmf_
> And all this without service interruption. And you get all this for free,
> just because you picked Erlang as the implementation language.

GDB or even VS's debugger can do a lot of magic even if you use C. Until
absolutely necessary, leave debugging information reasonably high and
optimization reasonably low or off and you'll get an amazing, dynamic
environment to inspect a live C system.

------
skywhopper
Reading about "correct software" as if it were an realistic goal makes me
itch. Sure, we should be striving for as few bugs as possible, but any non-
trivial application will never be made "correct", if only because it'd be
impossible to write the full specification to compare it to.

------
rurban
Oh nice. They have now almost a Common Lisp environment. Getting closer to
Greenspun's tenth rule.

~~~
bitwalker
Is it actually possible to do hot code reloading while preserving the state of
the running system in Common Lisp? I'm genuinely curious, I kind of had this
impression that it was a feature fairly unique to Erlang.

~~~
agentultra
> Is it actually possible to do hot code reloading while preserving the state
> of the running system in Common Lisp?

Yes.

> I'm genuinely curious, I kind of had this impression that it was a feature
> fairly unique to Erlang.

It's not. iIRC there's even a version of the JVM with code reloading.

What's interesting about Common Lisp is that not only can code be dynamically
compiled into the running image but class definitions can be changed and live
instances will be updated without stopping the program or reloading anything.
There are a tonne of features in Common Lisp like this that are geared towards
robust, maintainable software.

~~~
bitwalker
> It's not. iIRC there's even a version of the JVM with code reloading.

I've seen some info about the JVM implementation, but I could've sworn it
wasn't able to preserve state while hot swapping. I haven't dug in to it
though, so it's totally possible it handles that.

> There are a tonne of features in Common Lisp like this that are geared
> towards robust, maintainable software.

I'm a fan of Lisp in general, though my experience is limited to a tiny bit of
Scheme and experimenting with Clojure. I'm glad to see that stuff like this is
available in at least one of the Lisp variants! Question for you (if you
program in Common Lisp): What are the main selling points? I spend a lot of
time already with a couple of languages, Elixir/Erlang (for fun), C#/F# (for
work) - what is it that would make me want to switch from one of those to
Common Lisp?

Thanks!

~~~
agentultra
In no specific order:

1\. Configurable compiler. You can tell the compiler where it's safe to ignore
types and where to be strict about types. For the battle-hardened critical
sections of code turning off run-time type checking can be a pretty big win
performance-wise.

2\. Extensible compiler. Macros are like tiny compilers. It's possible to
create DSLs for the programmer to work with a problem in terms of nouns and
verbs that make sense for their task while still compiling to VHDL for your
application. Common Lisp becomes the substrate on which you build the language
you need to solve your problem.

3\. The best error handling system I've seen in any language. Signals,
handlers, and restarts allow higher-level code to choose an appropriate
strategy for recovering from errors under various conditions without losing
state and restarting the computation.

4\. Top-class introspection tools. The JVM has been able to do some of this
stuff for a while but with CLOS and the error handling facilities available
Common Lisp makes this so much better. You can inspect any object, change
values on slots, trace any function, step through the entire stack, and
recompile code into the running image. There are nice tools written on top of
these facilities which provide some of the most elegant debugging sessions I
have ever encountered. Stop your program? Why? I can connect to a running
image, inspect the running program interactively, and compile the fix into it
without restarting. Even over SSH directly from my IDE.

And that's just the stuff in the specification. I'm a fan of the newer ASDF
systems for defining systems and their dependencies (implicit file-system
structure for defining module composition? No thank you). There are some
really solid libraries available (iolib, bordeaux-threads) and a very nice FFI
for interfacing with alien code. And various implementations provide
interesting extensions depending on what you need. Clozure Common Lisp, for
example, has really good native Cocoa bindings. SBCL has a very slick compiler
for generating fast code. Most of them provide threading extensions and there
are libraries which abstract away the differences if you're writing a library
that depends on threads.

~~~
bitwalker
Thanks for breaking it down! I'll have to investigate further, but there is a
lot here to like.

