
Elixir on Google Cloud Platform and App Engine - shalabhc
https://cloud.google.com/elixir/
======
dazuma
Hey all! I'm one of the engineers who worked on this. If anyone has questions
or wants to get involved, feel free to open an issue on the github repos.

* Elixir clients for Google APIs: [https://github.com/GoogleCloudPlatform/elixir-google-api](https://github.com/GoogleCloudPlatform/elixir-google-api)

* Elixir runtime for App Engine: [https://github.com/GoogleCloudPlatform/elixir-runtime](https://github.com/GoogleCloudPlatform/elixir-runtime)

* Elixir examples: [https://github.com/GoogleCloudPlatform/elixir-samples](https://github.com/GoogleCloudPlatform/elixir-samples)

~~~
matlin
Thank you for your work! Is this an indicator that Google is interested in or
using Elixir internally or is it just a response to growing requests from the
community?

~~~
DannyBee
The latter. But you should realize Google is pretty conservative internally in
the main repository because of the amount of code/tooling/etc. While we let
people experiment in the main repository, they are controlled experiments.
Even if we decided, tomorrow, to move the main repositories away from our
current set of programming languages, that would be a fairly long term path,
not a short one.

Outside of the main repository, or on google open source projects, or etc,
people can generally do what they want, they just don't get the benefits of
the above if they aren't using a language we already support very well.

~~~
hultner
So which languages are use internally in the main repository then?

~~~
snappyTertle
Java, C++, Python, Golang, and Swift/ObfC for iOS apps

~~~
pmontra
No JavaScript? Or is that because it's not used internally but in the public
internet?

~~~
Ambroos
Google's "big" client apps (Inbox / Gmail / Calendar) are a mix of Java
converted to JS (GWT) and native JS with the Closure library/compiler. I
think, anyway.

~~~
SZJX
Not sure if GWT is still that relevant nowadays. AngularJS was developed by
Google many years ago. It was already pure JS.

~~~
lozenge
Inbox by Gmail launched in 2014, long after Angular JS was first released, and
uses GWT.

------
ninjakeyboard
I wrote a book on Akka with many years of experience with it and I have been
using Elixir professionally for 7 months or so on greenfield/replatforming
efforts and I have to say that the BEAM VM is really an incredible piece of
technology that leaves Akka on the JVM in the dust in several aspects because
the VM is actor-model native. I wish there was a statically typed language
that ran on BEAM but Elixir and Erlang are great technologies to run in
production if you can accept some of the associated risk of newer eco-systems
in Elixir (but erlang has your back!).

~~~
ihumanable
You may already know about it, but if you are looking for a statically typed
language that runs on BEAM, you might want to check out Elchemy.

[https://github.com/wende/elchemy](https://github.com/wende/elchemy)

It allows you to write elm-like code that transpiles into easy to read and
idiomatic (for the most part) elixir code.

~~~
smt88
THANK YOU. I didn't know about this, myself.

I love the philosophy and robustness of the BEAM VM, but I can't go back to
the stone ages of a language with a poor (or non-existent) static type system.

~~~
dnautics
I don't find staticness to be the gating criterion for me. Go, for example,
has a static type system that's IMHO very poorly designed - for example, if
you want to multiply a time interval by some integer, you have to convert the
integer to a time type and then multiply, which breaks my poor science brain
(violates fundamental concepts of dimensional analysis). You also have to do
things like pick between implementing something with slices versus fixed size
arrays, and generic functions are not an option. I couldn't write fn
dot_product{n}(v [n]int){...} for example.

What you probably want is a _strong_ type system. Functional programming
languages (I'm thinking Julia as an archetype of strongly typed systems)
especially are really good at dispatching over types, and can make robust
optimizing inferences using a well-designed type system.

BEAM languages are strongly typed, but there are far fewer types (and no true
custom types) like julia. However, gating based on functional guards is simply
_amazing_ and more than makes up for a limited type system in terms of code
clarity. As for compile-time vs runtime error trapping - part of the point of
BEAM languages is "who cares?" An error doesn't bring down your VM. It gets
logged, you can see it, and apply a hot patch and fix it if you need to.

~~~
smt88
I like a type system that leans closer to Haskell than not. As someone else
mentioned, my users prefer me to catch errors before compile time :)

Expressive, algebraic types allow me to tell my compiler, IDE, and code heirs
what I'm trying to do so they know just by reading my code if I failed. That's
indispensible.

------
sergiotapia
Really excited to see Elixir going more mainstream. It's a hidden gem that
solves so many painpoints of the day to day programming. It's POWERFUL!

~~~
richjdsmith
Likewise. I havn't come across a case of someone getting to know the language
and disliking it. It's truly a great tool.

------
tabeth
Excellent!

I recall someone on here (HN) saying that many of Elixir's strengths are
weakened significantly because K8 does similar things (auto distribution, hot
reloading, etc.)

Any truth to this? Also, can you distribute nodes on VMs and deploy them to
Google Cloud?

\---

As a side note, is there a front-end equivalent to Elixir? Elm is functional
but is non intuitive. Elixirscript is a thing but seems a bit too early.

~~~
hanrelan
I've used Elixir/Phoenix on K8s and there's some truth to this. For example -
kubernetes' "if the process dies, kill the whole container and restart it"
model doesn't work very well with Elixir (a process within the VM can die and
K8s will have no idea). To some degree, K8s is replicating some of the BEAM
functionality.

But there's still a lot from Kubernetes that Elixir can take advantage of. For
example - service discovery, autoscaling, clustering, secret management - all
made easier with a K8s deployment and Elixir.

I wrote a few blog posts about it a while ago if you're interested:
[https://medium.com/polyscribe/a-complete-guide-to-
deploying-...](https://medium.com/polyscribe/a-complete-guide-to-deploying-
elixir-phoenix-applications-on-kubernetes-part-1-setting-up-d88b35b64dcd)

~~~
sb8244
Hmm I'm not quite sure I follow the process examine. If your started process
crashes (let's say supervisor fails to start due to too many failures), k8s
would restart the process (your app) to try again. If a managed process in
elixir dies, you could make that bring down your whole app, but generally you
would want to handle it yourself.

This is similar to an app server in Ruby. A single worker might die, but it's
still being managed by a central coordinator. If the coordinator dies,
everything is down and the app restarts

~~~
dguaraglia
Not trying to be flippant, but by extending your analogy: if the Kubernetes
supervisor dies (which is conceivable, no technology is infallible), then you
are in the same scenario as something bringing your whole Elixir VM down. You
can always run your Elixir VM using something like supervisord if you need
another watchdog.

BTW, it's very hard to bring BEAM down. Most exceptions will bring kill the
faulty process, but leave the supervisor tree untouched. Only "unmanaged" code
(for example, calling a C library using a NIF) might crash the whole VM.

Then again, no technology is infallible.

~~~
noemotion
>BTW, it's very hard to bring BEAM down. Most exceptions will bring kill the
faulty process, but leave the supervisor tree untouched. Only "unmanaged" code
(for example, calling a C library using a NIF) might crash the whole VM.

Not completely true. Try to allocate more memory than available and then see
what happens.

~~~
dguaraglia
I'm actually curious. Do you have some first-hand experience with that kind of
issue? I've only played around with Elixir and my only experience deploying
BEAM-based applications on production has been RabbitMQ and CouchDB and both
of those were very solid.

Also, isn't ENOMEM a standard error that you can handle using the standard
error handling techniques?

~~~
noemotion
I have and it wasn't good. It is impossible to handle out of memory exceptions
inside the BEAM. At best, you can hope for a crash of the VM. At worse, it
just hangs. And one will most likely not find out about this until they hit
this scenario.

~~~
lozenge
If BEAM is allocating memory it must either be copying a message from one
process to another, allocating space in a process for its stack, or for some
bookkeeping like to spawn a new process. In the first two cases why doesn't it
just kill the process and send the usual link and monitor messages to let
other Erlang processes handle the failure?

~~~
noemotion
How would it send a message if it can't allocate memory anymore?

~~~
dguaraglia
Haha, yeah, that's the problem with ENOMEM handling in general: unless you
have a very clear path to free memory _immediately_ and then try to do some
reasonable cleanup, you are screwed. Back in the bad old days of running
Windows machines with 8mb of memory, I'd religiously handle that kind of error
code... but I can't recall the last time I actually bothered. Commoditized
hardware makes us lazy.

------
mijoharas
Could someone explain what would make the Google cloud platform a better
deploy target for my elixir application than any other?

~~~
brightball
You’ll need to evaluate GCP’s peripheral offerings for database, APIs,
pricing, etc.

The biggest thing that this does is give you first class access to the
ecosystem APIs and with native setups for AppEngine (Heroku-ish), Compute
Engine (do it yourself on a VM) and a provider in Gigalixir which runs on GCP
that gives a Heroku-like experience but is tailored specifically to everything
Elixir needs (hot loading, clustering out of the box, etc).

Between those two things, GCP is pretty strong.

~~~
mijoharas
Thanks for the nice answer!

I was reading up on Gigalixir thanks to this thread, is it basically a more
"batteries included" version of AppEngine specifically targetted for Elixir?

~~~
jesses
I'm the founder of gigalixir.com and happy to answer any questions you might
have. Also feel free to email me directly at jesse@gigalixir.com

------
ofek
Discord (a heavy Erlang/Elixir user) runs on GCP, right?

~~~
jhgg
Yes we do! We’ve been able to shake out a fair amount of platform bugs that
the Beam VM had on GCP too! Thanks to the google team for taking them all
seriously and fixing them. Some were particularly nasty :)

~~~
brightball
Thanks for doing the hard work so we don’t have too. :)

------
pselbert
Surprising development. I wonder if this will complement Gigalixir or compete
with it.

The ability to connect between nodes, avoid mandatory restart, avoid
connection limits and keep websocket connections alive easily gives compute
engine a significant advantage over platforms like Heroku.

~~~
holografix
Disclaimer: I just got a job with Heroku

As far as I understand Heroku lets you connect between nodes (Dynos within a
private space with dns discovery [0]) and also supports websockets [1]... for
the mandatory restart, wouldn't you want to design your process as stateless
anyway?

[0] [https://devcenter.heroku.com/articles/dyno-dns-service-
disco...](https://devcenter.heroku.com/articles/dyno-dns-service-discovery)
[1]
[https://devcenter.heroku.com/articles/websockets](https://devcenter.heroku.com/articles/websockets)

Sorry if I'm missing something obvious!

~~~
pselbert
Typically I design applications to be as stateless and tolerant of restarts as
possible, as they do get restarted during deploys anyhow. That is a minor
issue compared to the difficulties around connection restarts and
communication between nodes.

It is wonderful that Heroku now offers exec, which at least allows using
observer and other remote debugging tools. The tooling keeps getting
better—but there is no denying that it was designed for Ruby deployment, and
working the BEAM has significant production differences.

------
jshen
This is really cool and all, but I really wish Google Cloud would move things
out of beta faster. Cloud Functions are still in beta despite them being GA at
both AWS and Azure for a while now. Enterprises typically won't use beta
software which makes it nearly impossible to pick Google Cloud for an
enterprise.

But hey, they have Elixir support !?!?

------
qaq
Nice Elixir is becoming more mainstream

------
Exuma
Wow this is cool. Does it take advantage of Erlang's distributed
networking/nodes feature?

~~~
dazuma
We're not doing anything special to help set up distributed Erlang at the
moment. It would probably be difficult with App Engine because it's designed
to manage stateless containers. But I could imagine setting up such a network
with Kubernetes Engine, or definitely with Compute Engine VMs.

~~~
sb8244
At elixir ATL meetup we setup an elixir app on GKE and connected the nodes via
peerage in under 2 hours. Definitely possible!

~~~
brightball
How often do y’all meet up? I’m in Greenville SC but I could see a road trip
happening periodically with the Upstate Elixir group.

~~~
sb8244
It's once per month.
[https://www.meetup.com/atlantaelixir/](https://www.meetup.com/atlantaelixir/)

It's a pretty small group right now. I'm hoping that a new venue and push will
help bring in more people.

------
beckler
Okay.

I spent a lot of time last week trying to get elixir setup on gcloud, but
variables were the one thing I could never figure out and I eventually gave
up.

Is there some magical way to get variables and secrets in gcloud without
having to commit them to a repo?

~~~
derefr
Deploy to Google Kubernetes Engine; `kubectl create secret`; set the env-vars
of your Deployment to be derived from the secret.

~~~
skrebbel
Wait, you can't put secrets in GCP without first learning Kubernetes? That's
pretty messed up :/

~~~
derefr
More like, Kubernetes (and Google Kubernetes Engine) is the right level of
abstraction to actually have a first-class object-type of "secret."

Compute Engine is just raw IaaS, like EC2. You "store secrets" on GCE by
writing them to a volume and attaching it across your cluster, or putting them
in a common database. Or just pushing data onto your instances after they're
up using Fabric or something. You know, just like if they were real computers.

App Engine is made to be a complete and self-contained PaaS (much moreso than
anything like Heroku), to the point that it doesn't really _use_ external
credentials for anything. For example, you can access BigQuery tables from App
Engine without supplying any "BigQuery credentials." Oddly enough, if you _do_
need to connect to third-party services (or, say, Google Cloud SQL) from App
Engine, they recommend that you store the credentials to do so in... a
BigQuery table. GAE is proprietary and weird. It's basically its own "abstract
machine."

Kubernetes Engine, in comparison, is exactly for the use-case of "deploying
workloads" of regular potentially-stateful, potentially-long-lived POSIX apps
that need things like DB connection strings passed to them. If you're a modern
backend developer who burns Docker images like people 10 years ago burned CDs,
Kubernetes is the service you _should_ be using on any compute cloud you
interact with. Everything else is an impedance mismatch.

~~~
Aduket
thanks for the summary.

------
bpicolo
That’s a really interesting addition to app engine. Great idea too - try to be
first to an up and coming platform rather than try to move applications that
have deployed in one way for a decade.

------
skybrian
This seems to be using App Engine's "flexible" environment. It's less
appealing to me than "real" App Engine since as far as I know there's no free
tier.

------
keithnoizu
I'll have to look this over, I'm running a server cluster on GCE + Appengine
currently aggregating and triggering alerts against about 300k device readings
per minute. Huge pain point keeping things synchronized between GCE on java
and the elixir backend.

------
theshank
This is super exciting!

