
Elixir web development 101: collaborative todolist with realtime updates - blaze33
https://blog.openbloc.fr/elixir-phoenix-web-development-101-todo-app/
======
whalesalad
I'm so excited about Elixir these days. I would love to see more cookbook
examples of using Elixir but NOT as a Ruby/Rails replacement.

While the concept is not new to me, I have recently become totally infatuated
with the concept of green threads, actor models, etc... for dealing with
concurrent processing. This is literally what Erlang was designed to do, and
Elixir makes it user friendly.

The real beauty of Elixir/Erlang is being able to run hundreds of thousands of
concurrent "threads" (co-operative in userspace, not OS-level).

Anyway I do not mean to detract from the post here -- just feel like a todo
list is not a shining-star example of Elixir and would love to see some more
hackers post their concurrency work here.

~~~
jetti
>I would love to see more cookbook examples of using Elixir but NOT as a
Ruby/Rails replacement.

I would love to see more examples that get away from web dev with Elixir.
While it's great with the Phoenix framework there is so much more to Elixir
than that. Trying to deploy an elixir daemon has been a pain and I haven't
found much around that. There is escript and distillery and while I haven't
tried distillery for non-phoenix apps, escript works but deploying is still a
pain.

~~~
napsterbr
Hey, I've went through the "trouble" of using continuous deployment with
elixir and distillery.

Since the project I'm working on is open source, you can see how I've done it.
It's still far from perfect but maybe it could be a place to start.

Elixir app is on
[https://github.com/hackerexperience/helix](https://github.com/hackerexperience/helix).
You'd probably be interested on `lib/release.ex` and `rel/`.

For the ci, our infrastructure playbooks are at
[https://github.com/hackerexperience/infrastructure](https://github.com/hackerexperience/infrastructure),
and you probably want to look at `roles/{helix, erlang, elixir}`.

Hit me up if you have questions our would like to discuss further. Email is on
profile.

Btw, here's a great presentation about the topic:
[https://youtube.com/watch?v=H686MDn4Lo8](https://youtube.com/watch?v=H686MDn4Lo8)

~~~
jetti
Thanks for this, I'll take a look and definitely hit you up if I have
questions.

------
pmontra
> channel.on('update:todo' ...

> channel.push('delete:todo' ...

> Have a CRUD interface over websocket

REST over websockets. What's the advantage over HTTP? I know websockets allow
for pushing data to clients, but this is pull so apparently there should be no
advantage, only more code to write.

Edit: maybe you're sharing updates among all the users connected to the
server. Still, for sending requests HTTP is enough. Any performance advantage
by using websockets?

~~~
oelmekki
This is interesting when you want near realtime performances, as you don't
have all the culprint of a http request : you basically just send a few octets
(a json document, or even just a simple value), and you don't have to build a
request and negotiate a connection.

This is not something obvious because we have helpers to make http requests,
but there are many things going on while issuing a http request (building
headers, encoding body, etc) [1]. A socket (be it a websocket or a classic
socket) has this advantage that all the negotiating part is already done and
you just have to send the message, in whatever format you want. Decoding the
request on server side is way easier too.

[1]
[https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Me...](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Message_format)

~~~
ams6110
I've been out of front-line web development for a while. Are Websockets
supported widely enough that a developer can reasonably assume they are
available and not have to worry about workarounds such as comet/long polling?

~~~
bombtrack
Yes, I would say they're an expected browser feature these days, with over 94%
global browser availability [0] (basically anything >= IE10).

The cool thing about Phoenix specifically is that they provide a JS client to
integrate with a Phoenix Channels [1] backend that will automatically fail-
over to long-polling if window.WebSocket is not available [2]. These Phoenix
Channels are set up to be transport agnostic, so you don't have to write any
special backend or client code to handle one way or the other, it "just works"
for the most part.

[0]
[https://caniuse.com/#search=Websockets](https://caniuse.com/#search=Websockets)

[1]
[https://hexdocs.pm/phoenix/channels.html](https://hexdocs.pm/phoenix/channels.html)

[2]
[https://github.com/phoenixframework/phoenix/blob/master/asse...](https://github.com/phoenixframework/phoenix/blob/master/assets/js/phoenix.js#L693)

------
bsaul
Non-representative anecdote : i've recently heard of two teams that started
using elixir, they both came from a ruby background, and in both case they
weren't using OTP.

They both justified using elixir because it was "closer to ruby" (???), and
they both seemed to live a pretty difficult time with that language. Which
makes me wonder : why would you ever want to use such a special language
(purely functional isn't for everyone) if you don't have any scalability
issue, and don't even use OTP ?

~~~
dudul
I've seen a few (as in 2 ;) ) examples of this myself. And this is what
"scares" me the most for elixir's future/potential. I think it's a great
language/tech but ruby enthusiasts rushing into it thinking that it's "just
like ruby but faster" may have a very bad influence on where the language
goes. They may bring a lot of ruby best practices that don't transpose at all
in elixir/erlang world.

~~~
bsaul
That was indeed my underlying point. I see elixir more as "erlang with easier
syntax", than "ruby with a more powerful runtime".

aka : you should get to understand erlang design decisions prior to jumping
into the tech. Which probably means having more experience than building the
typical low load website made with RoR.

~~~
pmontra
I'm in the fourth month of a Phoenix project, which shields me from almost all
of OTP, and I agree with your point. Maybe even "Erlang with a Ruby syntax"
would be too much. Similarities with Ruby don't go any further than "def" "do"
"end" and many library modules and functions deliberately named to mimic
corresponding modules and methods in Ruby. Everything else is different, both
syntax and semantic.

After 10 years of Rails, all things considered Elixir and Phoenix are pretty
good.

The best feature of the language so far is pattern matching. It can be used in
the arguments of function definitions. It makes all the "if"s go away. It
seems strange after 30 years of programming, but a program without
conditionals is easier to read.

Not having to use an external background job framework is great news too. Just
spawn your processes to send a mail, process something asynchronously and
store the result in the database. No Sidekiq (or Celery if you're into
Python.)

On the awful side, Ecto is too low level and it feels like a self inflicted
pain. I understand the reasons behind its design decisions, but the typical
web application or mobile backend don't need all of that. Actually the team
working with me on this project wrote a lot of code to build queries and wrap
them into another layer to return {:ok, result} / {:error, reason} tuples.
We're calling them from controllers more or less as Model.get_something(args).
I checked it now and we have zero occurrences of " from " in controller code.
All the queries are confined inside a model. If we were using Rails
ActiveRecord would have written all that code for us. Instead we had to use a
more verbose query syntax, write our functions and tests. This is a net loss
of productivity.

There are some modules that can be mounted on the top of Ecto to make it look
like ActiveRecord or other ORMs for other languages. I'm looking forward to
using Ecto.Rut in my next project. I hope I'll never have to use Ecto
directly.

[https://github.com/sheharyarn/ecto_rut](https://github.com/sheharyarn/ecto_rut)

We almost didn't use the supervisor layer of the language (somewhat like
systemd/init for who's not familiar with it.) I used it in smaller projects
and I feel like it's a little too complicated. Same for defining a GenServer,
more complex and verbose than exposing the same functionality with methods
from a object. I know we have to split the code between what runs in the
client process and what runs on the server process, but I feel there should be
a vanilla GenServer that handles the most straightfoward case inside the
behavior, which is no code in the client with the exception of passing the
arguments to the server. Maybe use the name, sigils or module attributes to
declare if there will be a synchronous return value.

Anyway, Elixir is still young and there is time to smooth the rough edges. So
far it's an acceptable contender and definitely better than the popular
scripting languages when concurrency is important.

~~~
karmajunkie
Don't take this the wrong way, but it sounds like you're approaching this from
the perspective that Rails' way is the "correct" way when in fact almost
everything about Rails' doctrine will cause you problems over the life of a
project.

> We're calling them from controllers more or less as
> Model.get_something(args). I checked it now and we have zero occurrences of
> " from " in controller code. All the queries are confined inside a model.

Yeah, that's almost a best practice, except you're regarding the "model" (the
terminology is 'schema' in Ecto) as primary instead of building it as a
separate context or application (depending on your preferred approach to
modularity within Elixir/Phoenix). For that matter, I fail to see the
advantages of `Model.get` over `Model |> Repo.get(id)`.

> If we were using Rails ActiveRecord would have written all that code for us.

There's not a substantial difference between the kind of AR usage that's on
the happy path and the equivalent query in Ecto.

> I know we have to split the code between what runs in the client process and
> what runs on the server process, but I feel there should be a vanilla
> GenServer that handles the most straightfoward case inside the behavior,
> which is no code in the client with the exception of passing the arguments
> to the server

You mean like an Agent[1]?

[1]
[https://hexdocs.pm/elixir/Agent.html](https://hexdocs.pm/elixir/Agent.html)

~~~
pmontra
First of all thanks for the detailed answer. It contains many interesting
points.

I don't think that AR is correct, I think that AR is more convenient than Ecto
in many cases we run into in web development. With AR I don't have to write my
own queries and encapsulate them into functions (but it's kind of what we do
with scopes) and it has a more compact syntax. Ecto is more flexible, but that
flexibility is not needed in most cases. Anyway when we must really be
flexible we write queries in SQL and manually unmarshal resultsets. ORMs/Data
Mappers are for simple and medium use cases.

I still didn't run into problems with the AR approach. Maybe it's because all
of my projects were medium or small sized. I found AR to be perfect for them
to the point that I want to disguise Ecto as AR using Ecto.Rut.

> you're regarding the "model" (the terminology is 'schema' in Ecto) as
> primary instead of building it as a separate context or application

This sounds interesting but I fail to understand what you mean. Would you mind
explaining or posting a link? Thanks.

> I fail to see the advantages of `Model.get` over `Model |> Repo.get(id)`

It's shorter but not by much. One reason is that Repo doesn't mean much to me,
so it could be hidden. What I care about is Model. But getting values out of
the db is not such a pain. Inserting and updating is, because they are more
verbose. I quote Ecto.Rut for the insert

    
    
        Post.insert(title: "Awesome Post", slug: "awesome-post", category_id: 3)
        # instead of:
        # changeset = Post.changeset(%Post{}, %{title: "Awesome Post", slug: "awesome-post", category_id: 3})
        # YourApp.Repo.insert(changeset)
    

If all of those extra characters are for extra flexibility (maybe for using
more repositories in future?) then it smells of premature optimization. I'll
happily do without it.

Regardless of this discussion, IMHO a thing that Ecto should fix is requiring
developers to write both the migration and the schema. It's either the AR way,
migration first and auto generated model, or the Python way, model first and
auto generated migration. We got some bugs because we didn't write the same
things inside the migration and the schema. Mistakes happens and the tools we
use should help us not the make them. Ecto is not DRY but it should.

Btw, the compactness argument applies more or less to all of Ruby vs Elixir
because of the object.oriented.notation.is.shorter than the Module1.functional
|> Module2.way |> Module3.of |> Module4.composing. Aliases help to some degree
but they add clutter to the top of the file. It's a little nuisance but
pattern matching more than evens it.

I have to look more into Agent (thanks) but it seems exactly the opposite of
what I want: write in the module only the code that should run in the server.
Probably what I'm looking for is a macro that writes a vanilla GenServer for
me hiding all the functions run on the client.

~~~
karmajunkie
I think I came off snarkier than I intended to above, thank you for reading
that gracefully.

> I still didn't run into problems with the AR approach. Maybe it's because
> all of my projects were medium or small sized. I found AR to be perfect for
> them to the point that I want to disguise Ecto as AR using Ecto.Rut.

Perhaps its a stylistic thing, but recent versions of Ecto feel quite natural
to me. I frequently use Ecto _without_ a database by using it for embedded
schemas (which I don't embed)—they're basically just structs that I can use
with changesets a little more cleanly.

As someone else noted, you don't have to use changesets for the happy path
interactions like simple inserts. It really just comes down to whether you can
live with the Repo as the first thing you type instead of the schema.

> This sounds interesting but I fail to understand what you mean. Would you
> mind explaining or posting a link? Thanks.

Rails teaches us to look at the model as the primary point of business logic.
In my view this is putting the cart before the horse. By putting the Model
(big M, not little m) we limit our thinking around abstractions to what we can
represent structurally in the database. A well-designed, thought-out
application model (little m, not big M) encompasses much more than database
tables. With AR, even if you try to create behaviors on fat models, ultimately
the only vocabulary you have to work with are those nouns the database allows
you.

Ecto does something subtle but important: it demotes your data schema to just
that—data. Behavior is modelled in the messages you pass between processes,
which is why you see so many people in the elixir community jumping into
architectural techniques like eventsourcing. While things like Ecto.Rut add
some conveniences, they also encourage promotion of the data to the central
artifact of the system. After building Rails applications from small to very
large over 10 years, I can say for my part I want to stay as far away from
that as possible. Its convenient until its not, and when you can start to feel
the pain of it, its very difficult to unwind its effects throughout your
system.

re: agents, maybe what you're looking for is ExActor:
[https://github.com/sasa1977/exactor](https://github.com/sasa1977/exactor)

Worth noting, especially if you're in the first 12-18 months of using Elixir:
OTP is incredible, but it takes time to wrap your head around all its pieces.
I usually recommend new users coming from an MVC framework just try to muddle
along using Phoenix as they would Rails until they start getting comfortable
with things like supervision trees and GenServers... and then the fun really
starts. All the rest of it, like Ecto's hands-off approach to the database,
really starts to make sense around that time.

~~~
pmontra
ExActor is more or less what I was looking for. It deserves its 491 stars (492
now.) A big thanks for that!

After so many years of software development I don't like unnecessary
complexity, that's why I'm keen to shave off features from Ecto and GenServer
(and a lot of other tools, not only in Elixir) and settle for a subset with an
easier API.

That said, you have a point when you write that AR's approach is "convenient
until its not, and when you can start to feel the pain of it, its very
difficult to unwind its effects". The project I'm working on is an MVP. Who
knows where my customer is going to be in 12 month. What I know for sure is
that using Ecto cost them some extra time to deliver because of all that
boilerplate we had to write. Would I add an extra layer between the db and the
logic in a Rails application if its requirements imply complicated logic and
interactions? Probably yes. We can put any kind of objects in the models (or
lib) directory of Rails, not only ones derived from AR.

~~~
karmajunkie
I think I'm still left with the impression that you may be making things a
little harder on yourself than they need to be, or perhaps better stated, than
they need to be _now_... Ecto's API has gotten streamlined over the last few
versions, such that in most common operations, I can't see much more than a
cursory cosmetic difference in the APIs, like starting the call chain from the
Repo instead of the Schema. What I will readily concede is more complicated is
the support for things that are widely considered to be antipatterns, like STI
and polymorphic associations.

> After so many years of software development I don't like unnecessary
> complexity, that's why I'm keen to shave off features from Ecto and
> GenServer (and a lot of other tools, not only in Elixir) and settle for a
> subset with an easier API.

This is what I'm missing—there's essential and accidental complexity, and when
I look at GenServer, I see an API that's been shaved down by decades of
practice in Erlang to its most essential complexity. Even ExActor is just a
set of macros for generating those essential parts—it basically just saves on
typing, not skipping functionality. Ecto hasn't had the years of legacy that
OTP has, so its API has fluxed a bit in the last few years, but its still
something I think is cut down to the bare minimum for healthy database
interaction.

This kind of discussion is best handled by email, i think, and mine is in my
profile—feel free to send a gist of something that illustrates what you're
talking about, and maybe I'll have a better understanding of what aspects of
the API are headaches. I'm really curious what I'm missing about your
perspective here.

------
brightball
If you're doing development with a web socket layer, it's entirely worth it to
try out Phoenix Channels before you commit to another stack. It makes your
life so much easier.

There are a lot of other great reasons to use Elixir though.

~~~
sergiotapia
Phoenix Channels feels like cheating because it's just so intuitive and you
get so much performance straight out of the box. It's obscene! Maybe I'm used
to Ruby perf that this blew me away so much.

On free tier heroku dyno and the $7 postgres hobby dev, performance was around
27,000 messages sent per second before POSTGRES started complaining. It wasn't
even the app itself that was complaining, it was postgres. This was sent via
websocket using phoenix channels and persisted to the database.

If you're a backend guy, add this tool to your repertoire - you'll become a
wizard and just have tighter code with less headaches.

------
thejosh
I really like absinthe with apollo/subscripions on the frontend for realtime
updates.

------
hn_hates_tor
OT, but what software did you use to make those gifs on Ubuntu?

~~~
blaze33
Hi, I used gtk-recordmydesktop (beware it will freeze your desktop under
Ubuntu 17.10 Gnome but works fine under Unity). It gives you an .ogv file and
then there's some editing to get a gif. I described the process at the end of:
[https://blog.openbloc.fr/javascript-es2015-starter-
kit/](https://blog.openbloc.fr/javascript-es2015-starter-kit/)

Mainly convert the .ogv to .mp4 with ffmpeg (don't have the command here),
create a png palette from the video so your gif colors are more accurate and
use ffmpeg to create the gif:

    
    
        ffmpeg -y -i video.mp4 -vf fps=10,scale=800:-1:flags=lanczos,palettegen palette.png
        ffmpeg -i video.mp4 -i palette.png -filter_complex 'fps=1,scale=800:-1:flags=lanczos,setpts=0.25*PTS[x];[x][1:v]paletteuse' todoApp.gif
    

This gives you a 800px wide gif, with 1FPS, and sped up 4 times (setpts=0.25)
:)

~~~
worble
Why make a gif from this rather than a webm (seeing as you obviously know what
you're doing with ffmpeg)?

~~~
dagw
What does webm support look like these days? Last I checked neither Safari nor
IE had webm support.

------
shadykiller
Elixir is pure pleasure. Coming from a ruby background, I din't have huge
difficulty in understanding the syntax, and unlike other functional languages
like Clojure, it's not an abrupt change from procedural paradigm. You can
still assign variables in functions etc.

~~~
mischov
Elixir should absolutely be an abrupt change from the procedural paradigm, and
much closer conceptually to Clojure than Ruby.

In Clojure you can still stick something in an atom or volatile and easily
update it in place... doing that is a lot more involved in Elixir.

------
malloryerik
How would you compare this with the same app done in Clojure? How would
development and result be different?

------
isuckatcoding
I think as the tooling /IDE support for Elixir improves, so will adoption.

------
Alexa_Anthony
I really like absinthe with apollo/subscripions on the frontend for realtime
updates.

------
Alexa_Anthony
Does it generate real-time performances and analytics?

------
ttrbls
Elixir Deez Nutz

