
Show HN: Condor – A library for building scalable TCP servers in Erlang - s1n4
https://github.com/sinasamavati/condor
======
toast0
I'm not sure that a library is really the right way to do this. In my
experience, socket handling is a key part of the application, adding a library
layer will make it harder to deal with the inevitable cases where the
abstraction doesn't fit. An example of a simple, scalable TCP server (and a
scalable client, so you can demonstrate the server), would be a better
approach -- building a new server starting with the example would allow
someone to make the changes they need when they need to. Avoiding unnecessary
layers is a hallmark requirement of scalability, but this approach is adding
one. One thing I can definitely see where the approach shown would be limiting
is that often TCP servers would like to have a connection idle timeout, but if
your peer is sending you a large application packet over a very slow
connection, you wouldn't want to consider it idle because you hadn't received
a complete application packet in some time -- having received partial packets
should be enough to show that the connection continues to be alive (subject to
concerns about slowloris).

Also, it's frankly hard to judge your claim of scalability, when you haven't
provided any data. How many millions of connections can this handle (on which
particular hardware), how many thousands of connections per second, how many
gigabits of data transfer can you push, how many connections can you
gracefully close per second, etc. What erlang settings did you have to tweak,
what operating system limits did you have to tweak, etc to get there.

------
linkmotif
There’s no Netty type thing for Erlang? Netty is so nice to have on the JVM, I
don’t know much about TCP and it lets me be dangerous, asynchronously.

~~~
e_proxus
The Erlang VM (BEAM) is non-blocking by design, you don't need anything like
Netty on BEAM.

~~~
linkmotif
There's a standard, built-in way to handle events on a TCP socket? Like Netty
or Node's `net` package.

~~~
e_proxus
I'm not familiar with Netty, but in Erlang you set `active` mode which means
Erlang will send your process messages for all incoming events for a socket:
[http://erlang.org/doc/man/inet.html#setopts-2](http://erlang.org/doc/man/inet.html#setopts-2)

You can further use `{active, N}` to limit the "events" to a certain number as
to apply back pressure to the OS socket level and beyond.

------
knv
How Condor differentiate itself from Ranch?

~~~
s1n4
Ranch is merely a socket acceptor pool. Condor takes care of framing and
buffering, also.

~~~
vmsp
Doesn't native erlang handle that? With {packet, Len} and {active, once}?

~~~
s1n4
Not the buffering.

~~~
e_proxus
You can set a user level buffer size of the socket driver, and a receive
buffer size (see
[http://erlang.org/doc/man/inet.html](http://erlang.org/doc/man/inet.html)).

Why is another level of buffering needed?

