Hacker News new | comments | show | ask | jobs | submit login
Show HN: Condor – A library for building scalable TCP servers in Erlang (github.com)
65 points by s1n4 3 months ago | hide | past | web | favorite | 10 comments

I'm not sure that a library is really the right way to do this. In my experience, socket handling is a key part of the application, adding a library layer will make it harder to deal with the inevitable cases where the abstraction doesn't fit. An example of a simple, scalable TCP server (and a scalable client, so you can demonstrate the server), would be a better approach -- building a new server starting with the example would allow someone to make the changes they need when they need to. Avoiding unnecessary layers is a hallmark requirement of scalability, but this approach is adding one. One thing I can definitely see where the approach shown would be limiting is that often TCP servers would like to have a connection idle timeout, but if your peer is sending you a large application packet over a very slow connection, you wouldn't want to consider it idle because you hadn't received a complete application packet in some time -- having received partial packets should be enough to show that the connection continues to be alive (subject to concerns about slowloris).

Also, it's frankly hard to judge your claim of scalability, when you haven't provided any data. How many millions of connections can this handle (on which particular hardware), how many thousands of connections per second, how many gigabits of data transfer can you push, how many connections can you gracefully close per second, etc. What erlang settings did you have to tweak, what operating system limits did you have to tweak, etc to get there.

There’s no Netty type thing for Erlang? Netty is so nice to have on the JVM, I don’t know much about TCP and it lets me be dangerous, asynchronously.

The Erlang VM (BEAM) is non-blocking by design, you don't need anything like Netty on BEAM.

There's a standard, built-in way to handle events on a TCP socket? Like Netty or Node's `net` package.

I'm not familiar with Netty, but in Erlang you set `active` mode which means Erlang will send your process messages for all incoming events for a socket: http://erlang.org/doc/man/inet.html#setopts-2

You can further use `{active, N}` to limit the "events" to a certain number as to apply back pressure to the OS socket level and beyond.

How Condor differentiate itself from Ranch?

Ranch is merely a socket acceptor pool. Condor takes care of framing and buffering, also.

Doesn't native erlang handle that? With {packet, Len} and {active, once}?

Not the buffering.

You can set a user level buffer size of the socket driver, and a receive buffer size (see http://erlang.org/doc/man/inet.html).

Why is another level of buffering needed?

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact