
Short Chat Server in Clojure - fogus
http://kirindave.tumblr.com/post/272596413/clojure-chat-server-1
======
physcab
This is an aside--fogus, you are on fire today. 5 posts in the top 30? Where
do you find all this good material. :)

~~~
blasdel
He subscribes to a great many of the same feeds that I do, but he usually
submits the links first (he's the initial submitter on ~5 links a day!). It'd
be interesting to graph story upvotes over time, distinguishing /\ ones from
double-submissions.

I didn't see this one in my reader, though the author is a member:
<http://news.ycombinator.com/user?id=KirinDave>

~~~
mahmud
I wish for the both of you to stop doing this, both for your own good and that
of HN.

~~~
jacquesm
I've voted you up because I agree with the spirit of that, if HN really is
nothing but an aggregator for a bunch of quality feeds we might as well shut
down the input queue and automate it, that would nicely take care of the spam.

The problem I've got with posting stuff from 'the same feeds' is that it tends
to drown out the non-mainstream (and non techcrunch ;)) stuff, which is what
really makes HN interesting.

Between all the spam posts and the 'feed pushers' a lot of really good content
never makes it to the home page and so is lost to a great many readers.

Personally I've switched to the 'new' page as the homepage of HN long ago, but
even there if you look the other way for one or two days it's hard to catch up
because of all the fluff.

~~~
blasdel
At least for fogus and I, 'the same feeds' are geniuses like Christian
Neukirchen, not problogger noise.

I too despair over cool stuff sliding off the 'new' page into oblivion -- the
only surefire way for an article to make it to the front page is for multiple
people to submit the same URL, which can get rather unlikely for the most
valuable posts. I usually try to wait until a reasonable PST hour to submit
odd things, just to maximize the chance that clueful users will have the
chance to give it those first few upvotes.

I do think some countermeasures should be added to news.arc to handle the
mainstream spam-stream. I think a good first step would be good to stop
counting link-karma towards user totals, but I don't have a clear idea of what
else would help.

------
dhotson
Nice one! I'll have to learn some Clojure.. :-P

------
kikibobo69
I think I prefer the Scala/LiftWeb one: <http://lift-example.appspot.com/chat>

~~~
kikibobo69
Sure, the machinery is different; I was responding to the terse elegance of
it, not its self-containedness. I suppose yes it is outside the scope of the
original submission, but (for me) still compelling.

~~~
KirinDave
Certainly. Lift is an awesome framework. But this isn't a web framework
competition. :)

Perhaps you could write a basic chat server in Scala to show us? Really, I
found this exercise quite fun. The programmer's equivalent of cotton candy.

------
wvenable
The node.js chat server that was posted here earlier looks a bit cleaner and
easier to follow in my eyes:

[http://dhotson.tumblr.com/post/271733389/a-simple-chat-
serve...](http://dhotson.tumblr.com/post/271733389/a-simple-chat-server-in-
node-js)

Any difference in functionality?

~~~
KirinDave
Yes, I read that. It's what inspired this project. My version also supports
different rooms. It doesn't support a global list of users, but it does list
users in room. The Clojure version has one thread per client (that's what
create-server on line 74), and asynchronous writes.

The data model is slightly different, too. Clojure has very powerful
concurrency-safe data structures which made my data structure slightly more
obvious. It's just a hash table of lists, and the lists contain tuples of
[Name, StringWriter].

If you know Javascript and you don't know Clojure, it's probably natural your
eyes follow that example more easily. I wouldn't say either version is
particularly more convoluted than the other (although it's obvious we both
made some decisions that favored brevity over readability).

~~~
megaman821
Isn't there also quite a difference in scalability? I would expect the Node.js
version to support thousands users where is this version is probably only
hundreds.

Mainly just a curiosity question, I realize not many people will ever have to
deal with thousands of concurrent people chatting.

~~~
KirinDave
I don't know. I'd love to see an analysis of that. Do you believe Node.js can
handle that kind of load?

I'm probably going to extend this exercise by switching from the default
contrib library networking and standard java.io, and instead use netty. I'm
told that can give me superior performance and scalability, and I don't think
it'd take much more code. But the rest of the code should be fine. I'm quite
certain, from profiling, that the core data structures can scale to thousands
(and even millions) of users. Clojure's refs and agents are extremely well-
implemented.

But, I'm not sure the node.js version really would scale to thousands of
users. Every time someone connected hits enter the entire user set is looped
over, synchronously. I suspect this latency will add up. The reason that code
is so simple and direct is that everyone is just executing slices in one
thread. In my version, the rooms have a thread pool (implicitly), so it really
comes down to how many user threads java can allow.

~~~
dhotson
The broadcasting loop shouldn't take too long... I'm pretty sure that the
socket.send() calls are non-blocking. I could be wrong, I haven't tested my
(toy) chat server that thoroughly. :-P

In general, the single threaded evented model tends to scale better than the
process/thread per connection model for large numbers of connections. Having
to context switch between 10,000 threads/processes typically means the OS
spends a lot of time doing context switching instead of IO.

I'm not sure how Clojure implements agents, it may already handle having
10,000 concurrent agents efficiently. I'd be curious to know.

Also, there's a great article about network programming, which you may have
read. It's called "The C10K Problem", which talks about scaling to 10,000
concurrent connections.

<http://www.kegel.com/c10k.html>

.. really good article, well worth a read if you haven't already.

~~~
KirinDave
> In general, the single threaded evented model tends to scale better than the
> process/thread per connection model for large numbers of connections. Having
> to context switch between 10,000 threads/processes typically means the OS
> spends a lot of time doing context switching instead of IO.

Right, the evented model works, and you get that for free with Node.js. I
dunno if that specific codebase would handle "thousands of users" though. Test
it, it'd make a cool blogpost. :)

> I'm not sure how Clojure implements agents, it may already handle having
> 10,000 concurrent agents efficiently. I'd be curious to know.

Well right now it's 1 thread per connection, which I will have to change. As
for agents, they dispatch on a threadpool with a number of threads
corresponding to the number of cors in the physical hardware. So that part of
the code should scale of fairly arbitrarily (although a specific room with
many, many people may experience some latency, and many such rooms might bog
down the thread pool).

------
azakus
The only small problem I would have with it is the inability to be in multiple
chatrooms at once. Very nice though.

~~~
KirinDave
The data model obviously supports that. See lines 61-62, you can be in two
rooms at once, no problem. In fact, for awhile that's exactly what happened
(hence the separation of leave and join).

The only reason I didn't was that the original node.js was usable right from
netcat or telnet without prefacing anything with /say or any such command. If
I had multiple rooms, a /say <targetroom> <message> would have been required
and I didn't think that was very fun to type.

------
kunqiana
does anyone know a chat server in scheme? It would be helpful for me.

