

Do C10K testing with gevent - phillytom
http://engineering.monetate.com/do-c10k-testing-with-gevent

======
metabrew
C1M is the new C10K.

C10K is not even close to being a real problem any more. I expect my phone can
handle 10K connections.

Making a million outbound connections, on the other hand, requires a little
more care. Mostly because you need 16+ IPs. I ended up writing a small tool in
C with libevent to make 1M http connections:
[http://www.metabrew.com/article/a-million-user-comet-
applica...](http://www.metabrew.com/article/a-million-user-comet-application-
with-mochiweb-part-3)

~~~
svlla
you're only going to need 16+ IPs if you're making those outbound connections
to the same destination IP & port... which seems unlikely.

~~~
nivertech
You can do TCP listen on 0.0.0.0:80 (any IP on port 80), but I never saw TCP
listen on XXX.XXX.XXX.XXX:* (specific IP with any port) ;)

------
nivertech
1\. I already solved C1M problem, so C10K and C100K looks like a non-problem
to me.

EDIT: metabrew has very good tutorial - which should be a starting point for
everybody new to this space!

2\. The problem is, that Amazon EC2 doesn't support multiple IP aliases per
instance. To generate 1M connections, I had to launch a cluster of 17
instances to generate 1M clients. On physical server with a lot of RAM, you
can just create 17 IP aliases.

3\. Tsung is a great tool - but hard to control and has it's own limits. So if
you need to do large scale loadtesting - better write your own tool.

~~~
eugenejen
I have a stupid question here. When you have a large memory machine, why can't
you serve 1M clients on 1 ip without IP aliases?

I asked this question because if one uses 17 instances to handle 1M
connections, for each instance is around 50k. I am more interested in knowing
the possibility and performance for one machine to handle 1M connections.

~~~
nivertech
the real problem is not to generate 1M outgoing connections, but to handle 1M
incoming connections on a single node (C1M), while in addition also doing some
meaningful processing.

It's a solved problem for me. That is why I don't understand why many high
profile realtime web startups (like Convore), still using inferior solutions,
like Python eventlib or gevent, which at most let them handle hundreds-to-
thousands of clients.

EDIT: metabrew has very good tutorial - which should be a starting point for
everybody new to this space!

~~~
eugenejen
Are you referring to [http://www.metabrew.com/article/a-million-user-comet-
applica...](http://www.metabrew.com/article/a-million-user-comet-application-
with-mochiweb-part-1)

[http://www.metabrew.com/article/a-million-user-comet-
applica...](http://www.metabrew.com/article/a-million-user-comet-application-
with-mochiweb-part-2)

[http://www.metabrew.com/article/a-million-user-comet-
applica...](http://www.metabrew.com/article/a-million-user-comet-application-
with-mochiweb-part-3)

By Richard Jones of the Last.fm?

Thank you for your replying. The original C10k from Dan Kegel was a guideline
for me when I worked on project during 2003 - 2006. I am wondering what is the
latest progress in this area. I am glad this problem is solved to you. I will
check out niver tech in future when the problem comes up.

------
lusis
I actually was pretty fond of this:

<https://github.com/newsapps/beeswithmachineguns>

I don't remember how I came across it but it's the same concept: fire up a
swarm of EC2 instances to pound the shit out of your site.

------
justincormack
He doesn't seem to have tried Tsung which is a good solution for this scale of
testing.

