
How we fine-tuned HAProxy to achieve 2M concurrent SSL connections (2017) - piyushgupta27
https://medium.freecodecamp.org/how-we-fine-tuned-haproxy-to-achieve-2-000-000-concurrent-ssl-connections-d017e61a4d27
======
billconan
I have a stupid question, there are at most 65536 ports and some are not
usable. There is also a file descriptor limitation. How to have that many
concurrent connections?

[https://unix.stackexchange.com/questions/84227/limits-on-
the...](https://unix.stackexchange.com/questions/84227/limits-on-the-number-
of-file-descriptors)

~~~
chmod775
There are at most 65536 ports _per remote IP_.

TCP connections are basically identified with an _(ip, port)_ tuple.

Also you can set the file descriptor limits to whatever you want.

~~~
zokier
Aren't TCP connections more accurately identified by (srcip, srcport, dstip,
dstport) tuple? This is somewhat significant as you can easily have tons of
IPs in a single box.

~~~
guiambros
Yes, but after the initial syn/ack, the daemon will allocate an _outgoing_
port number for the connection. So if you have a single IP address and a burst
with hundred of thousands of requests, you will run into problems..

------
muststopmyths
One quibble I would make with the article is that you are not actually limited
to ~65k connections from a client. It is only ~65k per IP address on the
client (given that they're all talking to a single remote port)

You can add NICs or virtual IPs and bind your client instances to specific IP
addresses instead of INADDR_ANY.

~~~
ldng
It is addressed in the previous article of the serie.

------
andyidsinga
hot damn!

I really appreciate the walk through of the apache bench (ab) results and
learning process even though it didn't get them to their objective - I've been
thinking about using ab myself, and these are great things to know.

I took down a few notes while reading the article:

\- mentions use of apache bench ( ab ) command for load testing

\- mentions use of ganglia tool

\- mentions configuring HAProxy for multi-core using nbproc setting

\- mentions the 'parallel' tool for running commands in parallel

\- simulate long run requests by having the server delay a little vs client
(work around for ab deficiencies )

\- have the server also send different response lengths back simulating
varying load

\- pdsh tool to remote parallel shell (ssh) sessions

\- vegeta tool - that got them to their scalability / tipping point objective

\- nodejs (used for their backends) had a default request timeout of 2 mins

\- used dmesg to learn that haproxy was running out of mem (at around 1.2mm
conns)

\- pdsh to run vegeta tool on multiple machines (acting as clients) - script
included in article

\- mentions haproxy maxconn setting - and verification by checking proc fs
limits for the haproxy pid

