

Zeromq: Fast Messaging - nickb
http://www.zeromq.org/

======
PieterH
So the point about ZeroMQ's performance is that it removes almost all the
messaging overhead, leaving only the network stack and network protocol. The
main cost in conventional messaging is what sits above the network stack, in
the messaging products and in the applications.

ZeroMQ gets rid of almost all this cost, so lets applications approach full
network speed, as zinxq said, 10Gb on a 10Gb network.

There is a very big difference between a test program at each end of a
network, and an application framework like ZeroMQ, which includes routing and
queuing of messages based on the AMQP semantics of exchange and queue. The
real beauty of ZeroMQ is the way it lets developers build fast multithreaded
applications without ever worrying about how to scale - everything works as
asynchronous messages, between processes on one machine, or processes
distributed across wide networks.

It is possible to get about 2M kernel traversals per second, in one direction,
per core. So with a perfect scalable messaging product that does not bypass
the kernel and TCP/IP stack, you can hope for up to 1M messages per second per
core. Depending on the size of the message you can thus saturate any network.

Apart from ZeroMQ, no existing messaging actually manages this. The very best
closed-source products in this space manage 1M messages per second, which is
already a big number.

To get to the stated goals, ZeroMQ will need to bypass TCP, bypass parts of
the kernel, and probably move parts of its routing into silicon.

------
zinxq
The note that they get 2.5 million messages per second (and they mention
512byte messages). I don't doubt this is true but so what? All they are
quoting (from my quick math) is the speed of a 10G network.

I can write a small program to stream 512byte messages from one machine to
another on a 10G network and get 2.5 million messages across too. They're also
assuming the network connection is setup and that the messages are fully
formatted and ready to send. They also dont have any messaging acknowledgment
or anything.

Please check my math: (I was lax with 1000 or 1024, but results are similar)

k = 1024

meg = 1024 x 1024 = 1048576

Gigbit network = 1048576 x 1000= 1048576000

10Gigibit network = 1048576000 x 10 = 10485760000

Convert from bits to bytes:

10485760000 / 8 = 1310720000 bytes per second

i.e., 1310720000 is the number of bytes that a 10G network can push per
second. Note, thats raw - TCP overhead and such will eat some measurable
portion.

how many 512 byte messages fit in that?

1310720000/512 = 2,560,000 (just like they said)

I'm wondering if they actually tested this or just did this math. Because
again, this has nothing to do with their product, its just the throughput of a
10G network - and again, these numbers don't factor the 40byte TCP headers per
packet.

So.. if we made a webserver out of their message infrastructure (even lets
say, just a static webserver that only serves 512 byte pages ) - we can get
2.5million pages served per second?

~~~
socksandsandals
You should have probably read more closely before writin this diatribe: they
specifically note in several places that they do not use TCP and can also max
out on 4x Infiniband as well as 10GigE. They use UDP as their transport for
ZeroMQ messages.

------
stcredzero
The way it's written, it's not "Zero M Q" it's actually "Null M Q." On early
personal computers like the Apple II, the null-set symbol was used in the
display font as zero, so maybe that's a reference to this.

------
jvaucanson
> I can write a small program to stream 512byte messages from one machine to
> another on a 10G network and get 2.5 million messages across too.

Hey, common, do you really believe you can pass the networking stack up and
down 2.5 million times in a second?

~~~
napalm
you do not have to go up and down 2.5M times/sec - unles you are batching
messages. but have to take care about latency big care. btw on zeromq website
is nice chart latency vs. throughput - worth to have a look.

