

A performance test of SQS - sorenso
http://litt.no/tech%20stuff/amazon-sqs-scaling

======
jdub
There's some confusion in this post about the testing methodology vs. the
implementation of SQS itself.

SQS itself may well be implemented on EC2 instances with auto-scaling (though
I doubt it), but that's irrelevant to the test harness, which needs to scale
out the number of nodes pounding on SQS.

The important bit is that SQS doesn't exhibit any of the scaling bumpiness of
ASGs. It's a region-wide service, which you'd be very lucky to break on your
own.

~~~
sorenso
Thanks for commenting!

I'll be happy to answer any question about my implementation and testing.

In this test I'm using a standard SQS service i region EU-west. My integration
with SQS is done from a simple application written in Node.js. My application
is put inside an autoscaling group which scales up when the load on the
servers is above my threshold. At the end of the last test I was pushing
messages from 18 different servers at the same time. The request rate was
above 6K/sec but the insert rate into the SQS queue only managed to peak aroud
2.3K message insert/sec.

I agree that this is strange. But I've also experienced strange stuff with
Amazon ELB: [http://litt.no/tech%20stuff/amazon-elb-latency-
problems](http://litt.no/tech%20stuff/amazon-elb-latency-problems)

Best regards, -sorenso-

------
euphemize
In our experience of ELB/SQS, AWS has done a good job on these services. A few
things we noted:

\- ELB takes about ~15 minutes to scale out completely when you hit traffic
spikes.

\- SQS is simple to use, rather cost-effective and quite reliable. It will add
a bit of latency, but as the author notes, this can't replace Kinesis.

\- The redrive policy on SQS is really nice to catch corner cases.

~~~
sorenso
Thanks for commenting!

What is your traffic rate in request pr second? I would be interesting to
know.

SQS is a simple and easy to use service. I really like it, but as you read in
my post. I'm a bit concerned about the scaling. We are expecting to reach a
peak traffic of 3K req/sec next year.

Best regards, -sorenso-

------
mbell
> It seems to have a problem with handling more than 2.3K message insert/sec.

We've written to SQS at rates several times this using only 3 EC2 nodes
without issue.

The only issue we have seen is occasional and sporadic latency issues reading
from SQS which causes the queues to backup a bit but it generally clears
within 30 minutes or so.

~~~
sorenso
Hi mbell,

Thanks for commenting!

What kind of application are you using with the SQS (language, purpose,
usage)? It would be interesting to know for me :)

This test was only running for 50 minutes, maybe this could be the issue?

Best regards, -sorenso-

~~~
mbell
We use SQS as a buffer at the front of an analytics system. The enqueuing app
is written in Java using Dropwizard and Apache Camel. Average message size is
probably ~1KB.

------
noso
Really good write and useful for a project I am working on!

~~~
sorenso
Thank you for your comment!

Please do a write up on your experience. I'll be happy to read about it.

Best regards, -sorenso-

