

Stormforger – Cloud-based Load Testing as a Service - whereismypw
https://stormforger.com/

======
tisba
I have no idea, why this ended up here, but if you are interested in some
technical details what SF does: [http://blog.stormforger.com/2014/05/27/load-
testing-an-inter...](http://blog.stormforger.com/2014/05/27/load-testing-an-
interactive-tv-show-with-over-1-million-users/)

~~~
lucas_dohmen
Wow, this sounds really impressive, especially:

> Here are a few numbers: StormForger did a total of 1,213,583,187 requests in
> over 50 load test runs to the Quizduell system and a total of about 2.21 TB
> of data was moved. The error rate was at about 0.000000216% (1 error every
> 4,624,616 requests).

~~~
tisba
You are referring to the error rate? :) We actually did about 800M additional
requests two days after I've written the post.

------
oddurmagg
What other services like this are out there ?

I know of loader.io but I am specifically looking for ones which I can script
to simulate real users.

~~~
zentrus
Also [http://blitz.io](http://blitz.io)

------
syntern
Impressive. It's not their fault and they seem to be good and talented, but I
can't stop thinking about a possible marketing line: "DDoS as a Service".

~~~
opendais
Normally services like this make you add a DNS record and/or file to the
target to confirm you are the owner.

~~~
tisba
this is especially challenging actually. just because you own the domain name,
doesn't mean you own the infrastructure you are pointing your DNS records to!

So adding DNS records is pretty useless to provide proof that you own the
resources under test.

~~~
opendais
Yes _but_ at that point you are theoretically traceable and can be found. It
isn't like a botnet. Someone had to buy the domain.

~~~
tisba
Okay, that's true. But the question was about confirming the target's
ownership. And that is something you cannot ensure via DNS entries.

------
dyeje
Something I have been really interested in is being able to replay the actual
traffic our site is experiencing. So, we press record and everything that's
going to our server is recorded for playback at a later date. Does anyone know
any solutions like that?

~~~
tisba
There are a couple solutions that do that - none of them I find sufficient. If
you want to clone live traffic, that's probably okay, but recording traffic
and replaying it in a usable fashion is a non-trivial task. e.g. systems tend
to be dynamic so your recordings would need to be dynamic too in order to be
realistic.

------
mping
The api looks alot like tsung. I'm almost willing to bet that either they got
inspiration from it or are using it on the server. Anyway, tsung is great and
these services are great too.

~~~
felixgallo
if you look at the 'your data - your results' piece of the carousel, it has a
log which has 'pid' as a column and, e.g., '<7038,69,0>' as a value. So you're
probably quite right.

------
lfuller
I can't recommend Redline13
([https://www.redline13.com](https://www.redline13.com)) highly enough. They
act as a service that coordinates load testing using instances within your own
AWS account.

Really easy to use, and I can run tests that used to cost upwards of $5,000
for under $5. I don't know how they aren't better known amongst DevOps types.

------
evv
Load testing as a service- thats a brilliant idea!

But why tie yourself to Ruby? (My initial impression is "gross!", but I can't
say I'm much of a language purist as I spend most of my time in JS)

Would services like Stripe, Mailgun and Twilio have succeeded if they had
forced their customers to use a Ruby DSL?

~~~
hnbascht
I saw basti's talk on stormforger at last month's interactive cologne – and I
think he switched to JavaScript for the test definitions. Maybe the website is
a bit outdated. :)

~~~
larsvegassynpro
Indeed, the DSL is now in JavaScript.

------
bigdubs
How much better is this than a program like WRK on a couple laptops on
different IPs?

~~~
yoshida
WRK is a nice tool that can bring you quite far, for sure. But what about the
orchestration of complex sessions? As I understood that blog post mentioned in
another comment StormForger was actually able to play a game. A simple one,
but there is still a lot of complexity involved.

But even if you can build something like this with WRK (which should be
possible, due to the Lua support) you still need to setup your test
environment (reproducibly), collect all the logs from the machines involved,
monitor the test generators, create useful metrics from the test results and
make it possible to interact with them. And I'm pretty sure I forget a whole
bunch of stuff that needs to be done.

And I know, that I don't want to do all this stuff by myself. With a service
providing all this, I can concentrate on the real work: Interpreting numbers
and fixing problems ;-)

BTW: I'm pretty sure under the hood there is some load testing tool deployed.
Cannot imagine someone runs such a large scale test without such thing.

------
notastartup
what happens if I use cloudflare, can it mitigate such massive traffic?

~~~
iancarroll
Some load testing services ban Cloudflare testing.

