
AWS Device Farm - mauerbac
https://aws.amazon.com/device-farm/
======
stickydink
> unlimited testing for a flat monthly fee of $250 per device

Renting remote-controlled Android devices for $250 a month, is this even
remotely worth it? There aren't many devices that wouldn't pay for themselves
by the end of the 2nd month...

~~~
hexedpackets
You don't pay for a specific device. You pay for unlimited runs using 1
device, but that device could change each run. So its sort of akin to leasing
without an option to buy.

------
jastanton
Correct me if I'm wrong but does this feel pretty expensive?

> Pricing is based on device minutes, which are determined by the number of
> devices you use and the duration of your tests. AWS Device Farm comes with a
> free tier of 250 device minutes. After that you are charged $0.17 per device
> minute. As your testing needs grow, you can opt for our unmetered testing
> plan, which allows unlimited testing for a flat monthly fee of $250 per
> device.

~~~
aalbertson
I used to work for a mobile services company. We used technology similar to
this. It is NOT cheap, nor easy to build. Some places give you a web interface
that connects remotely to physical devices on the other side and some nifty
robotics to control the phones/push buttons/etc... This is down right
affordable!

------
Artemis2
Amazon is coming for Google's [Cloud Test
Lab]([https://developers.google.com/cloud-test-
lab/](https://developers.google.com/cloud-test-lab/)) and
[Nativetap]([https://beta.nativetap.io/](https://beta.nativetap.io/)).

~~~
ColinDabritz
On a brief look, it appears that all of these choices (Including Amazon's new
one) only support Android devices and not iOS?

I believe Xamarin Test Cloud includes iOS, although I'm not certain if it
integrates outside of the Xamarin framework. [http://xamarin.com/test-
cloud](http://xamarin.com/test-cloud)

Seems like a gap they should all be trying to fill.

~~~
rasmuskl
Xamarin Test Cloud is not limited to Xamarin apps for either platform.

Disclaimer: I work at Xamarin.

------
yla92
Pretty interesting. Now, we have more choices. The other day, I've found this
project on Github called OpenSTF[1] which allows you to set up your own
devices labs. It's pretty interesting and I've got to try yet. Generally, I'd
prefer to set up my own device labs rather than testing with 3rd party
services. It depends on your target market. For us, the apps we made are
target for the local market (people and devices). And the phones sold in our
market are different with the phone sold in States and I don't expect to have
some China devices to be available on 3rd party services.

[1] : [https://openstf.github.io](https://openstf.github.io)

~~~
shimo5037
Hey that's ours :) Right now it's heavily focused on remote control rather
than test automation, so it would not be a direct replacement yet. We just
open sourced last week, though, so perhaps we can start looking into that in
the near future after the initial batch of bugs is fixed.

------
martin_tipgain
Check us out at [https://www.testmunk.com/](https://www.testmunk.com/) we do
have iOS and Android support :) Don't hesitate to reach out if you have any
questions.

~~~
Someone1234
That Flipboard animation/video (showing the report I guess) is super
distracting/annoying. I literally deleted it with developer bar just so I
could read your copy in peace.

Also calling a $800 package with 25 hours and 5x accounts the "starter" level
seems a little odd. It is your lowest paid tier, but there's nothing starter-y
about it. It is your normal tier. The free tier is the starter tier.

I have no idea what a "project" is. Seems like an arbitrary limitation. What
do you care what code I am running with my hours? It is none of your business.
Seems like you created the concept just so you could "sell us more of it" with
the pro tier.

Also pro coming with only 5 users is odd. If I am paying for 100 hours/month
of usage, that just seems like an annoyance. You'll force businesses into only
giving out access to a handful of people rather than every single developer
(and if so few people use it, the company might decide the pro tier doesn't
make sense and downgrade to "starter").

Why is it so hard to find a list of devices you currently support? I actually
googled it and still cannot figure out what devices your service offers. That
seems like absolutely vital information for anyone even considering you.

Overall I like the service, but I like Amazon's model more. Just give me a
cost/minute and get out of my way.

PS - You absolutely CAN justify charging more than Amazon. iOS support alone
is a massive value add. I just like their pricing model, I am not proposing
you charge 17c/minute/device.

~~~
cscharenberg
Agreed on the animations. Why must everything be moving and sliding in? Just
put elements like pricing section where they should be from the beginning.

~~~
DJEchoes
Totally. I'm hungover, and all the animations made me nauseous.

------
RyJones
The most crucial bit for me, which I don't see on the one-pager, is the
ability to test devices in-market on carrier networks. Testing phones for
Korean or Japanese carriers (or British, or Brazilian) on simulations or using
GSM roaming is not good enough.

------
Wonnk13
Didn't Google introduce something similar to this at IO a few weeks ago?

~~~
yla92
Yes, Google announced Cloud Test Lab[1] in this year I/O

[1] : [https://developers.google.com/cloud-test-
lab/](https://developers.google.com/cloud-test-lab/)

------
saurik
I am always really curious to know how these kinds of services deal with
security issues: they don't get to control the security of the devices they
are running, and often will only even sort of have access to reflashing them;
how do they deal with someone testing an app with a kernel exploit, installing
a persistent backdoor, and then watching what everyone else later using that
device is testing?

------
nickpsecurity
I had an idea, more a need, for something like this many years ago. It came
back to my mind when OpenBSD needed funding partly due to all the different
pieces of hardware they test on. Wouldn't it be nice for portability-focused
projects to be able to pool together resources on hardware to cut down on it?

Anyway, awesome to see AWS doing it in practice. As usual, it will be _more_
interesting to see what happens when competition turns up. Cloud space has
more innovation and cut-throat competition than many IT sectors. Can't wait to
see what the competition costs. ;)

------
jakozaur
I wonder if they add IOS at some point. Maybe it's the Amazon way of doing.
Start with MVP and iterate.

------
varelse
So basically these guys?

[https://en.wikipedia.org/wiki/Keynote_DeviceAnywhere](https://en.wikipedia.org/wiki/Keynote_DeviceAnywhere)

------
ex3ndr
Why Amazon and not solutions from other mobile-focused platforms?

~~~
mwcampbell
Care to list any of those alternatives? I imagine a lot of companies will use
Amazon's device farm simply because Amazon is so well-known.

~~~
j_s
The primary reason to use an alternative is iOS support.

[https://appthwack.com/](https://appthwack.com/)

[http://keynote.com/solutions/testing/mobile-
testing](http://keynote.com/solutions/testing/mobile-testing)

[http://mobileportland.com/device-lab](http://mobileportland.com/device-lab)

[http://perfectomobile.com/](http://perfectomobile.com/)

[http://testdroid.com/](http://testdroid.com/)

[http://xamarin.com/test-cloud](http://xamarin.com/test-cloud)

etc.

------
mwcampbell
Curious about what the built-in, no-scripting-required test suite can do. I
wonder if this is what the app reviewers for the Amazon Appstore have been
using.

------
fudged71
I was hoping this would be a Raspberry Pi farm!

------
fbaptista
ohhh :) take a look at another monkey at www.monkop.com

