Hacker News new | comments | show | ask | jobs | submit login
Show HN: I built testelf - app testing service for your iOS app (testelf.com)
77 points by ssong on Jan 23, 2013 | hide | past | web | favorite | 28 comments

OP here. I quit my full-time job almost a year ago to travel and work on startup ideas, and this is the fruit of my labor in the past six months that I hope you'll find useful. I wanted to make it easy for startups and indie devs to test their iOS applications. How does testelf work? You upload a build of your app without providing any additional instructions, and we'll test your app from a user's perspective, logging what we did or saw. The goal is to charge a flat price per test, provide good coverage and actionable feedback, and have a 36-hour turnaround time.

Over the past few months, I recruited and trained a small team of testers, developed internal testing tools, and worked with a handful of startups who are piloting our service. The service is now open for beta -- I would love to hear feedback from hackers and startups on HN who can give it a shot.

"You upload a build of your app without providing any additional instructions, and we'll test your app from a user's perspective..."

It seems like a user should have at least some idea of what the app is. For example, I'll read the first couple sentences of an app's description and look at some screenshots/video before I decide to download it. This might be something you want to consider instead of having absolutely no idea what the app is before running it, since I assume most users have at least some context when opening an app.

It's usually a good idea when user testing to tell testers as little as possible about the product. It's true that real-world users have more context than that, but it can be worth it to get more unfiltered feedback— the first few moments you watch a user flounder to even understand what they're looking at can be the most valuable.

Good point! There is an option to attach additional information (description, notes, link to website/app store) when uploading an individual build. We do ask for a little bit of information about the app in general when the first build is uploaded so that we have some context.

This is a pretty cool idea, and something I would have considered trying to get my boss to pay for when I was a developer. I didn't see on your website - do you test on a wide variety of hardware platforms, or just iPhone 5 and iPad? Different versions of iOS?

Also, do you have any terms, in particular, regarding confidentiality?

Thanks for asking! We do test on common devices (retina/non-retina iPad, iOS 5/6), and we're going to carefully consider which devices to cover as we grow. I am also very aware of the importance of confidentiality, such as keeping test results private -- I think the service wouldn't make sense without it. Our exact terms will be available before we publicly launch.

Brilliant idea for a business. I would love if you guys would do simple games some day. Hoping to give your service a try soon.

Congrats, looks really nice! (already forwarded to our testing team)

If you add a 'proxy layer' to your solution, that would be awesome. To explain the scenario, it is really very valuable for us app startups to see how it behaves in China, in India, in Europe, in US etc. You may keep your current system intact, and 'if' the customer desires so, you can route your connection over those proxies(from real ISS'es) from all over the world and test it 'as if...' This is really important and not so hard to implement... But sometimes, you can catch bugs and usability horrors with slow or dropped connection or some weird proxy mambo jambo played by ISS providers.

I think this is a great idea too! It would be an easy upsell to international developers.

Looks excellently useful, good job. I must say the graph on the test page (http://beta.testelf.com/builds/share/0b78d05c4f5d11e2ab5a220...) is particularly wonderful, great use of D3.

Also, would be very interested to know if you have plans as to how (or if) you will overcome the issues you would have if you wanted to scale such a bespoke service?

Any reason this is limited to iOS? It seems like it could extend to Android/Windows/etc. extremely easily.

Thanks for the comment! Android is probably the next platform for us if all goes well. I started off with iOS because I'm more familiar with it and it has less device variations. Extending the offering won't be trivial though. Before we start support for a new platform, we'd need to build the testing tools, develop new training materials, and acquire new hardware.

Why is "No provisioning necessary"?

This is a good question.

You wrote that you don't test games...

Well, I hope we can send our more non-gameish apps :)

This looks great, and I think it's a really good price point. $200 is quite reasonable, IMO, for the kind of detail you guys are getting into.

Definitely going to consider this for future testing :)

Looks great, i run a mobile app agency and we are forever having issues due to developers not testing adequately, having some external company test would be really useful!

This is a great idea sir. I signed up and hope to hear from you guys. I can use more eyeballs looking over our work. I really like not having to worry about provisions.

How do you handle UUID signing? Do you tell me the UUID's I need to add? Just upload seems to not work with iOS apps, right?

You can sign ipa files with your developer (private) key. There's an app called InstaSign that I use for "testing" apps before purchasing them. You download a cracked version, sign it with your own key (so iPad thinks YOU have created this app, like when you do in Xcode) and see if it's worthwhile.

It's saved me a lot of money that otherwise would've wasted on scammy, buggy, "not-working-as-advertised" apps. If you're a developer (i.e., can sign apps and load them unto your device), give InstaSign a try.

They're probably using a similar method (as they seemingly test on iPad/iPhone, retina/non-retina, iOS5/6 devices and it might be a bit more work for you to add at least 6 new UDIDs).

I was wondering how you did that. How do we know you're not simply cracking these apps to resell? It sounds a bit fishy is all.

Just to be clear, I'm not with testelf!

Just wanted to point that out in case you get the wrong impression of them not replying to your (fair) accusation :)

Awesome! This looks super useful, I'll definitely give it a shot with my own apps.

Pretty cool, I have a rough time getting my lazy friends to test my iphone apps :/

Good. I'm tired of monkeys. We need something more intelligent.

This is very cool. Simple but useful. Great work!

This looks fantastic. Great work.

Congrats! Very useful service.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact