1. Testing isn’t always as easy as following a list of steps, and when it _is_ those get to be technical. Explaining those as text instructions could get harder than what most expect if they are to be useful.
2. This is a lesson learned from experience - unlimited plans rarely scale. Instead, they make those who would fit into an entry level (aka just a few tests) feel like they’re paying a premium and those who will fit a large plan not pay nearly as much as what they consume.
Not only that, but when the founders realize this they’ll have to change it and users will be upset, so they’ll grandfather them and we’re back to the beginning.
Side note: I love “breaking” new features personally and know it’s not a straight forward process. If all this aims to do is replace selenium I don’t see the need for it — selenium tests are very predictable where text-to-human isn’t. If it aims to go beyond, that can’t happen without real knowledge of the application.
1. I completely understand, being a software tester for over 10 years. We are going to build in functions in the software that will make it easier for users.
2. The unlimited plan is limited in the launch phase to certain number of companies already. We are going to shutting down sign ups after hitting that level to avoid disappointment as you have mentioned.
3. TestRise isn’t a replacement for Selenium. While automated tests have their place, using real users with real devices brings up a lot more edge cases.
Would be great to hear your thoughts in detail and swap notes about your experience with prior similar services. I can be reached at analyn (at) testrise (dot) com
A friend of mine hacked together something like this with mechanical turk. This seems like a much more polished version.
I hope you succeed! When I launch our product will definitely try your service out.
my thinking is that this a late stage April Fools day prank.
Companies who require faster turnaround times will be able to do so in the future as well. The Unlimited plan with a single parallel test works very well for early stage startups that can execute the tests at the end of their workday knowing that when they come back the next morning they will have their results.
The unlimited plan is not a good fit for people who are deploying multiple times a day and need much faster turnaround times.
The reason you’re running tests is to view the results and then quickly fix any bugs, re-running any failed tests to verify. Given the serial nature of testing and the turnaround time, you’re losing far more in productivity than $99/month. And if they get a sufficient number of signups at that price point, their ability to service tests is going to be severely impacted to the point where it becomes unusable.
They should really change this model and offer a limited offering for the $99/month. Keep the one-at-a-time model, but also limit the number of tests per day to something reasonable. Then introduce a $999/month plan which is unlimited. This is still way cheaper than even a part time QA tester, but as soon as you start firing sufficient load at the service you run into capacity limitations. Of course, at that point it makes sense to start offering dedicated resources for those who hit those capacity limitations, which results in a few additional offerings of $X,999 per month for various values of X. Assuming they can offer 24/7 coverage, screenshots, recordings, variables, API, etc. they’ll perhaps become a viable competitor for Rainforest QA [which I used at a previous job].
I think your idea of limiting the amount of tests is great and something we will definitely explore. When we were running the service for clients manually we did something similar. As for services such as 24/7 support along with neat features like variables and interacting with API's they are on the roadmap. We are bootstrapping this at the moment and looking for our first 10 - 20 customers to work with us as we refine the service. They get a very affordable QA service and we refine our process.
Biot if you have used rainforestqa before would be great if I could ask you a couple of questions over email if possible. I can be reached at analyn (at) testrise (dot) com. Your help would be greatly appreciated