Saw a video last night by Jeff Lawson of Twilio about a lean test he conducted on an email sending API. It's starts at the 9:30 mark:
He did a very thorough test using Google to drive traffic to his test website along with various A/B tests on pricing.
His conclusion was there wasn't a viable market. Tell that to SendGrid and their various competitors ;<). I've done the very same thing with several sites and was always left wondering was it the test or my idea?
My conclusion is that to do lean successfully you also need to get off the web and get out and talk with potential customers face to face. Sometimes if you listen carefully they will hand you an even better idea.
For example, I recently had an idea, so I did a bunch of interviews with potential users. That gave me enough information to form a quantitative hypothesis: if I offer users app X, then Y% will use it regularly over a two week period.
Whether that fails or succeeds, my next step will be to talk with users. If it succeeds, the interviews will tell me where to take the product next. If it fails, interviews will help me distinguish between a fatal failure (e.g., nobody will ever use this) or a fixable one (e.g., minor UI issues make friction too high for frequent repeat usage).
Can't repeat this enough. TALK to your (potential) customers. There's just too much to learn from them to ignore this step. Not knocking on the mostly valid assumptions one can make off a/b experiments, but to tweak your product to its best potential, you really should hear and see your customers react, interact, and critique your product.
"However, one thing I noticed after I moved back to India from the
Valley was that we often hesitate to talk about failures, and
what we learnt from them. People are much more interested in
learning from a success story. However, if you ask me, I’m more
interested in the 51 attempts of Rovio than just the one that made it
big. It’s the failure to learn that’s a failure."
I certainly think the Valley is more open to discussing failures and what worked and what didn't. In my experience, it's been hard to have honest, clear discussions in India with other entrepreneurs about challenges being faced and what people are doing to address them. There's just a lot of noise in the startup ecosystem. That said, all this is maturing, and I'm sure it'll get better soon.
not to mention this list too
The one thing that I think is missing from "what we could have done better" is this:
Be willing to CHANGE your ideas, big time.
You had people signing up. They wanted something. Instead of sticking with your first idea (video), why not try a bunch of stuff until something worked.
You have to be creative, flexible, and determined in order to make this type of business work. You had SUCCESS with many of the tests -- don't throw those successes away just because one test failed.
People didn't want video. Give them something else. Does it really matter what it is? Does it even have to be charades? Be willing to CHANGE on the spot in a BIG way.
Think about this:
1. People sign up and say "yes, I want to play charades".
2. They land on a page tht asks them to play charades. How the heck can you immediately engage them? By asking them to turn on video? Or...???
If you're giving up because people don't want to turn on video -- then you may as well not try. Think about this: If I want to have "online boxing", should I give up if people don't literally punch themselves in the face, in order to simulate real punches? Sure, it sounds silly, but it's not that different than asking people to turn on video. Online video can literally be harmful to people, especially when it involves strangers, and even when it involves friends and acquaintances. People don't want video charades. That doesn't mean they don't want charades.
Don't get stuck on the wrong thing!
An alternative improvement that I would feel comfortable -
1) Instead of email id, just ask "enter a nickname". More people would feel comfortable this way.
2) Once I'm in -
"Welcome to your private charade room. Invite your friends to join your private charade room by sending this url".
Provide a private url to that room that they can send to their friends. Let them send it by mail. And don't ask me to enter my friend's email id or link to my facebook / twitter accounts. Many have already fallen for this trick from Linkedin etc and paid heavily for giving them access to the friend's ids.
Now make it easy for me also to enter this room using the url. If my friends enter the room, we are ready to play.
Allow me to create multiple rooms. So I could play with my family, my friends or colleagues at different time. And all through the day I could watch which room has member and start playing ...
3) And now provide an option for them to join a public room... Here mention clearly that your video is now visible to the worlds and could even be recorded ....
To me, it seems like the best products are the ones that served the creators' wants and needs. Github was created because the founders wanted to share code more easily. Facebook was created as a better way to (eventually) get laid as a college student.
If you really love playing charades, and more importantly, love playing charades so much that you'll do it online, even with strangers, then the path to a viable MVP seems clear. As it currently reads, the OP's analysis seems sound, but without much of the passion that seems needed to drive a product.
I also think passion is something you can develop over time. I've worked on a number of things where I had no initial interest. But I do like making users happy, so I get excited about the domain via seeing what gets users excited.
Incidentally, I OH this on Facebook: We tend to underestimate long term impact of reason and short term impact of passion.
Less excited people simply won't bother building it.
It's that simple.
One thing I have to tell novice experimenters a lot: A failed experiment is not when you get a result different than the one you wanted. A failed experiment is one where you don't learn what you set out to learn. Which can also be fine as long as you learn how your next experiment can be better.
This was a successful experiment.
I did include a para further down in the article explaining this further. It's the failure to learn that is failure.
Sustainability implies pivoting into something people will pay for, you can build and convince them to use before cash runs out. If the team is highly motivated, they will not disband on a single failure but will find something else that people want.
Of a sampling set of people searching for Charades online, at least 25% will sign up
to check out the game.
Hope this helps. I'd love to hear how others are doing this.
I just went through a similar process and did a write-up of my own - https://medium.com/how-to-succeed/86e70e2c33c1