Surveying preference isn't a valid way of determining product viability, or even of comparing how well two products well sell.
This is not to say that you shouldn't build it. Building stuff for the sake of building stuff is a good thing.
This is also not to say that you can't make a business around product validation. There are multiple successful businesses already in the area of product validation.
I just wouldn't recommend the "Hot or Not" approach, because it's bad and generates bad data. Think who else is doing this and get an idea for what approaches there are to the problem - for example, Kickstarter.
> There are multiple successful businesses already in the area of product validation.
Any chance you could point me in the direction of some of those? I've got some ideas that I would like to validate and I've been unsure of how best to go about it.
I think the idea has potential. Maybe you should rethink the way feedback is provided. If I understand the idea correctly, it gives the 'customers' a choice between two products. When doing customer research you should never ask suggestive questions, which is what you do when you ask "Which one of these do you like?". Maybe they don't like it at all, but they still need to make a choice. This leads you to believe that one might be better than the other, while both ideas/products could be bad.
I think that you are mostly correct. In that, you do not want to lead your customers on too much and with simply testing one thing you will not be sure if the idea as a whole is bad. But there is a time and a place to ask a customer to decide between two choices. Usually this is done using discrete choice models and is pretty useful for ranking feature sets and even can help understand pricing.
In a SaaS application this could be extremely useful in seeing how much perceived utility people get from X GB more space or Y more users...
For the OP it might be cool to extend the idea to having multiple binary choices and performing the necessary math to understand the utility for features (multi-linear regression).
I like the idea... the only problem I imagine, is that there are only so many websites started each year. Seems like this is something a customer would use once.. then not use again for several years.. and by then, they may have forgotten you exist.
For example.. I've started quite a few websites... but that's only 5 or 6 in 10 years.
Also, small websites, people who are new to running commercial websites, etc... tend not to do ANY testing.
I'm about to launch my 4th business, and I should add to this: You can always think of a million reasons NOT to do something. The only questions you should be asking yourself at this point is: 1) Can you implement it and 2) Is it possible for to make money from it (ie: what price are you thinking? break-even point? how many customers do you think you can get? etc)
On the second one, note I said "possible". It doesn't mean you definitely will make money from it...
This looks interesting. I like the idea of a Product Idea Validation Tool, but I feel like there are enough sites that let me upload images for users to choose between them.
Perhaps something similar to what the http://survey.io guys do but with survey questions more for an earlier stage might be even more useful.
This is not to say that you shouldn't build it. Building stuff for the sake of building stuff is a good thing.
This is also not to say that you can't make a business around product validation. There are multiple successful businesses already in the area of product validation.
I just wouldn't recommend the "Hot or Not" approach, because it's bad and generates bad data. Think who else is doing this and get an idea for what approaches there are to the problem - for example, Kickstarter.