

Ask HN: Please review my site - PickFu.com - justinchen
http://pickfu.com

======
jusob
Looks great. I think the price is right. The website is very clear. Maybe the
examples should be more business oriented. You should also include the time
frame to get the 50 answers: 1 hour, 1 day, 1 week?

Can we choose, or at least influence the demography (only women, only people
below 35, etc.)?

~~~
johnli
another one of the creators here, thanks for the warm comments.

a follow-up question - how much more compelling would this be if you could
choose/influence the demographics and # of responses (potentially at a higher
cost)?

~~~
encoderer
I would personally find it more compelling, but I'd suggest that it shouldn't
be a premium feature.

When it comes to consume research, demographics are everything. Statistical
significance of sub-samples can plummet quickly.

If I'm marketing a product specifically to, say, 30-44yo men, it would be
valuable to restrict the sample exclusively to those men and I think i'd have
a better chance of seeing statistical meaning in the sub-samples I'm looking
for, eg, income, marital status, etc.

It seems to me now that I'd have to post it "at large" and the 30-44yo demo
I'm looking for would itself be a sub-sample.

I am not a marketing professional, but as a small business owner its one of my
many hats. I have used both Maritz and, more recently, Pinecone, and I'm
probably wrong but I don't remember even being presented with an option to
open the surveys to everybody.

And what I'm saying is.. I wouldn't even consider using your product without
that ability. You have a fantastic 1.0, though, so I'm not knocking the
product, just offering my $0.02.

And some demographics of your own: We've got a dozen of us on the payroll with
2009 sales looking to be a bit over $2MM.

And no joke, if I were you, I'd eat your dogfood and put up a survey to
determine the importance of this feature!

EDIT: __

I do want to add that I get that your site is supposed to be intro-level
market research. But if you're going to limit to 50 replies, demographic
targeting would be very important to me, for anything other than a "should we
get thai or pizza for lunch" questions.

If you do let me collect 500-1000 replies, I'd probably have more
statistically meaningful sub-samples, so as long as you give me a nice drill-
down interface I'd find it useful, even if i couldn't target my question.

~~~
SwellJoe
To take the other side...there is currently a _beautiful_ simplicity about
this idea and execution.

I pretty much just fall asleep when I start trying to deal with A/B testing,
market research, ad effectiveness, etc. This is so simple that all I have to
ask myself is, "Do I have a question I'd like to ask a bunch of people?" and
"Do I have five dollars?" That's a really good place to be, because everybody
in business has questions, and everybody has five dollars.

I will absolutely use this site. I do question _where_ the people are coming
from, which I guess is a demographic question...but I think any time you ask
of the user to choose it should be later in the sales process. For statistics
wonks, mention it in the FAQ, or something. But I wouldn't clutter up the
initial offer with too much stuff like that. I wish I had a product that was
this simple of a sell.

And, of course, the more involved things get, the more expensive it will have
to be. I _really like_ five dollar market research. I might not be so
enthusiastic about $25 market research.

~~~
encoderer
Fair enough... but if this is just a completely random, not-even-remotely-
scientific man on the street approach, then what problem does this solve that
a poll i can add to my website/blog/favorite forum via Fantastico in 180
seconds doesn't?

~~~
SwellJoe
It puts it in front of people that don't already know about your product.

I can ask usability questions of my users, but their existing knowledge will
effect what is "easy" and "intuitive" (intuitive generally means, "what I'm
used to"). I can ask marketing questions of them, but I've _already_ pulled
them into my site, so my current message will be the one that gets reinforced.
I can ask "compare these two products" questions, but I already know a
preponderance of people on my site prefer my product to our competitors (one
of which is Fantastico, by the way). I can ask any number of things, and we do
frequently ask our customers opinions on things, but by virtue of them being
on our site the results are known to be skewed in exactly the way we don't
want them to be for a large number of questions.

Also, I don't want to clutter up my site with constant polls. When someone is
on our site, we have very clear goals we want to achieve: Teach them about the
benefits, help them choose the right products, show them how to use it, answer
their questions. Asking them random questions isn't my idea of good design,
when it comes to those goals.

------
bumbledraven
I just used this site, and learned that 18-34-year-old women find a picture of
me without a baby more attractive, while the 50+ demographic definitely
prefers the picture of me holding my niece. Biologial clocks at work? Anyway,
worth $5! ;) Well, sort of... I wish there was a way to view the votes by
(gender,age-group) pair, but most of the respondents were female, so I think
I'm interpreting the results correctly.

Also, I just emailed a friend about this and shared the PickFu URL for the
results of my photo survey on facebook, which according to Derek Sivers is a
great sign (<http://sivers.org/purplecow>).

Finally, note that you can see all the PickFu questions as they are
distributed to Mechanical Turk (even the "private" ones, obviously) at

[https://www.mturk.com/mturk/searchbar?requesterId=AI2HRFAYYS...](https://www.mturk.com/mturk/searchbar?requesterId=AI2HRFAYYSAW7&selectedSearchType=hitgroups)

I hope your site can scale. I predict growth.

~~~
johnli
Actually, there is a way you can view the votes by (gender,age-group,etc.) -
click on the "Option A vs. Option B" tab right under "Who responded to your
question?" Is that what you were looking for? If so, it's our fault for not
making it more visible.

Thanks for the heads-up about the mturk job search - we're making tweaks so
that those previews show a sample question from now on. Helps keeps the
responses more random too.

Thanks for sharing it!

~~~
bumbledraven
No, "Option A vs Option B" isn't quite what I was looking for. Let us
stipulate that, once I see the results, I decide that I don't care what guys
think of my photo. I just want to compare the difference in response between
18-34-year-old women, 35-49-year-old-women, and 50+-year-old-women. I don't
see a way to get that information from your current interface.

~~~
johnli
Ah yeah, I get what you're talking about. It's one of the features I've been
wanting to add for a while too - filtering the responses by any demographic
combination. Not pri 1, maybe in the next version. =)

~~~
bumbledraven
I would be happy just to be able to access the raw responses in a .csv file or
something, and do the filtering myself.

------
IsaacSchlueter
My first thought: Why can't I answer the questions?

I know you're going to say something along the lines of "we want to ensure
that users are unique, and demographically diverse, blah blah blah."

Yeah, I know, but still. When I see "Mac or PC", I instinctively click on the
"Mac" option, and when nothing happens, I think "Well, _THAT's_ clearly
broken!"

I know you probably don't want to create just another hot-or-not site. But,
since you have public commenting that's separate from the "official"
responses, why not also have public voting that's segregated? Maybe if I give
you my demographic info, you could let me answer questions.

The bonus is that you don't have to pay me ;) So, perhaps you could offer a
few options: $5 for "valid" demographically-distributed statistically-
significant mechanical turk answers, or you can get all the answers you want
for free from the PickFu.com community (still organized by demographics.)

~~~
justinchen
Good idea. We've heard similar feedback before but haven't had a chance to get
around to it. We'll revisit it.

------
mdolon
Reminds me of a design client of mine... <http://www.feedbackarmy.com/>

Similar concept but a bit more information from the Turks at a slightly higher
cost.

------
run4yourlives
A user friendly MTurk front end. Not a bad idea. I wish you the best of luck.

------
poppysan
Micro-surveys. Cheap Price, Fast response. I could actually see using this in
the future.

------
edw519
I may not be typical, but of all the "review my site" postings, both good and
bad, this is the first one I actually plan to use. Pickfu.com is now a
Favorite.

1 vote with $ = many votes without $

~~~
ujjwalg
I think simplicity is the key here. It is going to be useful to masses and not
constrained to just big companies. I personally used it as soon as I saw it
and will be using it again and again.

------
grandpa
I misread it the first time and wondered what you had against Pi.

------
lukas
Great idea guys! If you have to time to come up to SF you should definitely
drop by our Crowdsoucing Labor/Mechanical Turk meetup on June 10th. You can
RSVP here: [http://www.meetup.com/Amazon-Mechanical-Turk-
Crowdsourcing-W...](http://www.meetup.com/Amazon-Mechanical-Turk-
Crowdsourcing-Work-Meetup-Group)

The Amazon VP of Mechanical Turk is going to come by to answer questions, and
I'm sure she would love to talk to you all.

~~~
johnli
Thanks Lukas, I plan on being there. Looking forward to meeting up - you guys
are doing some awesome things with MTurk.

------
shawndrost
Also see <http://ask500.com>

------
tkiley
Rockin' idea. I was just looking for something like this to do some quick-and-
dirty market research for my startup.

I don't have much experience with mturk, but my one concern would be that
respondents may simply click randomly rather than honestly answering the
question; do you have any data on how honest and accurate your responses are?

~~~
johnli
Our respondents don't get paid unless they actually explain their answers -
which both reduces false responses and provides unique value and insight to
the service.

For example, we used this feature to dogfood our own FAQ page:
<http://pickfu.com/4UQM7U>

~~~
crocowhile
I still think doesn't help. I liked the idea and did the turk for one of your
questions (this was my first time doing the turk, btw). I was asked to visit a
myspace webpage, listen entirely to 2 songs and say which one I like better
and why. All this for 3 cents? I did it, cause I am a good fellow but I have
hard time thinking of people who are willing to spend 10 minutes of their time
for 3 cents.

the FAQ example is a bad example: I can always answer "yes" and then supply as
reason "every questions I had was answered". No need to read the FAQ page, of
course.

I think if you want to insert any kind of validation system you would have to
count the time it took for the turk to answer your question and make sure we
are in that range for every hit.

~~~
johnli
As noted before, it's tough to know that 100% of the responses are truthful on
any survey. With PickFu, you can read the explanations to get a feel as to
which respondents put some thought in.

I'm sorry you had a rough experience with that specific music-listening job -
we've never had a more time-consuming question submitted. Most are in the
order of "which site do you prefer?", like this current one:
<http://pickfu.com/AGSIBT>. These get answered far faster by the MTurk
workers.

------
jgilliam
Another similar site:

<http://www.ask500people.com/>

------
pierrefar
Here is a controversial thought: Your price is too low. You tell me $5 for 50
comments, and I think, these are low quality comments.

You're underselling yourself IMHO. Position your service as a more premium
offering.

I'd do $50 for 50 comments. At least.

~~~
natrius
I disagree. I am almost guaranteed to use this in the near future because it's
so ridiculously cheap. At $50, I'd think about it a lot more and probably
attempt to do it myself. Are there more people like me, or more people who
wouldn't use it because it's too cheap to trust?

------
netsp
Great app. A few thoughts.

\- The demographic segmentation just doesn't work with 50 answers. Maybe
package it (say $5, $20, $50) & either only offer segmentation to the higher
ones or recommend against reading too much in to the lower ones.

\- Host the images & otherwise make the image part easy. I'm guessing this is
an upcoming feature. Since you clearly set out to make market research cheap &
easy, this is part of that. You'd be surprised how many people don't know an
image has a URL. Also make it easy to get online images from websites without
knowing images have URLs.

------
prpon
Nice site. Clear goals. One suggestion: Why limit at 50 responses? What if I
want more than 50 people to participate in the survey/market research? Can I
setup something? It wasn't obvious from the web site.

~~~
justinchen
It's limited mainly for simplicity. Since it's intended to be a cheap and
quick answer, we figured $5 and 50 questions seemed reasonable for most.
Perhaps we can keep the standard, advertised solution at 50 for $5 but when
you buy offer the option to upgrade to more questions.

~~~
jsatok
I personally think $5 is a great price point, but would make 3 tiers of plans.

1) Free, limited number of responses (maybe 5/month?), ad supported.

2) $5, Unlimited number of responses, no ads.

3) Skinable version for companies, allows them to manage questions/responses
asked by employees.

------
jackowayed
Really nice overall, but I have one UI issue:

When I go to an example page, it says something like, "Survey says: B! (18 of
33 answers)"

And then I never remember which was A and which was B, so I have to look.

Why not "Survey says: Gmail!" or "Survey says: Obama" instead? Especially for
the text ones. For the image ones it kind of makes sense to do A/B since they
don't necessarily ever tell you a name for each image (though you could change
that), but for the ones that are just text, it would be way more user-friendly
to put the name in.

------
tezza
I know domain names are hard BUT

.

your domain looks like :: PickFuck :: to a casual glance.

.

Similar to how the FCUK brand (intentionally) looks like FUCK.

\--

I think your site is good, although it seems the demographics are exclusively
American. Hopefully you could add some multinational demographic later.

Good luck and well done!

------
nreece
Impressive! Keep up the good work.

Just wondering, how has RPX ( <https://rpxnow.com/> ) Login
outsourcing/integration worked for you guys? Would you recommend it highly?

Edit: Btw, the Yahoo OpenId login prompts: "Warning: pickfu.rpxnow.com has not
forwarded their Privacy Policy."

~~~
justinchen
For a site as simple as this one it's worked out pretty well. Easy to drop in
and 1 less thing to think about it.

Thanks for the privacy warning - we'll look into it.

------
netsp
This is an excellent idea. Flying way, way under traditional solutions.

------
garply
Who is your target demographic (whom do you expect to purchase these surveys)?
Small companies?

~~~
justinchen
Anyone on a budget or looking for a quick, cheap answer. Small businesses and
entrepreneurs like ourselves were the initial target.

------
frossie
What is the geographic distribution of your responders?

~~~
justinchen
It's almost entirely US-based.

~~~
pclark
I'd make that clear if you haven't already :)

------
zackattack
Wow, this is pretty upsetting because I have been working on the exact same
idea as you. How long has it been up, what kind of volume have you been doing,
and how long did it take you to make it?

~~~
apsurd
How about change "upsetting" to "motivating" and you're good.

The world is a big place, and competitors mean the market exists.

~~~
zackattack
Competitors doesn't mean a significant market exists, it means multiple people
have come up with the idea.

------
sho
_"You can spend hours at a coffee shop polling 50 random people or bug
everyone you know, but you'll be getting a skewed population"_

Not to rain on your impressive parade, but I'd consider the type of person
that wants or needs to use MTurk to be a pretty skewed population too, just in
a different way.

Not saying it's not valuable, it may well be, and certainly more honest than
asking friends - but I would expect a fairly low-income or bored-teen
demographic.

Actually, an interesting meta-survey might be to survey the MTurk population
itself, see what they think, their incomes, political beliefs etc; might be an
interesting use of this very service.

But enough nit-picking - congratulations on a great idea, swiftly executed.

~~~
menloparkbum
_survey the MTurk population itself_

a few people have done this already

[http://behind-the-enemy-
lines.blogspot.com/2008/03/mechanica...](http://behind-the-enemy-
lines.blogspot.com/2008/03/mechanical-turk-demographics.html)

<http://waxy.org/2008/11/the_faces_of_mechanical_turk/>

