

FeedbackArmy.com - simple, cheap usability testing for your site - raffi
http://www.feedbackarmy.com/

======
aditya
Wow. There's a lot of negativity and criticism on here, don't be
disillusioned, you got something up and out in 2 weeks that says more about
you than most other people spending time "commenting" on here :-)

Best of luck, and get on that usability thing these people are talking
about...

~~~
raffi
Thanks for the encouragement. I've been meaning to play with mechanical turk
for awhile and this seemed like a good idea. I'm going to hire a professional
and get a snappy design going as quick as possible.

------
dmix
Hmmm purchasing usability testing from a site using a dated design and bright
green font on a black background?

No thanks.

~~~
joseakle
Here's my free usability review of that site:

The font is too small.

~~~
ivankirigin
I thought the same thing. I closed the tab a moment after coming to the page,
despite being in the market for such testing.

~~~
raffi
<3 I can't disagree with what you're saying. My design skills are weak but I
wanted to validate the concept first. Today I've seen enough interest to
validate moving forward with the project. I'm going to invest in a site
facelift to bring the presentation to Web 2.0 standards. Thank you for taking
the time to look and post your thoughts here. -- Raphael

~~~
ivankirigin
Checkout the account creation or sign in form on Tumblr.
<http://www.tumblr.com/login>

It's huge.

------
patio11
I went ahead and bought a review. I just spent a few hundred dollars launching
a mini-site, what is an extra $7? Rounding error -- if they catch a typo I'll
consider it money well spent.

I'll let you know about the quality of results. One request: my results should
not be available on a publicly accessible URL. It doesn't bother me,
personally, but it will bother some people.

~~~
patio11
Here are the results of my experiment, as promised.

Here's my little mini-site:

<http://www.christmasbingocards.com>

This is a holiday promotion aimed at my customers. They're overwhelmingly non-
technical folks, and not too sophisticated at the Internet. (I am well-aware
that most people here would not purchase my software. I'm totally OK with
that: you are not the audience.)

I asked the Turkers three questions (and will tell you my rationales):

1) Is it IMMEDIATELY clear what the site offers when you open it? [Rationale:
This site is a sledgehammer, not a scalpel. I want them to understand what it
offers before they hit their back button. If thought is required, I failed at
design.]

2) Try to print out Christmas bingo cards. Tell me how long it takes you from
me giving you that instruction to something coming out from your printer.
[Rationale: My target is that fourty-something mother with kids, with no
special computer skills, can accomplish this in 90 seconds. There is actually
a bit of an overlap between Turkers and my market.]

3) If you thought your friends would find Christmas bingo to be a fun
activity, would you recommend this site for it to them? Why or why not?
[Rationale: The Turking population has a significant overlap with my core
audience. Why not ask a quick thumbs up/down?]

What it taught me:

1) The site achieves its singular purpose fairly well. 2) It is attractive
enough to move several testers into mentioning it to their friends. (As in,
not hypothetical.)

The feedback lets me know "OK, the site is basically OK and needs to be built-
out more than rethought from scratch". Knowing that, and the peace of mind
from knowing that, are worth $7 to me easily. I hope this service continues
getting developed, because I can see a lot of uses for it in my business.

(Yeah, I could roll my own with the Turk API... but I value my time at $100 an
hour. This means I never want to be playing the sit-at-computer-refreshing-
Amazon-so-I-can-authorize-fourty-cent-payment game.)

------
brandnewlow
What are the odds that the idea came from reading comments here:
<http://news.ycombinator.com/item?id=372305>

~~~
raffi
Actually the idea came from <http://news.ycombinator.com/item?id=362459>

~~~
dmix
What value to add to the process?

Why wouldn't I just post up the job on Amazon?

~~~
raffi
Lets assume you've never posted a job to Mechanical Turk before. Here is what
you need to do:

1\. Create an Amazon Web Services account

2\. Figure out how much you want to spend (and don't forget to calculate
Amazon's share)

3\. Prepay the exact amount or more

4\. Create a Human Intelligence Task template

5\. Publish the Human Intelligence Task

6\. Login to Amazon's portal regularly and navigate several pages to find your
results. Have fun clicking refresh.

7\. Review the results and promptly pay your workers (or they may ignore
future requests)

How much is your time worth? Feedback Army spends the majority of the fee
towards paying the mechanical turk workers. The Feedback Army software saves
you from many of these steps and conveniently transforms the results into an
RSS feed.

~~~
dmix
I have never used Mechanical Turk before and I see that you are adding some
value.

If you improve the look of your own site you might have something here.

Another idea might be to offer larger scale tests with graphical reporting on
the results, basically using the low cost/test advantage that Turks offer.

~~~
raffi
Thanks. This is a pretty new project for me (< 2 weeks old) following the
release early, release often mentality. I've seen enough interest to validate
the idea and so I'm reaching out to a few designers I know to get a proper
face lift. I'll also dump a few dollars into making some design firms aware of
the service and see where that takes me.

------
sonink
I tried feedbackarmy - from a purely ROI perspective I think the usability
methodology is flawed.

It seems that the folks who give the feedback are actually trying to be a bit
too helpful - to the extent that they think about questions as a challenge and
try their best to answer them.

For example - If I pose a question that "What does this site do?" in a
usbility experiment, I am expecting to figure out if the audience immidiately
grasp the utility of a site.

However in this case it seems that the audience takes that as a challenge
modifying the question to mean "Can you figure out what this side does?" and
try out very hard to find the utility of a site and in the process totally
defeating the purpose of a usability review.

It might be an artifact of the way I posed the question, but then I was just
following the sites examples. I think using the site in the way it is setup
(messaging, mturk) the concept does not give a good ROI w.r.t usability.

However, one thing that I did like was that the turkers gave some suggestions
to improve the site - some of which was really valuable. So maybe if you
change your positioning to something like an active feedback/testing tool it
might work better.

------
grag
Anyone know if this uses Amazon Mechanical Turk?

~~~
raffi
it does (see the questions page)

------
iamdave
This is going to sound incredibly assholish, but it's incredibly true.

FeedbackArmy.com is going to tank. <http://www.feedbackarmy.com/feedback.slp>

1)That page is why. If someone has the capacity to produce (or have produced)
a website, they have the capacity to create a form with two elements and
garner feedback. It is incredibly lackluster, easy to replicate and
_certainly_ not worth $7 for 10 responses.

2) Based on the comments here people aren't sold on Usability Feedback on
websites when the website you've created doesn't suit the needs people are
coming to the site for in the first place.

3)The process lends itself quite easily for scammers and cons. There's almost
no security measures in place, the design is inherently broken and doesn't
provide any peace of mind from the process that you have to perform to get
"feedback".

I'm not sold. This can be done about 50 ways better than what it is now; I
really do not see myself paying any amount of money for the presentation
you've put in front of me. It's an effort, I wont say good effort, I wont say
bad effort. It's an effort made, and at this point I'm going to say no to
this.

~~~
thorax
It's $7, lighten-up. I'm about to spend $15/person/hour for deeper usability
tests and now I can ask less in-depth questions but as often as I want for
much less? Pure gold.

