

Review HN: Feedback Army - pclark

I thought I'd try out feedback army on my new marketing website. To ensure that what I'd written was in line with what users understood it as.<p>For $10 you get 10 people to anonymously browse your site.<p>You give it a URL, and you can ask the users upto six questions, the default questions are pretty great. Along the lines of "what is the site about? how does it work? how do you get it?" etc.<p>I expected them to give one liners - but holy cow, I got entire paragraphs. They left comments on blog posts, they clicked and interacted with every element.<p>I guess the "negatives" to this is you can't guarantee quality testers - do I know if they even checked my site?" I used clicktale to track where they clicked, and you can even reject reviews that are nonsense.<p>I've only got 8/10 "reviews" so far, and they're really excellent. I'll be doing it again before we post it on Hacker News :)<p>http://feedbackarmy.com<p>I'm not affiliated in anyway... maybe this was the startup I inspired, thinking about it, I'll have to check Hacker News archives, I just thought you guys would appreciate this service.<p>too lazy to read: for $10 you get 10 users to beta test your site, and they answer your questions about your site.
======
raffi
Feedback Army was actually inspired by an HN post a long time ago. I wrote
(and submitted) some history on it, here:
[http://killall.dashnine.org/2009/06/how-to-find-new-
startup-...](http://killall.dashnine.org/2009/06/how-to-find-new-startup-
ideas/)

That said, I'm an HN user too. If anyone has any questions you can contact me
via the site, call the number, or send a carrier pigeon.

Thanks for the review too. Always appreciated.

P.S. I just found the original thread,
<http://news.ycombinator.com/item?id=362459> yes pcclark, you and a few others
were the inspiration for this project.

~~~
icey
I discovered something new today with vhosts.

I clicked your link to feedback army from your dashnine page, and it took me
to AtD:

<http://www.feedbackarmy.com./>

(Thought you might like to know - click the link!)

------
staunch
I think I'd always choose UserTesting ( <http://www.usertesting.com/> ) over
this. You get the same kind of feedback, can ask questions/give tasks, and you
get to see/hear them go through the site in real time.

I'm not affiliated with UserTesting, except as a customer.

~~~
rrikhy
I wholly agree with you on the benefits of watching a user interact with your
site, but I find Userfly : www.userfly.com 's delivery model to be more cost-
effective. HUGE con against userfly is no user narrative, this proves to be a
big downfall when it comes to iterative development, because correlation =!
causation.

------
imp
Yeah, I've used it too. Got great feedback and it only took about 3 hrs to get
all 10 responses. Definitely worth $10.

~~~
jcapote
Seconded. Cheap, fast and was blown away by the quality of the feedback.

~~~
PStamatiou
only issue is people aren't going to create an account and give you the
feedback you really need - inside your site. Homepage/etc feedback is easy to
come by, by comparison.

That being said I have used Feedback Army once before months ago and received
decent to good feedback quality. I guess I just need to ask better questions.

------
dryicerx
Excellent for initial usability testing.

The only gripe I have with crowd sourcing apps is the inability to distinguish
those who write truthfully vs who lie (they might write legibly to get their
response accepted, but the response is a lie). It's not a problem with this
service, just a nasty side effect of the crowd sourcing and using people.

------
icey
I've read about this a few times, but never really knew what the quality would
be like. It's good to hear it's pretty well done.

------
eswat
I've only used it once previously, but I was also surprised at the verbosity
of the responses. (both a good and bad thing)

------
roryokane
I saw one example result that said the user came from "Turk". Presumably, the
site just generates ten Mechanical Turk tasks that pay, say, 75 cents, and
keeps the other 25 cents for itself. I wonder if you could just put the tasks
up yourself and get the same reviews cheaper, or if the site does something
more than this.

~~~
raffi
I address this on the FBA FAQ, question 10:
<http://www.feedbackarmy.com/questions.slp>

In short, here is the value add of Feedback Army:

\- it's easier to get started with and use \- results are available as an RSS
feed \- FBA worker form now has built-in grammar and spell check courtesy of
<http://www.afterthedeadline.com>

If you don't like a response you get, it's a one click rejection process.
Mechanical Turk lets you reject responses too, but it's a few more
clicks/pages than one.

------
profquail
You know what else what be a cool service for them to offer? Feedback from
_experts_. Obviously it'd be more expensive, but if you could submit the site
to be reviewed by a panel of UI experts, software engineers, etc., I think
that'd be a pretty worthwhile service as well, especially for startups.

~~~
thorax
For the slightly-more-expert take, you might like a site like this (seems like
99 designs but for usability reviews):

<http://usabilitytest.com/>

Also I saw this service advertised before:

<http://www.usabilityfeedback.com/>

I've never used either of those, but I have used Feedback Army and can
recommend it as a way to get sanity checks with multiple people. Sometimes
it's hard to tell if a certain stock photo is "creepy" unless you launch or
use a service like FA, so it ends up being well worth such a low fee (for us
anyway).

~~~
redorb
This may come off as corny but - who is more expert than real internet users?

\- I suggest that by being an "expert" (whatever that means) you would have
too much 'domain knowledge' to know what would work best from the average
internet user.

\- sounds like the biased sample fallacy
<http://www.nizkor.org/features/fallacies/biased-sample.html>

\- I think best idea would be for a/b split testing perhaps using the 'army'
to configure your different tests..

~~~
profquail
What I meant by "experts" is people who could look at your site and give it a
quick lookover according to their specialty. Web software engineers might look
at a site and see code that doesn't validate, or AJAX that could be optimized
a bit; UI designers might suggest that you reorder some of the buttons on your
page for better ease of use, etc.

------
pie
It was vexing that the "3. See results!" link does not point to the example
results page. I tend to read stenciled-text graphics last.

Feeding back the feedbackers.

~~~
pclark
I'll blog my results

------
thorax
Used it before and got great feedback. Highly recommended, but I think when we
did it initially it was slightly cheaper. Still very much a bargain.

------
pclark
ah, this was the original "introducing feedback army!" thread (not by me, but
I guess I'm inadvertently connected..) :
<http://news.ycombinator.com/item?id=785489>

~~~
dimarco
infinite loop!

