
Complaint-Driven Development - dieulot
http://www.codinghorror.com/blog/2014/02/complaint-driven-development.html
======
foolrush
The issue here is that it cleaves toward designing for the audience you have,
as opposed to the audience you seek to appease.

Often, the design goals might exist on an alternate axis.

Random people offering random opinions amounts to random noise. However, the
random noise will not appear neutral; it will appear as information.

If we consider many different things designed for particular audience members
such as jet cockpits, medical tools, racing automobiles, we will see traits
that exist that may seem nonsensical or otherwise when we divorce them from
their designed contexts.

Bill Buxton covers this in Sketching User Interfaces when he describes Inuit
coastal maps: "The Inuit have used. [...] Tactile maps of the coastline,
carved out of wood. They can be carried inside your mittens, so your hands
stay warm. They have infinite battery life, and can be read, even in the six
months of the year that it is dark. And, if they are accidentally dropped into
the water, they float. What you and I might see as a stick, for the Inuit can
be an elegant design solution that is appropriate for their particular
environment."[1]

Focusing on complaints of the above design in all likelihood would, given the
mad rabble of audiences online, result in discarding a solid bit of design.

[1] Bill Buxton, Sketching User Experiences, pg 37

~~~
jobu
Listening do user complaints doesn't always mean listening to their
suggestions in fixing the problem. You shouldn't necessarily remove something
that every user is asking you to remove if you feel it's a critical feature,
but it definitely means you need to rethink how the feature works.

~~~
DorintheFlora
From the article:

 _Since that change, I haven 't heard word one about our terrible, onerous,
awful default body and title character limit policies. Not one. Single.
Complaint._

So, they didn't do what the customers said they wanted, but they did work on
the issue until people quit complaining.

“If I had asked people what they wanted, they would have said faster horses.”

― Henry Ford

Plus, IIRC, for the Edsel, they did ask customers what they wanted. And it has
gone down in history as one of the worst cars ever.

~~~
mcormier
Yeah. Jeff didn't do what the customers wanted and he's smart enough to not
let users sway the design. I think the issue here is that it is a potential
pitfall that should have garnered a sentence or two for less experienced
people that read the article.

------
jorgeleo
I like the fact that this blog talks about the elephant in the room

It really does not matter what methodology, or tools you will use, if at the
end it does not pass the user acceptance test

So why consider it a failure if you have users telling you what they want and
how off you are? Better to embrace it and make a better product

~~~
csbrooks
This line toward the end is key too, though:

>That's how you suss out the rare 10% of community feedback that is amazing
and transformative.

So don't obsess over every bit of user feedback, but figure out what matters,
and what multiple people are complaining about, not just one loud person.

~~~
codinghorror
Yes exactly -- we weight common feedback from multiple Discourse instances
much more heavily than individual opinions from folks on meta.discourse, for
example.

(There is also the issue of observing what people _do_ versus what they say
they do. At least early on I find the problems are severe and obvious enough
that this subtlety isn't a big deal yet.)

------
wavesounds
This is great for a startup but can be depressing for long term consulting.

I've had the unfortunate experience of building a product for someone else
where the process was driven by a combination of Complain-Driven Development
and Upper-Management Wish-lists. This alone might have been fine but at the
same time anything positive like the real analytics about how successful the
product was or any non-complaint communication coming from the users was
hidden from me for fear, I suppose, that I might try to use that information
get my company more money.

This became incredibly depressing. Everyday you show up to work putting in
more and more hours into something that comes back with more and more
complaints. It was hell and I did everything I could to end Complaint-Driven
Development to no avail because thats how the customer liked to work.
Eventually I finally just gave up and left to find something more rewarding
and less soul crushing.

~~~
Mz
I am still trying to work out how to get useful feedback at all. Due to a
fairly unique situation, people divide up between those folks who tell I am
awesome (with zero useful feedback as to why) and folks who want my head on a
pike) and zero useful feedback as to why). So far, backing away from the
crowds of folks who want my head on a pike is the only constructive answer I
have come up with, which leaves me more and more and isolated. I remain
frustrated as to how to resolve this issue. I need skeptical feedback, not
hatred and not idolatry. Neither of those is productive for anyone.

So, yes, there can be big problems with this, depending in part on the problem
space you are addressing, I think.

------
mathattack
This process takes a lot of humility. It's not easy to say, "We are smart, but
we don't know, so let's just get something out there." It's antithetical to
the planning mania in so many big companies, and also why innovation like this
is best from small places. I wish them the best!

~~~
atmosx
Shakespeare thought otherwise, apparently:

"The fool doth think he is wise, but the wise man knows himself to be a
fool."\- William Shakespeare, As You Like It

~~~
RyanMcGreal
Or as it's known today, the Dunning-Kruger effect.

------
jackson1988
The only thing I've ever seen work is getting down deep and dirty in the
trenches with your users, communicating with them and cultivating
relationships. That's how you suss out the rare 10% of community feedback that
is amazing and transformative.

This is simple but practical advice I think far too many people ignore. You've
inspired me with this article. Glad to see you've been successful from it!

------
gingerlime
I totally agree. The problem for us however is actually deciding what's the
most complained-about request, and how to categorize those requests. I find
that we each have our pet hates and loves, and even if we don't mean to, we
develop selective hearing for complaints or praises.

We tried to add tags to support emails (via helpscout), but it's also hard to
remember to tag things, and it's easy to use different tags for similar
complaints.

I wonder about the best strategy to quantify complaints / suggestions and
'bucket' them correctly, so you can really choose the top ones.

~~~
frandroid
That's when you need to look back, and not just focus on the most popular
complained-about things, but also look at the qualitative feedback you're
getting, and they to synthesize commonalities between different lines of
feedback. Like another commenter said, sometimes feedback will help you reach
a local maximum, but you need to infer /why/ complaints come in the first
place, now just what people are complaining about, to know where to go next.
If you have so many complaints and there's no clear direction, either you're
at the nitpicking phase, or there are larger problems that you haven't
identified yet.

If your current tagging system isn't helping you, flip the axis on which
you're tagging to see if new patterns emerge. (area of complain on your
product, versus types of tasks users are trying to achieve, for example.)

------
igravious
Tried installing Discourse on Gentoo. Turns out to be a non-trivial project
and I'm no Ruby on Rails newb. I'm no code-jock but I'd love if the install
story was a tad easier. Maybe I should quit whining and figure out how to turn
my pain into an ebuild :)

~~~
codinghorror
Agree that one issue in the Ruby ecosystem is that installs are too hard. Can
you try our Docker install, we hope to move to this soon as our standard
recommended install for that very reason, would love feedback:

[https://github.com/discourse/discourse_docker](https://github.com/discourse/discourse_docker)

~~~
Mz
I know nothing at all about how the install works for this, so excuse me if I
am putting my foot in my mouth, but let me suggest you consider a stop-gap
measure of supplying a screenshot based tutorial for the install. (example:
[http://micheleincalifornia.blogspot.com/2013/12/jabiru.html](http://micheleincalifornia.blogspot.com/2013/12/jabiru.html)
\-- I spent two days trying to get this working, with tech support and I have
been told the tutorial I wrote works in 5 minutes or less)

~~~
codinghorror
Indeed, the community contributed a screenshot based walkthrough here:

[https://meta.discourse.org/t/beginners-guide-to-deploy-
disco...](https://meta.discourse.org/t/beginners-guide-to-deploy-discourse-on-
digital-ocean-using-docker/12156)

------
yoha
Something is not clear about the character count requirement: did they set it
to one or just finally found the right way to present it? If this is the
former as the messages from the dialogs suggest, I think he should have
highlighted the fact that the problem was not one of design (i.e. making users
understand the limit), but functional (i.e. not forcing users to pad a too-
short message).

~~~
EvilTrout
No we left it as is, but it is configurable in the admin section. This is very
important for non-English languages where one or two characters is often
enough for a descriptive title.

~~~
yoha
> one or two characters is often enough for a descriptive title.

I can't think of a many such titles though. Maybe "UX" for a general
discussion about user experience?

Edit: oups, I had read "English speaking" rather than "non-English" ; with
ideograms, it is actually quite obvious, thanks for the remark!

~~~
brianwawok
You cut off the non-English part of that quote. How much can you express in
Chinese with 2 characters?

~~~
gbog
Kind of the same as with two english words (nouns or verbs, not counting stop
words)

So, depending on your writing skills, it can be a lot. You could use the
begining of a poem or a sentence, which would be understood by the readers.
For instance if you say “温古" it refers to a Confucius quote meaning "warming
the past to learn the new".

But more interestingly, I think the restrictive and top-bottom rules edicted
in Discourse and Stack Overflow may be some kind of mistake, or ultimately
lead to dead ends. It goes against the natural grain of all things internet.
It is AOLish, if you allow the neologism.

The leading successes of internet, i.e. Wikipedia, Google search, have had
their massive growth and exitement when they were in extensively all-accepting
mode. Reddit and 4chan are also in this kind of mode.

It seems probable to me that the next break-through will be again lifting
artificial limitations there is on e.g. Stack Overflow.

------
dodger
One time I was talking to a designer about doing it this way. The designer
said "Yeah, that's a great way to find a (look of total disdain)
_local_maximum_." As usual, some truth to both points of view.

------
mrxd
Terrible design. “20 to go for the reply” is comically bad UI writing. And
then hiding it in a corner in grey text when the user is fixating on the field
they are typing in? Not surprised at all it didn't work. And the "nuclear
option" doesn't even meet basic usability guidelines of informing the user of
input field requirements upfront, not just relying on error messaging.

------
RogerL
While I grasp the reasoning in the article, and sometimes practice it, I also
think it is often either counter productive or impossible.

For example, I have friends that released a rather ho-hum mobile app. They
quickly garnered something like a 1.5 star rating, scathing reviews, and
almost no conversions. The business cycle on this was a year, and they are
still trying to claw back reputation and win users. It's a debacle. (the
problems weren't their fault, but that is irrelevant to this point).

Then you have companies with secrecy, like Apple. I think this advice would be
terrible for them (I have never worked there, and am open to correction). They
can't dog food it widely due to the internal silos, and they certainly cannot
test it with the public.

Then there are electronic systems - iterations on SW is easy, iterations on HW
expensive and hard, even with simulations, mock ups, and what have you. I
worked on an augmented reality hardware thingy several years ago; we went from
foam cutouts to a couple of very expensive prototypes, and that was it.

It is awesome when we can completely sidestep a problem, and this process lets
you sometimes sidestep the serious difficulty of UI design. I worry when it
gets bandied about as a truism, or The One True Way (not saying Jeff is doing
that, I'm remarking on the wider industry). Yes, Agile lets you sidestep the
problem of scheduling and estimation - sometimes. Try that when you are making
a new airliner, building a cloverleaf interchange, making a car entertainment
system, and so on.

edit: the converse problem is equally as large. Someone below mentioned the
'planning mania' of companies. I don't mean to downplay that problem, just to
point out the need to evaluate each situation on it's particular needs as
opposed to a 'best practices' (oh, how I hate that term) unthinking approach.

~~~
foolrush
"Then you have companies with secrecy, like Apple. I think this advice would
be terrible for them (I have never worked there, and am open to correction).
They can't dog food it widely due to the internal silos, and they certainly
cannot test it with the public"

I fear that this myth is capitalized on by Apple under the hypercapitalist
Auteur theory.[1]

The value of magic tricks is that they create awe and surprise, when in
reality they can often be a by-product of misdirection and street-level
sleights.[2]

This is not to suggest that Apple is going out onto street corners and asking
random individuals for their opinions, but rather that part of the corporate
myth making is likely obscuring process.

[1]
[http://www.youtube.com/watch?v=p9dmcRbuTMY](http://www.youtube.com/watch?v=p9dmcRbuTMY)

[2] I believe there is a court testimony document discussing Apple's market
research team via Phil Schiller. Contrast this with Mr. Schiller's other
comments about avoiding market research.

------
jchung
This definitely resonates. Although while feedback from discourse users
naturally takes place on discourse, gathering feedback from users can often be
much more difficult. We've enabled usersnap recently, which has been somewhat
helpful, but hasn't created the kind of vociferous user-to-user and user-to-
developer debates that you see on meta.discourse

~~~
maxerickson
When I'm giving feedback, I usually don't want the developer to tell me why it
is wrong. I don't mean that in the sense that my perspective is correct and my
feedback is never wrong, I mean that if I am irritated enough about something
to report it as a flaw, the last thing I am going to want to do is engage in a
debate about it.

BTW, discourse infinite scroll could behave better when the user input is the
down arrow on the scroll bar.

------
kitd
Did he consider that the way _he_ wanted Discourse to be used may not have
tallied with the way his users wanted to use it?

That's the underlying disconnection.

I am now working on my 3rd message-transformation/integration middleware
product for a large corp. In each case, the issue tracker has been stock full
of problems from people who took the product, read all the manuals about what
they were supposed to do, then did it their way anyway. Because, well if they
can, they will.

I think the basic problem is not challenging your personal assumptions about
how you want your beloved system to be used.

------
collyw
One problem with this is that users don't know what they want or what is easy
/ possible / hard.

I constantly get asked to add another "excel parser" to my in house Django
app. These days I refuse (it is way too error prone) and build it directly
into the web interface. Then they realize, that even with all the nice auto-
complete and cut and paste features that excel comes with, clicking a few
checkboxes and a submit button in a web interface is more efficient (and less
error prone) than cutting and pasting the data into Exel in the first place.

------
sopooneo
I feel like responding to user complaints can help bring you closer to the
nearest local maxima. But it often won't bring you beyond that unless your
users are themselves UX designers or developers.

So it's useful to make your MVC as good as possible so that your users aren't
forced to complain about the results of fundamental design shortcomings. That
can result in more and more complex fixes, none of which should be necessary.

------
leaxdc
We in our company preferred chair-driven development - it's in case of failed
estimate you are lashed out by chief's office chair.

------
xfalcox
Valve does this very well using reddit. A bad bug goes for a whole year on
Dota 2, and if someone complains on /r/Dota2 it gets fixed in two days.

It's great, users fells empowered, and you get the priority list for free with
reddit upvote system.

As for discourse were planning to deploy it this year at our intranet, and
we've got 130.000 employees.

------
the1
why not hire good UX experts who can capture these complains in product design
phase?

Contact me if you need help. I'm a good UXpert.

~~~
kbutler
Because even the best experts are not going to be able to identify
requirements the users won't discover until they begin using the product.

You can approximate some of this feedback by observing user interaction with
prototypes, but if you pay attention, you'll learn more when they start using
it in production.

------
boohoofoodoo
[http://laughingsquid.com/wp-
content/uploads/2013/07/homer2.j...](http://laughingsquid.com/wp-
content/uploads/2013/07/homer2.jpg)

~~~
codinghorror
Right, this is why you _listen_ to your community, but you don't blindly do
everything they tell you to do. Filtering out the best 10% of feedback -- and
resisting the transformation of the car you designed into a truck because "the
community demanded it" \-- is the way to go.

------
chris_wot
Perhaps this is something the Gnome guys could practice?

------
andyl
I also practice CDD. But my experience is that few people complain. Maybe 1
out of 20. The rest silently endure a bad UI, walk away from the app without
comment, or badmouth the app to their trusted friends.

Complainers have to be cultivated. Complainers can be your most valuable
asset.

~~~
codinghorror
True. Any complaints are already the tip of an iceberg you can't see.

And the people complaining should be treated like your best friends: they
_cared enough_ to complain. Nobody else cared.

~~~
FLUX-YOU
If you want to view a world where the physics of icebergs have only the tip
under the water, take a look at gaming development and feedback. :)

------
johnny635
I've seen this works extremely profitably in the past. Hard to argue with
results.

------
sergiotapia
Offtopic: I find it really tacky how this blog constantly links back to other
posts of his hidden in the text.

~~~
yoha
This. Websites are supposed to distinguish internal and external links. It
makes it way easier to now what kind of information we will be getting when
clicking. My first thought is for a blog [1] which does it pretty well:
(internal links are underlined.

[1] [http://www.bortzmeyer.org/](http://www.bortzmeyer.org/) (fr)

(I could not think of an English-speaking website with a similar
differentiation right away)

Edit: just look at Wikipedia
[https://en.wikipedia.org/wiki/Hacker_news](https://en.wikipedia.org/wiki/Hacker_news)

~~~
tragic
Wikipedia does.

But really, the status bar hover thing in every browser since IE5 is more than
enough for me to work out where a link is going to go. (Guess it's not so
useful on mobile.)

~~~
aestra
The funny thing is on mobile I long press to see where a link is going to go.
I do this all the time.

On Dolphin long press gives you the link at the top (usually truncated though)
then a list of options like "open in new tab" "open in background" "save link"
etc. It just seems important to me to see where a link is going to go for
whatever reason.

