Hacker News new | comments | show | ask | jobs | submit login
Complaint-Driven Development (codinghorror.com)
243 points by dieulot on Feb 18, 2014 | hide | past | web | favorite | 70 comments

The issue here is that it cleaves toward designing for the audience you have, as opposed to the audience you seek to appease.

Often, the design goals might exist on an alternate axis.

Random people offering random opinions amounts to random noise. However, the random noise will not appear neutral; it will appear as information.

If we consider many different things designed for particular audience members such as jet cockpits, medical tools, racing automobiles, we will see traits that exist that may seem nonsensical or otherwise when we divorce them from their designed contexts.

Bill Buxton covers this in Sketching User Interfaces when he describes Inuit coastal maps: "The Inuit have used. [...] Tactile maps of the coastline, carved out of wood. They can be carried inside your mittens, so your hands stay warm. They have infinite battery life, and can be read, even in the six months of the year that it is dark. And, if they are accidentally dropped into the water, they float. What you and I might see as a stick, for the Inuit can be an elegant design solution that is appropriate for their particular environment."[1]

Focusing on complaints of the above design in all likelihood would, given the mad rabble of audiences online, result in discarding a solid bit of design.

[1] Bill Buxton, Sketching User Experiences, pg 37

Listening do user complaints doesn't always mean listening to their suggestions in fixing the problem. You shouldn't necessarily remove something that every user is asking you to remove if you feel it's a critical feature, but it definitely means you need to rethink how the feature works.

From the article:

Since that change, I haven't heard word one about our terrible, onerous, awful default body and title character limit policies. Not one. Single. Complaint.

So, they didn't do what the customers said they wanted, but they did work on the issue until people quit complaining.

“If I had asked people what they wanted, they would have said faster horses.”

― Henry Ford

Plus, IIRC, for the Edsel, they did ask customers what they wanted. And it has gone down in history as one of the worst cars ever.

Yeah. Jeff didn't do what the customers wanted and he's smart enough to not let users sway the design. I think the issue here is that it is a potential pitfall that should have garnered a sentence or two for less experienced people that read the article.

> but it definitely means you need to rethink how the feature works.

Or how it's presented. It's pretty common for a feature to be the subject of complaints because no one actually realizes why you put it in.

Isn't it more likely that the tactile maps are used because they are superior to any other kind of map the Inuits could have created (non tactile drawing, or a rock that doesn't float, for example)?

A tad of a chicken-egg dilemma there. While I can only speculate, I'd surmise that the Inuit are / were well aware of their contextual circumstance and designed accordingly, and quite likely iterated on the design.[1]

The issue I raise is a cautionary tale that when we try to use some abstract evaluation well removed from the context, the "data" can end up being non-information. Worse is when design evaluation comes with hidden cognitive bias or underlying ideology.

That is, even a valid concern / complaint of a given design my have no merit given the design context. If we take for example a vehicle, one might complain that it has a lack of cargo space or a child seat restraint. Valid complaint given a very discrete set of design contexts. To follow along the original example in the post, these otherwise valid complaints of a design are moot if the vehicle in question is an F1 automobile designed to be raced in unique circumstances with unique needs.

[1] There is also the possibility that some of the design contexts include more abstract needs such as cultural, religious, historical, aesthetic, or even the darker notions of power dynamics. To diminish the potential influence of these facets would be an ignorance in our design evaluation.

I'd encourage anyone interested in design context to read Marco Van Hout's article on the rather broken UIGarden.net site. Here's a link via the Wayback Machine: http://web.archive.org/web/20130719161851/http://uigarden.ne...

I like the fact that this blog talks about the elephant in the room

It really does not matter what methodology, or tools you will use, if at the end it does not pass the user acceptance test

So why consider it a failure if you have users telling you what they want and how off you are? Better to embrace it and make a better product

This line toward the end is key too, though:

>That's how you suss out the rare 10% of community feedback that is amazing and transformative.

So don't obsess over every bit of user feedback, but figure out what matters, and what multiple people are complaining about, not just one loud person.

Yes exactly -- we weight common feedback from multiple Discourse instances much more heavily than individual opinions from folks on meta.discourse, for example.

(There is also the issue of observing what people do versus what they say they do. At least early on I find the problems are severe and obvious enough that this subtlety isn't a big deal yet.)

This is great for a startup but can be depressing for long term consulting.

I've had the unfortunate experience of building a product for someone else where the process was driven by a combination of Complain-Driven Development and Upper-Management Wish-lists. This alone might have been fine but at the same time anything positive like the real analytics about how successful the product was or any non-complaint communication coming from the users was hidden from me for fear, I suppose, that I might try to use that information get my company more money.

This became incredibly depressing. Everyday you show up to work putting in more and more hours into something that comes back with more and more complaints. It was hell and I did everything I could to end Complaint-Driven Development to no avail because thats how the customer liked to work. Eventually I finally just gave up and left to find something more rewarding and less soul crushing.

I am still trying to work out how to get useful feedback at all. Due to a fairly unique situation, people divide up between those folks who tell I am awesome (with zero useful feedback as to why) and folks who want my head on a pike) and zero useful feedback as to why). So far, backing away from the crowds of folks who want my head on a pike is the only constructive answer I have come up with, which leaves me more and more and isolated. I remain frustrated as to how to resolve this issue. I need skeptical feedback, not hatred and not idolatry. Neither of those is productive for anyone.

So, yes, there can be big problems with this, depending in part on the problem space you are addressing, I think.

"This became incredibly depressing. Everyday you show up to work putting in more and more hours into something that comes back with more and more complaints. It was hell and I did everything I could to end Complaint-Driven Development to no avail because thats how the customer liked to work. Eventually I finally just gave up and left to find something more rewarding and less soul crushing"

My current gig is very customer-driven and we hire a number of PMs to handle and incorporate the feedback. Our customers love us because the alternative is a particularly stodgy, poorly performing and relatively unchanging product. With end-users, yes. The flood of complaints and the complaints about the fixes for the complaints and on and on can be soulcrushing.

Finding a niche who loves you and your product is hopefully possible.

It also depends what stage you are at.

This article is about early stage products that are very new. If you're looking at a mature product that has been on the market for 10+ years, it's likely (I hope!) that most of the major obvious pain / complaint points have already been addressed. I don't think CDD would be as useful in that scenario -- you want a different model.

"Software auf Zuruf" ;)

This process takes a lot of humility. It's not easy to say, "We are smart, but we don't know, so let's just get something out there." It's antithetical to the planning mania in so many big companies, and also why innovation like this is best from small places. I wish them the best!

Shakespeare thought otherwise, apparently:

"The fool doth think he is wise, but the wise man knows himself to be a fool."- William Shakespeare, As You Like It

Or as it's known today, the Dunning-Kruger effect.

The only thing I've ever seen work is getting down deep and dirty in the trenches with your users, communicating with them and cultivating relationships. That's how you suss out the rare 10% of community feedback that is amazing and transformative.

This is simple but practical advice I think far too many people ignore. You've inspired me with this article. Glad to see you've been successful from it!

I totally agree. The problem for us however is actually deciding what's the most complained-about request, and how to categorize those requests. I find that we each have our pet hates and loves, and even if we don't mean to, we develop selective hearing for complaints or praises.

We tried to add tags to support emails (via helpscout), but it's also hard to remember to tag things, and it's easy to use different tags for similar complaints.

I wonder about the best strategy to quantify complaints / suggestions and 'bucket' them correctly, so you can really choose the top ones.

That's when you need to look back, and not just focus on the most popular complained-about things, but also look at the qualitative feedback you're getting, and they to synthesize commonalities between different lines of feedback. Like another commenter said, sometimes feedback will help you reach a local maximum, but you need to infer /why/ complaints come in the first place, now just what people are complaining about, to know where to go next. If you have so many complaints and there's no clear direction, either you're at the nitpicking phase, or there are larger problems that you haven't identified yet.

If your current tagging system isn't helping you, flip the axis on which you're tagging to see if new patterns emerge. (area of complain on your product, versus types of tasks users are trying to achieve, for example.)

We've been using User Voice for the computer game Torment: Tides of Numenera. First, we used it to get community feedback on our rewards, tiers, and approach for our Kickstarter campaign. It was very helpful in testing our ideas in advance.

Throughout development, we've been using it to get feedback from the community: on their own ideas, their thoughts on design aspects we've shared, and questions they'd like us to answer.

User Voice lets people suggest and vote on ideas. We can label them with customizable tags like "Seen," "Considering," "Not Planned," as a very time-efficient means of communicating to many at once, without the expectation of personal replies.

The system isn't perfect, and we're not using it perfectly, but it's been helpful for getting and organizing feedback.

That's why I really liked Dell's Idea Storm. Instead of worrying about what a single user thinks, you can only worry about the current most popular issues or requests.

I don't think it has really helped Dell as to many of the things they simply said they aren't doing it (e.g. standardise power supplies in laptops) but the concept seems decent if they weren't very open to the feedback received.

Tried installing Discourse on Gentoo. Turns out to be a non-trivial project and I'm no Ruby on Rails newb. I'm no code-jock but I'd love if the install story was a tad easier. Maybe I should quit whining and figure out how to turn my pain into an ebuild :)

Agree that one issue in the Ruby ecosystem is that installs are too hard. Can you try our Docker install, we hope to move to this soon as our standard recommended install for that very reason, would love feedback:


I know nothing at all about how the install works for this, so excuse me if I am putting my foot in my mouth, but let me suggest you consider a stop-gap measure of supplying a screenshot based tutorial for the install. (example: http://micheleincalifornia.blogspot.com/2013/12/jabiru.html -- I spent two days trying to get this working, with tech support and I have been told the tutorial I wrote works in 5 minutes or less)

Indeed, the community contributed a screenshot based walkthrough here:


Wow, thanks for responding!

I'll give the Docker install a spin. I'll give feedback on the Gentoo install and the Docker install when I am able to give both my full attention.

Your blog posts are frequently thought-provoking and inspirational and mercilessly bullshit-free. I'm so glad you've moved to Ruby on Rails and welcome you with open arms into the open-source world, you may not fully appreciate how much goodwill that move generated for you :)

Something is not clear about the character count requirement: did they set it to one or just finally found the right way to present it? If this is the former as the messages from the dialogs suggest, I think he should have highlighted the fact that the problem was not one of design (i.e. making users understand the limit), but functional (i.e. not forcing users to pad a too-short message).

No we left it as is, but it is configurable in the admin section. This is very important for non-English languages where one or two characters is often enough for a descriptive title.

> one or two characters is often enough for a descriptive title.

I can't think of a many such titles though. Maybe "UX" for a general discussion about user experience?

Edit: oups, I had read "English speaking" rather than "non-English" ; with ideograms, it is actually quite obvious, thanks for the remark!

> non-English languages

In Chinese and Korean, a single unicode codepoint encodes a full syllable, so two codepoints can be a word or even a short sentence.

You cut off the non-English part of that quote. How much can you express in Chinese with 2 characters?

Kind of the same as with two english words (nouns or verbs, not counting stop words)

So, depending on your writing skills, it can be a lot. You could use the begining of a poem or a sentence, which would be understood by the readers. For instance if you say “温古" it refers to a Confucius quote meaning "warming the past to learn the new".

But more interestingly, I think the restrictive and top-bottom rules edicted in Discourse and Stack Overflow may be some kind of mistake, or ultimately lead to dead ends. It goes against the natural grain of all things internet. It is AOLish, if you allow the neologism.

The leading successes of internet, i.e. Wikipedia, Google search, have had their massive growth and exitement when they were in extensively all-accepting mode. Reddit and 4chan are also in this kind of mode.

It seems probable to me that the next break-through will be again lifting artificial limitations there is on e.g. Stack Overflow.

One time I was talking to a designer about doing it this way. The designer said "Yeah, that's a great way to find a (look of total disdain) _local_maximum_." As usual, some truth to both points of view.

Terrible design. “20 to go for the reply” is comically bad UI writing. And then hiding it in a corner in grey text when the user is fixating on the field they are typing in? Not surprised at all it didn't work. And the "nuclear option" doesn't even meet basic usability guidelines of informing the user of input field requirements upfront, not just relying on error messaging.

While I grasp the reasoning in the article, and sometimes practice it, I also think it is often either counter productive or impossible.

For example, I have friends that released a rather ho-hum mobile app. They quickly garnered something like a 1.5 star rating, scathing reviews, and almost no conversions. The business cycle on this was a year, and they are still trying to claw back reputation and win users. It's a debacle. (the problems weren't their fault, but that is irrelevant to this point).

Then you have companies with secrecy, like Apple. I think this advice would be terrible for them (I have never worked there, and am open to correction). They can't dog food it widely due to the internal silos, and they certainly cannot test it with the public.

Then there are electronic systems - iterations on SW is easy, iterations on HW expensive and hard, even with simulations, mock ups, and what have you. I worked on an augmented reality hardware thingy several years ago; we went from foam cutouts to a couple of very expensive prototypes, and that was it.

It is awesome when we can completely sidestep a problem, and this process lets you sometimes sidestep the serious difficulty of UI design. I worry when it gets bandied about as a truism, or The One True Way (not saying Jeff is doing that, I'm remarking on the wider industry). Yes, Agile lets you sidestep the problem of scheduling and estimation - sometimes. Try that when you are making a new airliner, building a cloverleaf interchange, making a car entertainment system, and so on.

edit: the converse problem is equally as large. Someone below mentioned the 'planning mania' of companies. I don't mean to downplay that problem, just to point out the need to evaluate each situation on it's particular needs as opposed to a 'best practices' (oh, how I hate that term) unthinking approach.

"Then you have companies with secrecy, like Apple. I think this advice would be terrible for them (I have never worked there, and am open to correction). They can't dog food it widely due to the internal silos, and they certainly cannot test it with the public"

I fear that this myth is capitalized on by Apple under the hypercapitalist Auteur theory.[1]

The value of magic tricks is that they create awe and surprise, when in reality they can often be a by-product of misdirection and street-level sleights.[2]

This is not to suggest that Apple is going out onto street corners and asking random individuals for their opinions, but rather that part of the corporate myth making is likely obscuring process.

[1] http://www.youtube.com/watch?v=p9dmcRbuTMY

[2] I believe there is a court testimony document discussing Apple's market research team via Phil Schiller. Contrast this with Mr. Schiller's other comments about avoiding market research.

You can't release crap, even in your first version. Not saying your friends did that, but you have to have the kernel of something decent to show people before you release. Otherwise you're blowing the launch by taking all that initial interest and squandering it.

(I'd also argue mobile apps have a severe updating problem, that web apps do not.)

This definitely resonates. Although while feedback from discourse users naturally takes place on discourse, gathering feedback from users can often be much more difficult. We've enabled usersnap recently, which has been somewhat helpful, but hasn't created the kind of vociferous user-to-user and user-to-developer debates that you see on meta.discourse

When I'm giving feedback, I usually don't want the developer to tell me why it is wrong. I don't mean that in the sense that my perspective is correct and my feedback is never wrong, I mean that if I am irritated enough about something to report it as a flaw, the last thing I am going to want to do is engage in a debate about it.

BTW, discourse infinite scroll could behave better when the user input is the down arrow on the scroll bar.

Did he consider that the way he wanted Discourse to be used may not have tallied with the way his users wanted to use it?

That's the underlying disconnection.

I am now working on my 3rd message-transformation/integration middleware product for a large corp. In each case, the issue tracker has been stock full of problems from people who took the product, read all the manuals about what they were supposed to do, then did it their way anyway. Because, well if they can, they will.

I think the basic problem is not challenging your personal assumptions about how you want your beloved system to be used.

One problem with this is that users don't know what they want or what is easy / possible / hard.

I constantly get asked to add another "excel parser" to my in house Django app. These days I refuse (it is way too error prone) and build it directly into the web interface. Then they realize, that even with all the nice auto-complete and cut and paste features that excel comes with, clicking a few checkboxes and a submit button in a web interface is more efficient (and less error prone) than cutting and pasting the data into Exel in the first place.

I feel like responding to user complaints can help bring you closer to the nearest local maxima. But it often won't bring you beyond that unless your users are themselves UX designers or developers.

So it's useful to make your MVC as good as possible so that your users aren't forced to complain about the results of fundamental design shortcomings. That can result in more and more complex fixes, none of which should be necessary.

We in our company preferred chair-driven development - it's in case of failed estimate you are lashed out by chief's office chair.

Valve does this very well using reddit. A bad bug goes for a whole year on Dota 2, and if someone complains on /r/Dota2 it gets fixed in two days.

It's great, users fells empowered, and you get the priority list for free with reddit upvote system.

As for discourse were planning to deploy it this year at our intranet, and we've got 130.000 employees.

why not hire good UX experts who can capture these complains in product design phase?

Contact me if you need help. I'm a good UXpert.

Because even the best experts are not going to be able to identify requirements the users won't discover until they begin using the product.

You can approximate some of this feedback by observing user interaction with prototypes, but if you pay attention, you'll learn more when they start using it in production.

In my experience as a designer that changes very little - In reality it can often mean you did a whole lot of upfront work just to have it change anyway. You can put 100 UX experts in the product design phase and users will still have complaints the second you release the product.

The point of this process is to start that complaint processes as early as possible so you're not delaying the inevitable or spinning your wheels on something users don't want anyway.

Right, this is why you listen to your community, but you don't blindly do everything they tell you to do. Filtering out the best 10% of feedback -- and resisting the transformation of the car you designed into a truck because "the community demanded it" -- is the way to go.

Perhaps this is something the Gnome guys could practice?

I also practice CDD. But my experience is that few people complain. Maybe 1 out of 20. The rest silently endure a bad UI, walk away from the app without comment, or badmouth the app to their trusted friends.

Complainers have to be cultivated. Complainers can be your most valuable asset.

True. Any complaints are already the tip of an iceberg you can't see.

And the people complaining should be treated like your best friends: they cared enough to complain. Nobody else cared.

If you want to view a world where the physics of icebergs have only the tip under the water, take a look at gaming development and feedback. :)

I have one tip for you: grovel.

I've found that people don't respond unless they know that you really, really, really care. That's why most of my support emails include lines like "We're here to support you in any way we can so if you run into any problems whatsoever please let us know," or "We'll do everything we can to help you achieve X, so please don't hesitate to contact us if we can help in any way."

I've found that anything less strongly worded than that comes across as formality rather than a sincere offer.

This goes even for employees of established organizations who don't interface with customers: don't wait for your yearly appraisal; be asking for feedback, always look for ways to improve, and try not to get defensive.

1 in 20 complains??? That is actually a great number. My assumption for my products is really really not a 1/20 thing. My estimates are rather that a crash bug will be reported by 1/100. The others will just down-vote my android-app and uninstall. My estimation is based on broken design on square displays and on Thai font issues that I was made aware of far too late.

On the other hand I give instant feedback and keep a relationship to any complainers that are my happiest customers now.

I've seen this works extremely profitably in the past. Hard to argue with results.

Offtopic: I find it really tacky how this blog constantly links back to other posts of his hidden in the text.

I've been writing in a blog for over 10 years and it's really hard to not refer to posts you have written in the past. A blog is a website and that what links are for.

To be honest, I should do it even more because I find very annoying when I write about something I've already written (well, it makes sense: if it was a good idea to write about that back then, that's why it's a good idea to write about it now).

Yeah, I hate it when I'm reading a book, and it includes a list of other books in the series... So tacky.

This is a strange perspective. Any reader of the blog obviously considers the author's information or opinion on the topic useful enough to be reading the article. Isn't it likely that the reader would consider the author's writing on related topics useful?

Would you rather have an out-of-context list at the end: "If you liked this, you may also like posts X, Y, and Z"?

Yes, this can be abused in link-bait-seo-overoptimized ways, but that doesn't invalidate the utility of the practice.

Links are meant to provide additional information, or context.

Often, the author's views on related topics are both - perhaps they're less valuable if you've already read the archive, but ... really, sane site internal links are almost always some of the most useful ones there for long standing blogs.

This. Websites are supposed to distinguish internal and external links. It makes it way easier to now what kind of information we will be getting when clicking. My first thought is for a blog [1] which does it pretty well: (internal links are underlined.

[1] http://www.bortzmeyer.org/ (fr)

(I could not think of an English-speaking website with a similar differentiation right away)

Edit: just look at Wikipedia https://en.wikipedia.org/wiki/Hacker_news

Websites are supposed to distinguish internal and external links.

Supposed to according to whom? That seems to be far from a universal custom. And if it's important, the browser can do it.

Wikipedia does.

But really, the status bar hover thing in every browser since IE5 is more than enough for me to work out where a link is going to go. (Guess it's not so useful on mobile.)

The funny thing is on mobile I long press to see where a link is going to go. I do this all the time.

On Dolphin long press gives you the link at the top (usually truncated though) then a list of options like "open in new tab" "open in background" "save link" etc. It just seems important to me to see where a link is going to go for whatever reason.

Wikipedia (and mediawiki in general) distinguishes between internal wikilinks and external links.

Good point. Or I'm pretty dumb. Or both.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact