Hacker News new | past | comments | ask | show | jobs | submit login
They Might Never Tell You It’s Broken (pointersgonewild.com)
417 points by ingve on Nov 2, 2019 | hide | past | favorite | 211 comments

I saw this happen once at work. An engineering manager scheduled a meeting with an internal client to establish a better relationship between developers of an internal tool and the users of it. No specific agenda items, just general improvement of the communication. He invited everyone from both teams.

He went through some material that he thought mattered. People found that kinda useful, but then there was an open Q&A. One of the users asked when the dev team was going to fix the bug where if you enter data in one place, the system acts like it saved it but silently discards it, so you have to go elsewhere in the UI and do it there as a workaround.

The crowd reaction was interesting. One half nodded in agreement; the other half was surprised and perplexed. The bug had been around for literally years. 100% of the people on the client team were familiar with it (probably from being bitten by it personally). 0% of the dev team were familiar.

Other than don't assume and if you see something say something, I guess the moral here is that the manager's instincts were right on. That meeting needed to happen. There was some kind of communication barrier that needed to be broken down. I'm not sure, but maybe the client felt the developers were unapproachable or unreceptive or that the organization didn't view them as a priority.

-Across at least 4 versions of iOS, I’ve found that if you start the Personal Hotspot while the Wi-Fi is/was recently connected, potential Wi-Fi hotspot clients won’t be able to connect.

-My personal workaround is to turn off Personal Hotspot, turn of Wi- Fi, then turn Personal Hotspot back on. This produces a prompt that asks if I want to `Turn on Wi-Fi` or use `Bluetooth and USB only`. Selecting `Turn of Wi-Fi` resolves the issue.

-I’ve been doing this for years. I never thought much of it, until at the end of a recent haircut- My stylist was once again fumbling with her Clover card reader. Usually when this happens, I just pay cash. This time I asked if I could take a look. She explained that the Clover gets internet from her iPhone, but occasionally and inexplicably it disconnects. She said sometimes rebooting the iPhone resolves the issue.

-I showed her my work around, and she sheepishly admitted that she’d probably forget. She came to the US as a refugee from Vietnam- age and language aren’t making this any easier. I ended up using stitched together, marked up screenshots to save PDF instructions for her.

-Until last week it never occurred to me to report this “bug” to Apple. I can’t even remember how I arrived at my workaround.

Bug reporting should also be quick & easy for it to be effective. The Apple bug report process isn't the quickest. It doesn't help that minor bugs/usability issues like that never get corrected (from my experience) so I've personally given up.

I wish they'd repurpose the stupid "shake to undo" to report bugs instead and make it automatically start screen recording, allow you to reproduce the bug, edit the video to hide any PII and then upload that along with system information.

The sceptic in me says that the cost to go through all those reports, even if they were just text as opposed to video, far outweighs the potential benefit they’d get from fewer bugs in the OS. I don’t think very many people will decide to switch to Android based on a buggy iOS experience.

Surely it's a computer, and not a person, handling triage of bug reports. At Apple's scale, bug reports are easily ignorable until there's multiple.

I wonder if a new product that would be similar to UserVoice or Fixya could help for this problem.

Imagine a site that has elements from the above mentioned and also StackExchange or even the Apple Discussion boards. The input would be people reporting bugs they encounter: is it reproducible? known workarounds? maybe severity ratings. A "page views" count.

The output would then be either selling or giving away aggregate reports to vendors, like Apple. "Hey Apple, here are the most important bugs people are reporting. We suggest you fix these first."

If anyone has interest discussing this further, get in touch.

It’s not like Apple can’t use UserVoice — Microsoft does for some products. Apple just doesn’t need to care about customer feedback very much.

Apple doesn’t make it easy to report a bug. Most apps don’t. I don’t know why the industry hasn’t settled on a standard icon for “send feedback”.

I think the industry has settled on a standard practice of "if you provide a way to send feedback, that feedback point will be rapidly inundated with the communications of idiots".

Security teams are better about fixing bugs. If you can demonstrate an impact things can move faster. Of course, YMMV.

The latest version of iOS makes it so I have to repeat the turn off wifi turn off hotspot turn on hotspot toggle wifi process many times for non-ios wifi connections. My Macbook connects immediately over bluetooth which is nice but the hotspot wifi is more useful to me personally.

This is my standard procedure even for my iPad mini 4. Fun times.

> Until last week it never occurred to me to report this “bug” to Apple. I can’t even remember how I arrived at my workaround.

Pretty much no bug I've ever reported to Apple has been looked at, let alone any kind of fix.

So, I stopped reporting anything long before leaving the Apple ecosystem.

I just always assumed personal hotspots were supposed to NOT work correctly 100% of the time. It never seemed to work right for me; as in ever. I just assumed it was because they were in bed with people who make money off of your paying for more devices.

Curious, would something like this be solved with one of the new shortcut automation actions? You might be able to create a button to toggle it on and off for you if those settings are exposed.

I’ve never been able to connect my ipads to hotel wifi on demand. iPhone, mac, no problem. They popup or load the hotel page once I select the ssid.

iPad, never. Randomly, after 10-30 minutes, popup appears. A dozen Apple support pages tell me to move closer to the router that my iphone connects to. Each page has eleventy-billion “I have this problem too” with no other mitigation.

I’m convinced no Apple employees use hotels. Probably airbnb it or dont travel.

When this happens, in safari go to web address - this almost always triggers the connect page. Used to be a problem at my office for the open-guest network til we got updated wifi network hardware.

neverssl.com is set up for precisely this problem.

Isn't this now CloudFlare's IP address? Didn't they make a blog post about both hardware and software incorrectly hardcoding to do things instead of doing it properly?

I'm surprised Safari does this...

It isn’t Safari, I’ve done this from Chrome on Windows pcs as well. It seems many networks that use a page you have to read to accept terms before getting wifi enablement use, perhaps others too dunno, as a capture page. Once you’ve accepted terms the address passes thru like normal.

One example from Cisco: https://community.cisco.com/t5/wireless-mobility-documents/w...

I vaguely remember reporting bugs to apple used to be easier.

I tried bugreporter.apple.com and you need to log in as a developer. Has that always been true?

I've seen this as well. And the companies I've worked for were fairly small and had good relations across departments. Despite asking for people to report bugs and having processes to make it easy, Customer Service almost inevitably decides at some point that it's too hard or stressful or pointless to report bugs, and they stop. And they give the customers work-arounds instead of reporting the bug.

And it goes the right way for a while after a huge embarrassing bug that causes a problem with a major client and the bosses have to scold everyone, and then slowly stops again.

Unless the dev team is super responsive, and well resourced, reporting bugs can soon feel futile for support.

Then they become susceptible to internal fake news,e.g "I heard someone in Dev say they'd never fix that, it's too hard to test". Or "Dev seem to have a policy of only working on new features".

As well, support knows that many bugs go unfixed for years/months, and/or the issues that do get fixed don't seem to make sense (to them anyway).

So eventually they stop reporting or pushing for bugs.

The next stage after that is that this world view gets propagated outside the company - support (maybe subconsciously) lets customers know that there's really not much chance of this issue being fixed.

Then customers get the vibe that the product is not being maintained/ the company's priorities lie elsewhere.

In turn the customers talk about themselves and perhaps move that little bit more towards going elsewhere.

IMO it's a very dangerous situation and companies should take extremely strong measures to keep a positive mindset in the support team, whatever that takes.

Reporting bugs is an act of faith. It takes effort, and you have to believe that the effort will be rewarded. If people aren't doing it, it may be because their faith hasn't been rewarded in the past.

But, I think customer support people are also trained (somewhat rightly) to shield the rest of the organization from complaints. They view it as part of their job to be the ones who handle stuff so other people can concentrate on other things. Sometimes they just get confused and take that too far.

If you're in support, sometimes the people who maintain the product need you to run interference for them. Sometimes they need you to hold them accountable. The line can be a little non-obvious at times.

Yes, good vs bad shielding is what it all comes down to. Some examples I remember from my struggles in the past:

Working hard to remove duplicates is great, but frequent reports have to be signaled as urgent to the devs.

Then there is the tendency to only forward stuff that is somewhat reproducible, especially for unique reports. This makes entire classes harder to get through.

Sure I could find more... so yeah, many structural aspects of the job are set us up to break the link between customers and devs unless everyone is mindful.

I once worked with a pretty good customer support manager. They had two mechanisms for trying to handle that. Both were based on having the support people categorize issues. (Presumably based on error messages, some manual review, symptoms, etc.)

They'd then compile a weekly-ish report of (1) top 10 most common issues they were seeing, and (2) issues that were new this week or went up a lot from previous weeks.

That information as pretty easily digestible for the development team, and it helped us understand the size of the impact of known issues. And ones we didn't recognize, we could request more information on.

The support manager also made a point that if a new issue came up and seemed to be particularly prevalent or major, they would proactively contact dev instead of waiting for someone to read a weekly report.

can I have a meeting like that with Apple? I've got some bugs

I've even sent a few bug reports, and one of them got a request for more info.

None have been fixed though.

Years ago I submitted several bug reports for things that were critical to me. For example on my Macbook Pro, when turning the screen brightness all the way off, the screen would randomly switch to full brightness after 5 to 20 minutes. I would listen to podcasts at night and so wanted the screen off for sleeping (this was before having smart phones with speakers on them). Needless to say the screen suddenly going full brightness was extremely disruptive to my sleep. I started "patching" around it by draping a piece of a dark curtain over the screen at night. It's ridiculous that such a premium (meaning expensive) company who completely owns the entire stack (all hardware and software) would have such a terrible bug, and even worse that they ignored it for years.

> Most of the time, if it doesn’t work, they won’t report the problem. There are a few reasons why this might be:

There is one more reason the author didn't mention as it doesn't apply to his case: people don't want to sign-up for new accounts in every bug tracker just to report a single bug. If it's GitHub and the issues tracker is enabled - it's ok for most of the people as they probably have a GitHub account already (but I still believe there are people who don't). If it's a BugZilla instance (which means you have to sign-up), let alone a mailing list (which most of the people don't even know how to use) - that's a serious obstacle for bug reports. If you want all the bugs to be reported - declare this a visible way (and promise you won't respond like GtFO and RTFM) and make it easy, e.g. by accepting bug reports via plain e-mail (a couple of alternative ways like Twitter and/or Reddit won't hurt).

I've found bugs with Xubuntu and started trying to report them, but I just gave up after 15 minutes since the process of reporting the bug was more of an inconvenience than the bug itself

Whenever I've found any kind of bug in Xubuntu or Ubuntu I would just rise a question about how to fix that on askubuntu.com. If it was my fault rather than a bug the folks would help me, if it was a bug they would confirm that. Some times I would even post a question and answer it myself immediately in cases when I've found a problem and a way to fix/workaround it myself. This way I make sure I'm not the only to know about the bug, the bug and the fact somebody is affected is known to the public. I just hope Canonical engineers actually read the site occasionally.

Weirdly enough someone reported a Ubuntu bug I had and I was able to just respond to the email chain with more useful information as to the cause and that was that. I think mailing lists are powerful they just need to be clearer on how to use them and have some tutorial links towards the bottom. Dont ask me how you start a new email chain for a new bug though. I wouldn't know. To be fair I dont remember how I responded initially.

I agree though sometimes I dont bother reporting bugs if its not on GitHub or a system I am already registered for.

Yet another reason people might not report bugs is there can be an attitude of hostility towards reporting bugs by open source maintainers whose priorities don’t align with helping their users.

It’s exceedingly common for open source maintainers to reply on bug threads with disingenuous reasons and mental gymnastics to avoid calling a bug a bug, and try to pass it off as an obscurely defined feature request or grumble that whatever use case is hitting the bug “shouldn’t” be supported or would require structural changes to the design (which is what caused the bug) that make it the user’s fault for wanting something that would require scrambling someone else’s arbitrary design choices.

A big example from my recent memory is bugs with various cloud storage APIs that result from blob storage not behaving like a real file system, but user semantics still relying on path delimiters that group things into a hierarchy of directories.

People get disinfranchised because you’ll be given some parochial, nonsense dismissal saying that the backend being undifferentiated blobs somehow means the user shouldn’t rely on filesystem-like abstractions, instead of the library maintainers just doing the work and adding indexing and making the API functions automatically deal with delimiters & key prefixes so that it “just works” like a set of folders to the user (who doesn’t and shouldn’t care about an unrelated implementation detail like blob storage).

This is just an example, but illustrates a wider point. You often feel like what’s the point of reporting this bug? I’ll just be given some empty dismissive answer about how it won’t get solved and I should philosophically refactor my entire use case from first principles so as to not need what I need.

What you describe sounds very much like a feature request, and one that would add overhead compared to the existing implementation.

> If it's GitHub and the issues tracker is enabled - it's ok for most of the people as they probably have a GitHub account already (but I still believe there are people who don't).

80% of 'creators' on the internet have never heard of GitHub [0,1]. If we use 1 repo per person, we get about 1M people using git. Assume the 1-9-90 ratio, and you get about 100M people that have heard of git or clicked on something like it even once. About 5B people use the internet today. Say the same 1-9-90 ratio applies to users of the internet. You get ~500M 'creators' on the internet. Assume all of the people that have used git are 'creators' on the internet. 100M out of 500M is ~20% of the 'creators'. Meaning that if a random 'creator' is selected, there is a 1/5 chance they have used git in addition to using whatever thing they are currently using. This is a quick and dirty run-down neglecting a lot of stuff (english language, age ranges, sex differences, firewalls, etc) and smoothing over a lot of stuff too (1 repo/person, age ranges, 1-9-90 ratio, etc). But I hope it demonstrates that it is not anywhere close to safe to assume that anyone paying you for a product has any idea what GitHub is.

[0] https://www.openhub.net/repositories/compare.

[1] https://www.allianttechnology.com/how-many-people-use-the-in...

If anyone is reading this from Apple or Jetbrains, take notes. Your bug trackers are obnoxious to log into and use.

Jetbrains had absolutely fantastic support when I first started using their stuff 10+ years ago. it's kind of sad that the support has gone downhill since then, but on the other hand I don't run into problems as often either.

The Jetbrains buck tracker is not that bad and it is worth registering at because the fact you have found a bug in a JetBrains product suggests you probably are in using it seriously enough to find more soon (there are many), and there are no real alternatives to Idea anyway so you are doomed to keep using it if a text editor or a less smart IDE is not enough for you.

There are plenty of alternatives (I primarily use CLion).

The issue is more that it takes more time to log in, load the issue tracker, search to see if it's already there, create a new issue, wait around for a response email, record/send the logs, and then wait to have them tell me they couldn't reproduce the bug.

Just like with Apple I stopped reporting bugs to JetBrains, because with most products I can send an email with reproducible steps and that's the end of it. With them it's a whole process.

And JetBrains, why is the default notification for tracking a bug/feature request sending an email for every person who upvotes it?

> why is the default notification for tracking a bug/feature request sending an email for every person who upvotes it?

This is incredibly frustrating. I commented / followed a bug a few years ago that they clearly will never fix (start-up stealing focus on Linux), and have received tens, maybe hundreds of emails from upvotes and "me too" comments.

Fix, Wontfix, anything from the team, I'm in, but wtf would I ever need to know about upvotes?! You could send me a monthly stat email on my bugs. That would be great, actually. It's weird to see things that annoying from a team that makes such a great software suite. Almost as annoying as having my IDE steal focus upon start-up.

The one for me was a Ninja exporter for CLion. Ironically, I found out that the feature was recently added because I saw a post from them on LinkedIn. Not the bug tracker.

Apple completely redid their bug reporter recently, and all you need is an Apple ID to log in.

Hmmm... I found this and it just needs an email address:


for example:


It is stupid though. It says:

> If you provide your email address, you agree that we may contact you to better understand the comments you submitted.

which makes it seem like it is optional, but then below, the email address field is marked "required"

This gets at a few issues:

- Account-based authentication for numerous systems has reached a point of increasingly negative user returns. Experian (citing unspecified research, probably a 2015 Dashlane study) notes that the typical user had about 100 accounts -- 130 for the US, growing at 15% per year, or alternatively, doubling every 5 years.


- Crud reports are crud. I've got an issue with an Android app I use heavily (Pocket), which has been crashing with exceptionally annoying frequency (30x/week by one Android report, which suggested uninstalling the app). I've taken to simply reporting those, for various reasons without the ability to report this as the Pocket user, simply noting "app has crashed again". Pocket have finally responded (after weeks). The problem itself has been ongoing for years.

- There should be an analogue to James C. Scott's Seeing Like a State for software / SaaS vendors. Call it "seeing like a software developer". The power of software is that a small group (and frequently a single individual) can create a tool that is useful for and used by many -- hundreds, thousands, millions, billions. The weakness is that gathering and assimilating useful feedback is extraordinarily difficult. Most feedback channels can be overwhelmed by a few high-volume (and often not-particularly-useful) actors. Quality information is difficult to acquire. Sorting useful from non-useful information is expensive.


- Grouping similar bug reports is a challenge.

- Numerous projects fail to prioritise interests of advanced users. Something I've termed "the tyranny of the minimum viable user". Given the vast range of user skill levels, and the overwhelming domination of low-skill users, this has profound implications for the future of software, computers, and systems.

At some point in time I felt the need to make an anonymous Github account. It's quite annoying to know that people can google your name and see what you did in the last few years.

Realistically that should be the default for everything on the internet. Everything I do online is under a persona, I try to sign up for as little as possible under my own name, and email address. Subsequently the email addresses I use to sign up for most accounts have been pwned (on https://haveibeenpwned.com/), but my real email address is fine.

That is a frustrating truth.

At least you maintained separate email addresses so compromises are sort of firewalled.

I've found that the CloudFlare VPN bug reporter is the most user-friendly.

Click on the bug icon. It takes a screenshot. You can then choose not to send the screenshot and add information about the problem before you send the report.

I still haven't been able to get the CloudFlare VPN to work, but they're at least super nice and very responsive to bug reports.

Safari used to have a bug icon. It really should have one again.

I accept bug reports both via email and via Github.

I do get bug reports via email that nobody had previously reported to Github. So it's definitely a win.

But bug reports via email are still rare.

I once shipped a version of my app that had a really serious issue (text editor crashed whenever you pressed backspace). From automatic crash reports, I know that hundreds of people were affected. My app was practically useless.

But I only got a handful of bug reports (I think it was less than 5 customers who complained). The vast majority of users just accepted that there is a bug and did not realise that they could email me and I would fix the issue within a few hours)

Folks should allow quick anonymous bug reports as well.

github requires an account.

email requires disclosure of an email address.

> they probably have a GitHub account already (but I still believe there are people who don't)

Well of course there are people who don't; most of the users don't. Basically only a part of the dev community have Github accounts, that's far from representative of the userbase.

> a mailing list (which most of the people don't even know how to use)

My biggest problem with a mailing list is -- I have no idea how to use it but avoid spam.

But obviously they are still used, so there must be some way? How do others solve that problem?

Requiring an account to report a bug is a bug.

Any barrier to bug reporting is a footshoot of the highest order.

I ran into this a few months ago. To show off my StyleGAN-generated anime faces, I made https://www.thiswaifudoesnotexist.net/ to load random samples. It went viral in China. After about 450,000 unique people visited (according to my Google Analytics), with the overwhelming majority being on mobile, I thought I'd better double-check how it looked in mobile as opposed to just desktop to see if it could be improved. Turned out it could - it was almost totally unusable because the image wasn't resized appropriately etc. Not a single person pinged me about it.

It reminds me of something Neal Stephenson noted in "In The Beginning was the Commandline": one of Debian's advantages was that it had a open public bug database (still not universal! eg Apple), and you could very easily and conveniently file a bug with 'reportbug' from your terminal.

Perhaps a solution could be to have a button that simply says “something is broken,” and while it could open up a form for a user to submit a brief error report, just the click alone could send fingerprint information about the browser, so you could determine from the fingerprinting that most issues happened on mobile, from China, etc.

That way if a user doesn’t end up submitting useful information, or it’s in an unfamiliar language, or if they don’t even submit the form itself, you still have data to work with.

That's an excellent idea! Call it "drive-by bug report"? Or "casual bug report"?

I am now thinking how to combine the two channels - one is the regular support channel where everything gets tracked and answered (we put a lot of work into it), and one is for drive-by reports (so we just collect them and make statistical inference but don't work hard on each incident). Most of my users are logged in so I know who they are.

How about this:

1. Is something broken on this page? [button]

2. [button clicked] We've made a note of your report! We analyze all such reports periodically. Would you like to expedite this particular issue? [yes/no button]

3a. Form abandoned, no further UI action.

3b. [No] Ok, we will treat this as a low-priority issue. If you have any details to share please put them here (optional field).

3c. [Yes] Ok, we will treat this as a high priority issue. Once you fill out this form someone will get in touch with you asap. (What were you trying to do) (What was the desire outcome) (Is it something that used to work but no longer does) (...).

So 1-2-(3a|3b) is the casual path, and 1-2-3c is the firm path...


Adding a Do Not Send button to the form would be nice so that you could choose not to send back telemetry if you care about that type of thing and you accidentally hit the button.

From 2, ‘Would you like to add any comments or further information?’

Have a text field, a continue and an exit button. From the users perspective, leave it there. A link to the bug tracker could be added.

It's not just more information. The "Expedite" choice both requests more details and puts the user directly into support queue.

Reducing friction for reporting bugs is key. In Slack for example, you can report bugs via slash-command: "/feedback my X breaks when I do Y" opens a ticket and subscribes you to it via email.

Who's going to parse all that stuff? Remember we're talking about unpaid open source software.

I have a label in my Gmail with tens of thousands of unread autogenerated email bug reports from a long abandoned Windows Phone app. Before it was abandoned they were useful enough without much more parsing than can be applied with a gmail filter

SpaCy is pretty good for stuff like that.

The user still needs to consent to an agreement with a list of all collected data for this to be legal. Otherwise, this is likely a violation of the user's privacy.

Fingerprinting is done all the time with zero notification or repercussions.[1] Still a violation of privacy, but perhaps if it only happened for a bug report, it could be opted out on with a button on the form.

[1] https://www.washingtonpost.com/technology/2019/10/31/think-y...

Slightly off-topic, but I never thought I'd come across the guy behind that site in a HN thread - especially since I just found it today. Baader-Meinhof, I guess.

One of the smartest things I think the rust project has done, is made it so that when the compiler hits an unexpected error it tells you unambiguously that it's a bug, and asks you to report it. It avoids any question of "well maybe I screwed up, and the compiler did the right thing by crashing". Sure, you might have screwed up, but so did the compiler even if just by printing the wrong message.

    note: the compiler unexpectedly panicked. this is a bug.
    note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

GCC has had this since forever:

cc: internal compiler error: Segmentation fault (program cc1) Please submit a full bug report, with preprocessed source if appropriate. See <URL> for instructions.

(the URL may be customized by the distro)

Still I would never file a bug report to GCC. Hostile community, hostile bug tracker, hard to reduce test cases to an acceptable state. clang is way better in this.

Honestly this is an example of exactly what the OP is saying is bad behavior. The link to actually create a bug report doesn't even show up til the 5th paragraph and clicking on that link just takes you to a github sign-in page.

It may not be optimally helpful, but it's one of the articles's recommendations:

> So, what can you do? You can encourage people to report bugs. I visibly write in my GitHub README that reporting bugs is encouraged and welcome: “Please tell me if something is wrong, you’re helping me make this project better.”

Nothing in the post is about frictionless report submission, mostly observation that if something is broken most users won't bother reporting it. If the suggestion has been made that bug reports are welcome, that might help, and a good place to make that suggestion is in an error message, especially for a development tool (though it is also in the main README).

And the link to …/CONTRIBUTING.md#bug-reports doesn't actually work, because the relevant heading has id=user-content-bug-reports" rather than id="bug-reports".

Are you sure about that? I just tested to be sure, and it works for me (with firefox).

I agree that the id in the html is "user-content-bug-reports", but for some reason the link works anyways. Further if you hover over the heading github displays a link icon to its left, and it points at #bug-reports.

Hmm, you're right; there's an href="#bug-reports" that is somehow ineffective on my other machine.

Npm tries, but there’s often such a wall of text that people can’t sort it out and ask me instead.

I feel I've hit an error in npm regularly that said it was a bug in the tool, for it still to end up my mistake. While you may well have gotten the tool into an unsupported state, you were never supposed to get there. That makes these "it's a bug" messages less useful to me.

You still found a bug - the error claimed it was a bug, but it wasn't!

The point of technical error messages, imo, is to tell the developer what they did wrong to get into this state. They can't be allowed to be wrong, otherwise developers stop trusting them (this bites us a lot - we get dozens of incidents along the line of "error says very specific thing related to my scenario is configured wrong - is it actually?")

I actually had to plan this problem into my project. I run https://buttplug.io, an open source library for controlling sex toys. Due to the context of usage for the library, a lot of our users will never want anyone to know they’re using our software, much less reporting bugs on it.

This means I have to test early and often, and rely on people who are open enough to even discuss usage in the first place in order to get bug reports. This is on top of the fact that we support over 100 pieces of hardware, of which I have maybe a third. It’s a odd position to be in.

using a .io for this is genius lol

For small projects, sure. The probability that a bug will be reported is

    number of users * prominence of bug * the "reportability easiness" reported in your article
I have an open-source project with 100k active users, and I mostly handle issues myself, so I require the whole 9 yards of signing up for a GitHub account (reduces the number of lazy issues) and filling out ~10 questions in an issue template for both bug reports and feature requests. It works very well for increasing the average quality of bug reports and filtering duplicate, nonsensical, emotional, and information-lacking bug reports.

If you're on the other side of the spectrum and are just begging for people to report bugs, then you should make it as simple as possible by allowing bug reports from all mediums (GitHub, email, forum, Twitter, phone call, whatever) and engage in the repetitive conversation it requires ("Can you send a screenshot?", "What does the error message say?", "Can you send your log.txt file?", etc.) But once the number of bug reports is more than you want to handle for the project's salary, you should implement methods to send users through a standard pipeline to avoid asking these questions every time.

Can you provide an example of your issue template?

A sentence suggesting a way to upload the file (maybe creating a Github gist with it?) could streamline this a bit for the submitter maybe.

Perhaps, but in the GitHub issue form, it says "Attach files by dragging & dropping, selecting or pasting them." so I'd just be mirroring the UI text.

TIL, I thought that only worked for images. nevermind...

There's some expansion that can be done on that third option. Even if I'm not very invested in a project, I'd be happy to mention a bug if it were easy to do. But my experience has been that, if I report the bug, I'm going to start getting asked for details about my system, and asked to try a bunch of things to try and sort out the reproduction steps, and possibly even have maintainers getting frustrated with me if I don't want to fix the problem myself.

Consequently, my unofficial policy is that, for an open source project, I'm generally not going to bother to report a bug unless I'm also planning to fix it myself.

The author is right, though, being super up-front about welcoming issues might help break the ice a little.

> There's some expansion that can be done on that third option. Even if I'm not very invested in a project, I'd be happy to mention a bug if it were easy to do. But my experience has been that, if I report the bug, I'm going to start getting asked for details about my system, and asked to try a bunch of things to try and sort out the reproduction steps, and possibly even have maintainers getting frustrated with me if I don't want to fix the problem myself.

A quality bug report should include system details and detailed reproduction steps. Expecting "something didn't work for me" to leat to a fix is unreasonable, especially when you expect unpaid vonlunteers to do the work.

And there, I think, is the rub. So many open source projects see bug reports as something of a zero sum game, where simply mentioning that something isn't working quite right is seen as being a sponge.

I think that's a sad culture, but I go along with it. Hence the policy of not even bothering to report bugs I'm not prepared to fix. The reason that's sad is that it creates a culture where, as the author of TFA describes, critical bugs that severely hinder the project's popularity by rendering it unusable to many users might go undetected by the project's maintainer(s) for a very long time. Because it creates a situation where potentially every single person who encounters the bug will decide that the path of least resistance is to quietly go find something else to do without bothering to mention that there's a problem.

It's almost like a sort of cultural prisoner's dilemma. If we're all charitable and try to give each other the benefit of the doubt, we'd probably collectively be a lot happier and more productive. But the presence of a significant subset of people who take a more stand-offish attitude means that everyone's optimal strategy is to be a little bit stand-offish, for better or for worse.

I have an odd hobby: I keep a close eye out for people with busted brake/running/reverse lights so I can alert them.

One day I waited for a couple of minutes in a parking lot so I could tell someone his brake light looked like it was out.

He turned to his co-worker who had followed him into the lot to confirm. His friend told him it had been out for months and he just assumed he knew.

I don’t know what the takeaway is, but that mindset baffles me.

Interesting. My reverse light isn't working and I only discovered this because I was reversing at night in a dark spot. I tend to do most of my driving during the day and most areas have street lighting or I park in parking lots with lights. I don't know how long the light has been out. Perhaps you could add reverse lights to your odd hobby :-).

I do. Just told someone today theirs was out.

Not unlike this guy, Chris Sewell, profiled in an old This American Life episode they happened to replay today. Homeless guy who became adept at alerting people and institutions that the government owes them money:


World needs more people like you. Keep up the good work!

Why can't cars tell their their own lights are burned out? Seems like a simple check of "is the circuit open or closed right now"?

Some do. No idea why it’s such a widespread problem other than “the government hasn’t forced me to implement this yet.”

Most modern cars will. Like you said, it's not hard to detect.

This is why client & server side telemetry is so valuable.

People on this forum like to get all up in arms about it, but having it makes for a much higher quality end user experience, as you can see all the problems people run into. Not just errors, but bad ux flows, That cause people to bail etc. It’s really been invaluable for all the products I’ve worked on in the last 10 years.

Because yeah, most people won’t tell you.

Companies used to do this with focus groups and beta testers. Now they outsource the work to their customers, at a high privacy cost.

If client-side telemetry is benign, then it should be fully removable for people who don't want it (Firefox good, Windows 10 bad, web session replay terrible).

> Companies used to do this with focus groups and beta testers.

Even with beta testing, you still run into serious bugs that are completely unknown until the code hits production. There's far more diversity in the runtime environment among non-enthusiast regular users than you'll ever encounter in dedicated test efforts.

I've used products with insane amounts of telemetry and the quality wasn't much better. I guess Windows 10 is the proof of that, as its quality has severely degraded from Windows 7 despite the increase in telemetry.

Somehow I feel like it's almost intentional to drive the UX to a "search for everything" model.

It's almost like telemetry encourages optimising for idiots because they are the majority of the userbase. The problem is by removing features they remove the opportunities for the non-tech-savvy people to learn and become power users, so it's a feedback loop.

More generally, it can lead to UI decisions made by clueless people misinterpreting the telemetry data, instead of opinionated leadership that decides how the classic google.com page or the iPhone should look like.

Well, this os probably not a bad idea where unskilled users are concerned. This way, They don't have to learn domain vocabulary and arcane incantations to make a strangely shaped box do their bidding. I can totally understand where this is coming fron.

However, that god awful search in Windows 10 is the exact opposite of this ideal. It seems to get worse and more inconsistent in every iteration. I hate it in particular when a web search result gets displayed instead of my locally installed app that I spelled out and when I absentmindedly hit enter before reading the small print in the result, a new Edge windiw greets me. It's not even my default browser. How's that for bugs?

My problem is less the existence of telemetry (some negatives were already mentioned), but that this data is collected into gigantic silos describing the totality of my digital life and shared/sold.

Going with a self-hosted solution instead of GA goes a long way to gain my acceptance.

That's because people here value their privacy far more.

It is not. I was about to write "I hope someone doesn't think telemetry is the answer to this", thankfully it took quite a while - because it really isn't.

The answer to this would simply be to test it on OS X. Or just state that OS X isn't supported - or just do nothing. All three are acceptable for a project like this.

Telemetry is not.

Contrary, not even HN users care if their web searches, sites they visit, and other programs they have installed, is shared. Either that or they didn't read the EULA and privacy policy of the tools they are using.

Its not just open source; a lot of people won't tell you about bugs in commercial projects or internal systems either. You need really good telemetry/monitoring infrastructure and proactively find the bugs yourself. But it isn't really practical to apply that strategy in open source projects.

Out of all the remote offices, 90% of the calls and complaints came from one guy, we'll call him Larry. Larry was one of those users who knew 'just enough to be dangerous', he was a power user who often bragged about going to 'computer school'.

All of the help desk techs thought of him as a big thorn in their side, to them it seemed like he called about all the most insignificant issues, and it was almost like he intentionally tried to break things.

This crept over to the developers and management, who would ridicule the guy. But what they didn't know was that he was probably their best tester! All of the other remote offices didn't care that things didn't work right, and assumed the bugs were already known and were never going to be fixed.

I truly felt bad for the guy, he was just trying to get his job done, but ended up being the victim of both bad software, and bad technical support.

Yep, done IT for decades and one thing you find out is in office groups one person becomes the spokesperson for software problems. Other people report their problems to that one person who contacts technical support.

At my moms job she was the leiasion for printer problems. You'd think she broke the printer a hundred times, but instead shes the only person that would call support when they happened.

Every company has a over enthusiastic “Larry” who went to computer school.

I wish I had had one at last company.

Hah. We've been working on an internal project for a couple of years, with several internal projects using our API.

Yet the feedback has been almost non-existent. We just have to assume it all works as intended, but we know that's too good to be true. When we grab people they say "oh yeah, I was going to report a bug but now I can't remember" or we'll see that people just stop using it because of some game-breaking bug, which they never report. Some people even say "don't use it, it's broken" and just assume we magically know about the bugs.

The worst part is many of these projects are explicitly made to help test the API, and help improve it. Yet they don't report anything.

I would go further. It's worse in commercial projects. Most of the time, no one will be fired if you report a bug in an open source project.

But, and this is the important part: depending on how the company is structured, reporting a bug in a closed source project will result in unplanned work. If the company takes a "everyone is responsible for all bugs, we don't assign blame" then it is likely someone will report it.

If the company, whether explicitly or implicitly, is communicating that if you report the bug, you fix it, then there is a good chance you will not report it, if you own it.

If you own something, you can be fired if you break it. So there is a perverse incentive to not report your own bugs, especially if you inherit a mess.

Bugs are inevitable in software, the best anyone can do is try to minimize them. I don’t know any companies that would fire someone for introducing a regression unless it was from malice, or a level of carelessness indistinguishable from malice. Maybe this is different in other countries or industries? If you ever find yourself working for a company that would fire someone for that, definitely find a new employer ASAP, because that company is far more interested in meaningless ass-covering by management than they are in great engineering or shipping great products.

Unfortunately some of live in regions where getting a new job isn't an option. The parent post raises a very valid point which I have observed in companies whose core business isn't IT but have an IT department. You end up having a fair number of non-technical managers who don't understand the intricacies of software development They don't understand that bugs do not mean you didn't test enough or you are a bad developer. Bugs delay projects or require resources and reporting them is likely to make someone look bad so there tends to be push back from IT. It isn't right but it is what it is. Its valuable to be aware of this prepare accordingly.

That was really what I was wondering about, it sounded strange to me but that's the bubble I'm in. Different industries or different countries might have very different norms surrounding this kind of thing.

In my bubble, you have to be very careful about hiring because it can be very difficult to fire anyone at a large company. Or it just ends up being such a long, awkward, multi-stage process, that even objectively terrible engineers will get transferred elsewhere within a large company rather than get fired, because it's that much easier for the manager who wants to get rid of them.

Some of it might be a cultural affinity for giving people second chances, but I think a lot of it is just the hoops you have to jump through to avoid the possibility of a future wrongful termination lawsuit if you fire somebody. But I've heard that it can be very different elsewhere, in different industries or countries. I bet it has a lot to do with the legal system too, like whether or not there's a framework of worker protection laws in place.

I once discovered and fixed a bug in a commercial ordering system which prevented one of the addon-products to be ordered. I checked version control - the bug must have been there for months. Nobody noticed that the product could not be ordered.

It probably wasn't the most popular product to begin with, but I am still amazed that nobody even noticed the drop in sales.

They often have no way to do so. Your CTO, who signed the deal, will get emailed some support information. They'll promptly archive or delete the email.

Worse still is putting your manuals behind a login. I know some companies believe they're valuable secrets for some reason, but most of your customers will simply never be able to log in to get them.

There are many industries where the workflow espoused in the manual IS the secret sauce, and, once replicated, can be replaced by a much cheaper product.

In these cases, you will never recoup your NRE.

Except, it's easy for competitors to get a copy if they care to. They can ask around for a copy, or just buy your product and get a login.

Oracle has sued competitors for handing out copies of the manual, suggesting requiring a login isn't much of a barrier: https://www.bizjournals.com/sanjose/news/2013/06/18/oracle-w...

If it's possible, I think it's really useful to watch the users using the software for a while.

I agree completely. Just observing users stumbling through what you had thought was a blindingly-obvious, low-friction workflow is an eye-opener.

Sometimes, you have used your own code way too much to judge effectiveness.

Having a no-agenda meeting with users is also a great idea, but it is important to set scope on feature requests before it snowballs into a request for the moon.

Internal systems have their own, complex, reporting failure modes. Generally that is mixture of political concerns and internal deprioritization of tools (a cost center) vis-a-vis projects that directly produce revenue.

After a crash, the software often like to send a crash report. So you get just these errors. But until now, I never saw a "Report a bug" button somewhere in a menu.

I have snuck in user studies at every place I’ve worked, just by watching coworkers fat finger things or hit the wrong button or twist in the wind.

You have to learn to really watch people when they’re using your team’s code. It’s so much clearer than listening to them bitch about the three things that triggered them.

Thank you for linking this article.

> Nobody wants to look stupid or get told to RTFM, and so, they choose silence.

That's a great insight.

I wonder how many users are lost to "setup / entry point problems" in projects.

These problems may not even be "bugs" but issues such as difficulties in configuration, too many steps, something not entirely straight forward.

One thing that comes to mind is analytics to see conversion funnels, that may work for websites but not so much for tools or libraries.

It's a great motivation for testing, and perhaps user studies. I would be willing to pay a few bucks for someone to try my project and tell me what issues they ran into. Unfiltered.

I've been that person before. If I install something and it crashes or segfaults before I can even use it, the impression I get is "I guess this isn't ready for use right now" - more often than not I just move on at that point.

It's different of course if it's a large well-known project, but there are enough smaller projects that really aren't ready that it often doesn't feel worth it to report things like that.

Analytics inside libraries is the sort of thing that if HN finds out about it you will get death threats.

Even if you want to help not everyone wants you looking over their shoulder.

Yes, that is unfortunately the case...

I guess that is why some companies have open beta versions where in the eula you agree if you use this version you will be analysed. See Microsoft Windows insiders edition, Jetbrains, etc.

They’re fine, just as long as they are opt-in.

That’s your opinion but it’s not universally shared and the people who take more extreme positions also, in my experience, tend to be the most vocal about any perceived abuse. All you need are one or two of them to spoil you on open source for awhile.

It is not an extreme position to think that software spying on its users without their consent is rude and inappropriate.

It's fine to collect telemetry, just as long as the user agrees to permit it. Otherwise, your software is malware: it benefits you, at the expense of the user's privacy.

The problem is how you define those terms: “spying” has a strong value judgement and opinions vary widely, as does the threshold for what people believe constitutes informed consent.

that's a feature not a bug.

If you go down and try pin those terms, you will get "language layers" that will try to go as close to the line as possible and when they cross it argue the definitions.

Be as open and forthcoming as possible.

I think you meant “language lawyer” or, given the context, just “lawyer” but it’s not really what I was talking about: my point was just that projects can define what they collect, what they do with it, and how to control it, and they will still get angry rants from strangers on the internet. Analytics is a really useful tool but it is also a lightning rod for criticism and conspiracy theories.

“They didn’t say no” isn’t consent.

I think you should stop swinging at that straw man long enough to read what I actually wrote.

Your perspective depends a lot on where you're looking from. Small projects have no need for opt-out ("spying") telemetry. Explicitly reported bugs and telemetry from self-selected users will provide more than enough stuff to work on, and there really isn't that large a space for bugs to hide in anyway.

If you have 100 million users, depending in the bias inherent in opt-in reporting is deadly. You need the statistical correlations for a lot of things to be tractable. You can't afford to staff your development organization as a percentage of your user base, not if you have any competition at all. You'll have to get comfortable with some things that feel like software malpractice at smaller scales - such as accepting that bad RAM really is a significant source of problems when you have millions of users, and you'd better get a feel for what the resulting problems look like.

This is soooo true. Launched a software product at my company, of course it was supported 3 divisions over. Very few end users reported any bugs, which then very few of those were passed along to our division, of which management/business decided few of those were worth fixing. Once we added proper telemetry it was shocking how many errors end users were seeing.

Why would I report every random bug when it’ll just get closed in five years as WONTFIX? Let’s be realistic: most FOSS projects are run by people who already have a lot to do, and at least from what I’ve seen, the ability to get any specific bug fixed is pretty low. So that’s why I don’t bother.

Similar experience here, only with an addition from the other side: I'm subscribed to some issues, and get seriously irritated by the number of people who barge in with “This is a show-stopper for our company! We're not migrating from ProductX until this is fixed! Surely it would take just a bit of time to fix this!!!” Very often, such attitude is displayed on feature requests that barely qualify above ‘cosmetic.’ Even on software for programmers like Intellij Idea or Node.js, people whip out the ‘it's an easy job’ argument, or post endless ‘me too’s while failing to comprehend differences in their use-cases.

I see quite a few projects that get steady development, but have hundreds of open issues, of which most are probably feature requests. In my experience, having the tracker flooded with thoughts for features that it would be nice to have someday, is completely useless and probably actively toxic. Ideas are cheap, while implementers' time is finite. Ideas are best dumped into an ‘inbox’ list―most of them will never surface from there anyway.

As a result, I almost completely stopped posting issues, except for most egregious bugs or when a feature would be a one-line change. As a rule, I now prefer posting pull requests, which of course is a lot harder―but helps keeping things in perspective.

This is a problem with expectations. If we treat a bug report as a gift, then the only thing we should expect is that the recipient says "thank you", not that they will ever use it.

But, somehow both sides end up treating it as a work request, and that causes bad feelings.

Before I decide to use an open source library, I check bug reports and see attitude of the library maintainer before I use it. If there is a lot of WONTFIX or arrogant attitude, I simple stay away from such a library.

Absolutely right I think. Getting feedback from users is surprisingly hard and it's important to show gratitude for it even when it's bad.

I've found it quite hard to get feedback even when I push out beta releases of an application specifically for gathering it. I think people feel that if it isn't working, that's just "because it's a beta" and they'll wait and see what the release is like before worrying about it.

At work I actively reach out to people to ask how things are working, to the point that I have a weekly scheduled check-in with each group to ask if they've encountered anything wrong or that they feel is just too complex for what the task is.

Once people are comfortable with both complaining and having things actually get fixed, combined with I lucked out with department managers who are competent and I can trust what they say and their assessments, this process works extremely well.

It's really the small things that make the big differences. We added a feature, but didn't add a link in the admin to the report for the customer, so what should have been clicking a single link became a 10step process. Having those lines of communication established, they messaged me and I got the link added quickly. Sure, sometimes things get queued, but a lot of their asks are small things that make their lives so much better.

That is such a great way of operating a software company! I wish more people would get on board with this

I tend to assume anyone who reports a bug is likely about the 10th person to experience the issue if it’s a big issue. Probably 100th or more if it’s a more minor bug. You can sometimes see this for yourself if you have logs that report 404/500 or other error and compare it to the bug reports you get - of course often with open source you don’t have that info.

As a user it’s more effort than it’s worth to report bugs and when you do people argue that it’s not a bug or take a year to fix it, which is then only in some dev version you won’t see for years. This includes stuff I pay for like Spotify.

Assuming that users, customers, or employers will proactively tell you when there is a problem is the source of a lot of poor quality products, services and work environments.

Many times that I've stayed in hotels there has been some small thing wrong, like a blown lightbulb, or the toilet flush doesn't work well. I almost always forget to mention it because its a small thing, but it impacts my opinion of the hotel. These things fester until they become more serious.

I've used software and services with bugs and not reported them because I assumed the surely they must know of this glaring error, but apparently nope, the didnt.

And I've known work environments where people just left rather than try and improve things or air their grievances (including me). Previous attempts were met with, at best dysfunction/ineffectualness, and at worst hostility, to the point that people stop bothering.

The author discovered this once he started a chat room. That is also a lesson: create an easy, low stakes way to let people connect with you (a slack or discord being a great example in 2019) and you’ll be more likely to get these reports. Partially because they will be less effort for people to write a full big report (mod joining the server) but more importantly people may feel they’re more likely to get immediate help if such a channel exists. (Which may be a burden for larger projects, but in exchange for that burden you’ll get goodwill and the reports themselves.)

Monitoring user conversations about your product is absolutely critical, IMHO. Even if you don't directly interact with them, just seeing their chatter can be invaluable. Often times you, as the developer, will spot clues among their rambling that point to actual fixable issues. Meanwhile, these same clues will get completely filtered out and ignored if there's any sort of "user support layer" acting as an intermediary.

Sure, there's a lot of chaff to filter through. As such, its okay if you don't want to be the one interacting directly with the users 99% of the time. But simply paying regular attention to what they're saying is worthwhile.

"I’d never tested it on MacOS myself, but I couldn’t see any reason why it wouldn’t work. I told this person that the problem was surely on their end."

Her reaction to the report says all you need to know about why people don’t report things.

It's a self-reinforcing feedback loop of communication failure: users don't report even the most obvious problem; maintainer assumes that if such a problem were a general issue for all users it would have been reported by any of the previous 100 users - but it hasn't been so the one user that actually takes the time to report it must be holding it wrong.

Also, "I couldn't see any reason why it wouldn't work" seems to me like a very dangerous assumption.

I personally try my hardest not to declare that X works on/with Y until I've seen it with my own eyes. Not 2 computing environments are the same, and everything receives updates all the time.

* her

I tried to build Ruby 2.7.0-preview2 from a source download 10 days ago.


Build failed.

Spent 5 or 10 minutes trying to figure out how to report the problem, and then gave up.

Googling “ruby report a bug” gives me a How To Report Bugs in Ruby page, given that it’s a preview release wouldn’t it make sense to use those instructions to report? Or the instructions didn’t work?

It’s usually a hassle. You have to be very clear about what you were doing, what your environment is, what the expected result was, what actually happened, and how to reproduce it. And even your comment doesn’t link directly to the page, so googling is yet another step.

Steps add up. I just think the process shouldn’t be so expensive. Goodwill is nice, but people aren’t getting paid for it. (If only someone could figure out how to convert bug reports into money. It’s possible; bug bounties tend to be limited to security, but they’re successful in general.)

"You must register your account and activate it via an email before reporting issues."

Two other possible reasons, both having to do with how users view software developers:

(1) They are naive enough to believe programmers have a better understanding of their code than they really do. It never even occurred to them that it would be possible that you don't know. Surely people who make software know what's going on with it! How complicated can it get where you wouldn't know that? (But we programmers know otherwise.)

(2) At the opposite end of the spectrum, they are cynical enough to believe that you could fix it but just don't want to.

Basically, there is a story they have written in their mind (as is human nature) about why it's not fixed, and the correct answer about why is ignorance, but they may have assumed another reason.

Depends on the platform and type of bug I guess. When there is a bug in the app I am publishing (and I don’t have many users) I do get loads of reports.

In this particular case it seems like there was no mention of macOS in the project by default. In this case I would assume that people thought it was not supported rather than a bug. Especially for low level projects it’s safe to assume that it working on Linux does not necessarily translate to macOS

> They assume that someone else has already reported the problem, and there would be no point in saying anything. The bigger your project, the more likely people are to assume that someone else has already reported the issue.

This is similar to "the bystander apathy" where bystanders are less likely to help victims the more people are present around.

So, when I find a problem on a large project without finding corresponding bug tickets, while I may then submit one, I may as well retreat, wondering if I'm using the thing in entirely the wrong way, and rethinking my approach.

This applies to your startup too... If your product doesn't work (either entirely or in certain edge cases), most people won't tell you about it.. they'll just move on to something else.

That's why it's important to develop a closer relationship with your first customers (and ideally, they're also depending on you to solve an important problem). Otherwise there's little incentive for people to go out of their way to fix an issue with a product they never got far enough along to adopt.

> It’s a horrifying thought, but it could be that for every one person who opens an issue on GitHub, 100 or more people have already tried your project, run into that same bug, and simply moved on.

It could be 100, or just 1. And sometimes it’s hard to know which without a lot of effort.

This is very typical of all problems, not just software issues.

Yep. And if it’s truly an unknown, you have to assume the worst, it does no one any good to assume the best.

One reason companies do things on "the basis of only three complaints".

If three people complain, there might be hundreds or thousands who didn't bother.

Flipping this around: As a user what message would motivate you to respond?

I'm asking because I'm currently have the following happening in my SAAS:

1. People sign up (paid,CC)

2. People start onboarding (~5 steps)

3. People drop out due to confusion/bugs/something

This is a developer-oriented service and I've tried emailing personally as the founder reaching out for feedback. I've tried automated emails from developer support.

Would live chat help? Offering t-shirt/amazon gift card for feedback? Something else?

Do you email them asking for their feedback, or saying that you want to do anything you need to to make them successful with your product?

This is still small enough scale that I'm sending a personally written email to them at signup asking for feedback or issues.

The reason I don't file many bugs any more: in most cases, maintainers will either ignore them, or come up with some excuse for not fixing it. Yes, even for crashes.

Lots of projects (both open-source and proprietary) have a particularly nasty habit of killing the entire project and starting fresh with something else. So my bug might not even be relevant by the time they get around to it. Or they might wipe the bug database, and I'll need to file it all over again. JWZ calls this the "CADT Model".

I don't want to waste my time on the off chance that this particular maintainer is a better than the rest.

This is why open-source maintainable software is so valuable to me. If it's something I can fix, I'll fix it, and send the patch upstream so you can use it, too. If I can't fix it myself, I'll move on and find some other tool, because I don't want to be stuck with some software that's too complex for me to touch on day one. It's guaranteed to be too complex for me to touch on day 157 when I need a complex issue resolved for a production system right away.

Somewhat related, but am I only one who're annoyed by the "Did you find this information helpful?" form at the end of some help page? I'd usually just click Yes/No, but then some sites forced me to type in the reason. At this point, I'm like "screw this, I'm out" and just close the tab. Wonder how many times this happened to others.

Apple TV+ (tv.apple.com), just launched with a billion dollar launch campaign is pretty broken in the most popular browser/os combo on earth: chrome/windows. (It's also equally crappy on firefox/windows.)

(And there are some really weird design decisions in the Apple TV app. I think they're fundamentally stupid and/or bugs, but the designers probably disagree.)

It seems like they fixed some of the bugs since yesterday.. but.. they had like two years to build this thing. Not impressed. There are plenty bugs left that are trivial to trigger. (Random example: press space while playing to pause, then press play to continue. Observe the glitchy weirdo animations.)

Now that I think of it, I assumed someone else would report the individual 3-4 P1 bugs I've found. (or well, that they would test this themselves...)

I think I held off because I figured that if they didn't catch these obvious bugs, they didn't care. But now I read that Chrome is actually a supported browser for Apple TV+, so.. I dunno what they doing during those two years.

This is why I always worry when some company thinks its okay to launch a v1.0 product directly to a massive userbase, starting from entirely internal testing. There are so many cases you simply cannot or will not adequately test internally, that its a disaster waiting to happen.

Internal users have a known system configuration, and a predictable set of usage patterns, that will not be duplicated by the real world.

You'd think they'd at least bother to test with the most common browser/os combo...

Its possible they did, but were so locked into a prescribed way of using the product that they never ran into the sort of bug an "untrained" user would hit in the first 2 minutes of product use.

(Of course its also possible they didn't, or falsely assumed it would work exactly the same as Chrome/OSX.)

I've read Disney+ has insane levels of DRM, maybe it's the same for Apple TV?

Some of the issues on tv.apple.com seem like they may be related to DRM (sometimes playback simply doesn't work, even though the m3u8 playlist and segments load nicely), but most of the problems are just really (severe) UI glitches.

They're probably not doing any more DRM than e.g. Netflix. Anyway, it's pointless - the launch shows are already available on The Pirate Bay. I'm sure the video/audio/subtitle quality is excellent.

This is basically the Bystander Effect as applied to software:


If this effect is in fact, in effect, here, then the number of visible stars on your project might only amplify it, ironically!

Source: I was a happy Psych major who is happily doing dev now

This is why in every site I build I have it email me whenever an error happens. Unfortunately this is only for the backend code and I really need to do this for the frontend javascript as well.

At volume, Sentry is a better tool for this than email; Sentry makes charts than can help you visualize when a big started happening, lets you filter by code version, user, OS, etc. and groups bugs my error message and/or stack trace.

dude... thank you for that suggestion. Briefly checked out their site and the welcome video looks very interesting. this is something I definitely have to consider adding to my stack:


The other thing that happens is that I'm reluctant to research if someone had already submitted a bug, and so I won't risk submitting a known bug, because the dev gets mad.

Gotta love one-star reviews like "it doesnt work". I actually have a solution to this problem. But it probably do not suit all businesses. When signing up have a checkbox that say "I will report all bugs". Then have an easy way to report them, where other users can help with workarounds. Then follow up and answer all bug reports. I think managing bug reports is a social problem. You need to have empathy more then technical skills.

My username is a little handy here. Often I would put together sites, applications, etc., that would be used by people who were at least at one remove. I would ask my stakeholders, "Have you heard anything?" Occasionally I would hear that "well, this one guy said he didn't like it about six months back." That's it. I tried to have people look at what was going out before release but nobody could seem to find time to do so.

I had even posted my direct phone line on some of these sites, as well as a troubleshooting walk-through in the form of a clickable flowchart and a form that would be sent right to my inbox, yet I got only crickets. Faced with this kind of behavior, I would literally face users in their dens to ask them. If I saw a student using it when I was out and about, I would say, "Hey, I'm the guy who put that together. Please tell me about your experiences, good or bad." Oftentimes, despite my requests to meet with other users (I even offered to drive to people's offices to help), I would be told that the stakeholders would "collect feedback." Often my chain would be yanked for my attempts. No feedback was collected, or if it was, I never heard about it.

This was not just me. We might find out that a printer had been having a problem, and then be told, "Yeah, it's been doing that for a few months now."

I am not sure entirely what was going on, but it wasn't for lack of effort on my part that I did not find out. To some degree, I believe there is a culture among users, management, and stakeholders that programmers can be blamed for only by trying very hard to do so.

Most of the time other people have already reported egregious bugs. Spending hours investigating a bug and reporting it, especially to a big project, is much more often than not wasted time:

- Search for a duplicate for 20 minutes and find nothing. Spend another five minutes on a search engine because the ticketing system search results seem to have nothing to do with your search terms.

- Spend five minutes signing up for the N+1th bug tracker.

- Two or more hours gathering details because every single bug tracker wants all the details upfront rather than just a problem description. Understandable but frustrating because of what comes later.

- Adapt the findings to the N+1th bug template in the N+1th inferior Markdown imitation.

- Submit.

- Wait for an unknown length of time, sometimes years, without a human response.

- Once you get an answer is when it gets really frustrating, because most of the time it is not a new bug. Issue tracker search is just not anywhere near smart enough to save the community millions of hours every year filing duplicates. I must've filed two duplicate bugs for every non-duplicate, and believe me, I really try to save those hours by searching first.

I've worked on websites where people paid a monthly fee, and when certain things didn't work right people never told us over email or phone. Only when we met in person at a convention did they tell us.

The users having the problems were Chinese though, so I'm not sure if it's a cultural thing. Our regular US users did seem more willing to tell us about broken stuff.

I think a big reason people don't report bugs is they've come to expect that nothing will come of it. For instance, there are major bugs with Quickbooks' multi-currency support that I have reported repeatedly over the course of years, with detailed steps to reproduce, and received not so much as a reply let alone a fix. On the other side, any time someone reports a bug on one of our websites, they seem positively shocked when we answer and usually are able to fix it that same day. (We have also found, as with the OP, that bugs can affect a lot of users before one will send a message, so we definitely do try to catch as much with testing and code review as possible!)

I do mostly in-house stuff but I always get nervous when we roll out something and don’t hear anything back. Either it works perfectly (which it rarely does) or it doesn’t work and people have given up. Usually it’s the latter.

I can't tell you how many times I've heard this from customers. There have been so many times when a customer thought a bug was their fault or expected behavior. That's even more surprising because our customers are developers themselves.

Automated error monitoring is becoming more and more important, especially when you don't have the time or resources to add proper integration or platform testing to the project.

This is one of the reasons we built https://rollbar.com

Fyi - I'm one of the Founders

and then consider the typical project on Github which makes it intentionally hard to open up an issue, like with a template where you need answer 10 questions right down to your underpants size...

It's a tough balance. You start a one-dev project for free that gets big, suddenly it's next to impossible to keep up with support requests.

If your project is targeted at developers, you might be able to find someone you can trust to help you out, but if it's targeted at non-technical users that's like finding a unicorn.

I'm not arguing in favor of keeping people out of course, just that it's not as straightforward as "issue templates bad" vs "RTFM noob".

It sounds a bit like the opposite side of the customer acquisition cost versus lifetime value.

Traditionally if you have a high user lifetime value you can spend more money on getting more customers by increasing your marketing spend so your CAC (customer acquisition cost)/LTV (lifetime value) stays at a good level. So you can keep a good growth curve.

In this case when your ltv is negative for each issue because of low quality, or simply too many of them then you make it harder for new issues to come in, which should ideally result in fewer but higher quality reports?

I have a question mark in there because I did not spend a lot of time considering this, but I like the thought.

yeah, I understand the practical side of the problem and is certainly not easy as you say. as for issue templates, reporting guidelines etc., they make it even less likely that people report a problem. as the article mentions, even on a project that encouraged feedback it was still not reported that it doesn't run on the mac. so my takeaway is that there shouldn't be any guidelines for issue reporting if someone wants to catch even a glimpse of the real issues. I'd say there should be a good process in line that deals with the issues and asks for necessary information afterwards, so potential reporters are not driven away by issue requirements. knowing this I'd say projects that have issue guidelines hurt themselves in the end.

People sometimes add unnecessary difficulty to the process of reporting a bug. But there's also some unavoidable difficulty.

You need to gather some good information, You need to explain it in terms that are understandable, and you probably ought to put a little effort into making sure you weren't creating a duplicate bug.

Not that that invalidates your point. It's enough work on its own without adding extra unnecessary work.

I agree, what I was getting at was that requiring detailed information upfront is going to give you even less reports, so it is advisable to get the information you need afterwards as I explained here:


And then you get people complaining about being asked to provide additional details. You can find some of them in this thread.

Some people don't want your feedback, and that's ok.

there's a difference between showstopper DOA-type bugs that cause a project to simply not work at all and bugs that prevent the product from working to its full capacity (intermittent crashes, perf issues, lack of or sub-optimal features/ui).

the author is writing about the former but in my experience (both as a maintainer and contributor) people are a lot more motivated to report and work on the latter type rather than spend hours just to get off the ground...and it makes sense.

Do you think google is not aware of how terrible Chrome's auto-correct suggester is? Mind blown.

IE - it has no idea what words I'm trying to type: civillian, millenial, EMBARRESSED. (for some reason the last one works fine in lower case, but can't guess in upper case - even though it guesses some words in upper case)

It seems to struggle with double vs. single letters. I've always just assumed they don't care. But maybe they don't even know.

Applies to many things. I once went to a Hackathon where shortly after it was over the organiser tweeted about having "nailed it" .... At the exact time a bunch of attendees were at the pub round the corner complaining about it. As far as I know, none of us told them about our issues - but also, they didn't do anything to encourage us to provide feedback.

Reporting bugs is considered as work for most. I gave up on Apple cause I am not paid to report.I only report those which affect my apps.

"I explained that I’d never tested it on MacOS... , but I couldn’t see any reason why it wouldn’t work"


Indeed, if you haven't tested your code on a completely different OS, you have to assume it doesn't work.

He explained later, that contributors used it on Mac and it was working for them.

I truly wonder how many people this affected but didn't think to open an issue on GitHub https://github.com/mjackson/unpkg/issues/229

Or, by the time they tell you it’s broken, they’re so agitated that it’s difficult to get good info out of them.

Sometimes instead of giving up, people just get more upset.

People so rarely Take the time to report issues, I go by a 10x rule. If I hear about it, at least 10 other people are experiencing that issue.

This might explain the Windows Home unstoppable telemetry. With all this information Windows better become flawless.

> The more important lesson, that I didn’t understand until that point, is that you can’t count on the people trying your project to quickly and reliably signal bugs to you.

Took awhile to get to this line, and I feel like it was the thesis.

Applications are open for YC Winter 2023

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact