What is the typical scenario where one really benefits in open over closed source? How often does it occur? I am curious.
Open Source is particularly useful for the long tail. That particular library that is very relevant to you and maybe 20-50 other people in the world. It makes sense to join forces and make it better - but it will never be absolutely mainstream.
Secondly, I say your point 'partially true for mainstream things' because even in mainstream things, there will always be customizations/particularities that again are useful to you and 30-50 other people in the world - thus you join forces (e.g., a obscure linux module).
Agree with you that 99.9% of the people will not
It's all a matter of long tails. Without open source, all long tail efforts would be each independent person doing it for themselves (when you have 30-50 people in the world that should be sharing!)
Although the modern web is making it viable for small teams w/little capital to go after long-tail in a commercial way (many YC startups are a testament to this) --- although I don't imagine them doing this for a 30-50 people target market.
It is really not about fixing something but understanding what behavior underlying libraries expect from software you are writing. The situations where documentation is technically correct but practically useless.
Using software is different than building software. When you're using most software for it's primary function, it's a well worn path. Others have encountered the problems and enough people have spoken up to prompt the core contributors to correct the issue. But when you're building software, you're doing something new. And there are so many ways to do it, you'll encounter unused bits, rusty corners, and unfinished experimental code paths. You'll encounter edge cases that have been known to be broken, but were worked around.
Sometimes, the documentation isn't complete. Sometimes, it's wrong. The source code never lies. For an experienced developer, reading the source can often be faster... especially if you're already familiar with the package's architecture.
I'm in a medium-sized co-working space with several startups. A lot of the other CTOs and engineers come to our team for guidance and advice on occasion. When people report a problem with their stack, the first question I ask them is: "Well, did you read the source code?"
I encourage developers to git clone anything and everything they depend on. Initially, they are all afraid. "That project is too big, I'll never find it!" or "I'm not smart enough to understand it" or "That code is so ugly! I can't stand to look at it". But you don't have to search the whole thing, you just need to follow the trail. And if you can't understand the platform below you, how can you understand your own software? And most of the time, what inexperienced developers consider beautiful is superficial, and what they consider ugly, is battle-hardened production-ready code from master hackers.
Now, a year or two later, I've had a couple of developers come up to me and thank me for forcing them to sink or swim in other people's code bases. They are better at their craft and they wonder how they ever got anything done without the source code in the past.
When you run a business, if your software has a bug, your customers don't care if it is your fault or Linus' or some random Rails developer's. They care that your software is bugged. Everyone's software becomes my software because all of their bugs are my bugs. When something goes wrong, you need to seek out what is broken, and you need to fix it. You fix it at the right spot in the stack to minimize risks, maintenance costs, and turnaround time. Sometimes, a quick workaround is best. Other times, you'll need to recompile your compiler. Often, you can ask someone else to fix it upstream, but just as often, you'll need to fix it yourself.
Closed-software shops have two choices: beg for generosity, or work around it.
Open source shops with weaker developers tend to act the same as closed-software shops.
Older shops tend to slowly build the muscles required to maintain their own forks and patches and whatnot.
True hackers have come to terms with a simple fact: If it runs on my machine, it's my software. I'm responsible for it. I must understand it. Building from source is the rule and not an exception. I must control my environment and I must control my dependencies.
It's very empowering to read the source code because one can learn a lot.
A few more things to add from your points:
To work-around a software, sometime I have to understand how it works internally and given the source code, I don't have to spend a lot of time to reverse-engineer it.
Having said that, sometime the customers do care if it's my fault or IBM faults if we stated clearly in the deliverable that "hey, we're using IBM WebSphere for this...". They would give IBM some time to fix it, or ask us if we can work-around it, or they would bend certain requirements (i.e.: work-around the requirements, not the code).
This is why sometimes, probably a dying trend given the quality of open source software, an outsource project chose software based on the company behind it.
It can be a lot faster and accurate searching the source code for an answer than googling it. My most recent example: the .NET WebClient throws an exception ("ProtocolError") if it receives an HTTP 304 ("Not Modified") response. This makes no sense, but is it a bug? Did I use the client correctly? Or is it by design - i.e. a vexing exception (http://blogs.msdn.com/b/ericlippert/archive/2008/09/10/vexin...).
If you google for the answer you get a bunch of other confused people and a bunch of different advice, none of which is appealing. If you look at the source code you can quickly answer the question: yes, they really did design things in this brain dead way, and now my caching layer - which is supposed to be fast - is going to have to handle an exception every single cache hit. (When you're issuing over a million requests per day and you're paying for CPU time by the hour this design decision actually has a cost since exception handling is slow.)
Granted, this is not going to be applicable in the same way to something like the ASP.NET etc. but open source works, for me.
In a previous project (jQuery/Python) I had an issue with a Python library that I needed to patch. It worked fine on my collaborator's Mac but not on my machine.
Even .net - I created a downloadable product that happened to require a fair amount of custom UI work and being able to look under the hood (with Reflector) helped a lot. There are still a couple issues that I could never get resolved that are buried in the stack.
The scenario you describe likely happens more often than you think. Sometimes (often) documentation gets out of sync with code.
I've never had the need to get into ASP.NET or the .NET framework itself but being able to get into components built on top of these lower level technologies has been very helpful.
I think that Microsoft has finally realized that the best way to build and maintain a community of software professionals is to treat them like esteemed associates. It's too bad Haack left, but with Scott Hanselman evangelizing (he even came to my mid-sized midwest city a few weeks ago) and ScottGu running the show, things are looking bright for MS devs.
To give some context, my first job out of college was in a small WebForms/MVC shop in LA. It was very progressive in that my managers always pushed me to question the conventional "MS" way of programming/architecting. However, what frustrated the hell out of me was all the other MS companies around us: they were so damn dependent on what MS gave them.
I've heard countless MS devs in local meetups screaming bloody murder when MVC first came out, "Microsoft, you HAVE to train us on MVC! It's a black box, we're effed!" This shocked me, because these people wanted to literally be force-fed with MVC. Microsoft (intentionally or not) encouraged this kind of behavior with their close-mindedness, and this dependency/babying was crippling to the MS dev community.
Accepting pull requests signifies a tremendous step on MS's side, because they're finally admitting that there are actually smart developers OUTSIDE OF MS BUILDING who can help out with the framework. Their closed-software drove me away from them to RoR. Now, I have some peace of mind knowing that the bleeding of good devs away from MS stack may finally slow down. I can't wait to see the next generations of Scott Hanselman and ScottGu pushing the envelope for MS.
Hopefully with a bit of luck I'll never be using or coding on MS stack ever again though!