1. If x86 hardware is so terrible (and I have heard that the architecture really is bad many times), how come we don't have competing chips out there which are many, many times more efficient? I know ARM outperforms on the low-power front, but not in terms of perf to my knowledge. Do such chips exist? And if not, why not if this is true? Even with x86 backwards compatibility concerns, you could bring out a theoretical amazingly powerful chip and just port compilers to it to leverage it and gain some (possibly niche) market hold that way.
2. I think he is underestimating the vast complexity of computing, and deeply underplaying the techniques which have evolved to get us where we are. Yes, I have again heard that the Von Neumann architecture has shortcomings, but look at what computing has enabled us to do - it has changed the world + enabled amazing things which are evolving at breakneck speed. Again - if there really is an ideal alternative approach, why isn't it being pursued? I don't buy the conspiracy theory crap about vested interests holding it back. Again, you could do well in a niche if the alternatives really are that much better.
3. I think it is likely that as things evolve previous ideas/approaches will be overturned and old approaches thrown away, like any pursuit. As @deweller says, this is true of any field. Ask any professional and they will tell you how their industry is fucked in 100 different ways. It is frustrating to hear software engineers talk about how terrible immature + hackerish + crap it all is while assuming that things have some sort of crystalline beauty in other industries. I did Civil Engineering at (a good) university, and it turns out that most consultancies use linear models to model everything, then apply an arbitrary safety factor, resulting in hopelessly inefficient over-engineered structures all over the place + turning the job into something of an uncreative administrative role in some respects (with no disrespect intended for all the civil engineers out there).
4. It is my theory that any attempt to interface with the real world and actually do useful real things results in an enormous amount of uncertainty and chaos in which you still have to make decisions however imperfect and however negatively others will judge it going forward. I think that's true because the world is chaotic and complicated and imperfect, and to influence it means to be touched by that too. That doesn't excuse poor engineering or not caring or stupid decisions, etc. but it is a factor everywhere I think.
5. So change it :-) there are efforts afoot everywhere to struggle to improve things, and from the sounds of it the guy has perhaps not tried enough things. I feel go is a great step in the right direction for a systems language which removes a lot of the crap you don't want to think about, as does C# (though it does potentially tie you to ms crap).
6. Programming is difficult, solving real problems is difficult, abstractions are nice but leak, and sometimes it's quite painful to have to put up with all the crap other people hacked together to make things work. But the value is in the end product - I don't care that my beautiful macbook air runs on a faulted architecture and some perhaps imperfect software, the beautiful ease of use is what matters to me. We mustn't lose sight of these things, though again this is not an excuse for crap code. There is definitely a balance to be struck.
7. Zack Morris is clearly a highly accomplished + competent hacker given what he's done and the obvious passion he writes with (to care enough to be disappointed + upset by these things is indicative), which I think has a bearing - the deeper your knowledge, the better your skill, the more faults you notice by dint of your intelligence + competence. There is a definite curse at play here.
Anyway, just my 2p :)
A quick answer to your @singular's questions:
1) Existing code - this trumps writing everything from scratch.
2) I don't agree, I believe the compuation is straightforward, my belief is that what you perceive as 'progress' is mostly just 'go really really fast.' I showed a Microsoft engineer at the Vintage computer festival installing an RDBMS on VMS while four people were playing games and exploring VMS on four terminals connected to the machine, then I fired up and ran the test code to show the code had installed and was usable. It did not impress him that I didn't reboot the system once, nor did the other four people using the system notice I had installed a new capability that was available to everyone using the OS. Those are not design goals of a 'personal' OS like Windos/DOS/NT, although they could be. The stuff you learn in CS classes about architecture and layers and models and invariants, can make for very robust systems.
3. My experience is that programmers program. Which is to say that they feel more productive when they produce 10,000 lines of code than when they delete 500 lines of code and re-organize another 500 lines. Unlike more 'physical goods' types of industries it is easier to push that stuff out into production. So more of it ends up in production.
4. Not sure where this was going.
5. This is something I like to believe in too, its just code, so write new code. Hey Linus did it right? The challenge is that it will take 4 - 5 years for one person to get to the point where they can do 'useful' stuff. That is a lonely time. I wrote the moral equivalent of ChromeOS (although using eCos as the underlying task scheduler and some of the original Java VM as the UI implementation.) Fun to write but not something picked up by anyone. You get tired.
6. I'd take a look at eCos here, one of the cool things about that project was a tool (ecosconfig) which helped keep leaks from developing.
In the 'hard' world (say Civil Engineering) there are liability laws that provide a corrective force. In software it is so easy to just get something put togehter that kinda works, that unless you are more interested in the writing than the result, you may find that you're spending less time on structure and more on results.
2. Sure, I guess what I'm getting at is that we've done amazing things with what we've got, I'm by no means suggesting we shouldn't take a broader view and replace crap, or at least work towards it where market entrenchment makes things difficult. The point is, again, that if there exists such a plausible alternative to the Von Neumann architecture, then why aren't there machines out there taking advantage? Again you could probably fill a niche this way. I suppose, in answer to my own question here, that you would be fighting a losing battle against the rest of the hardware out there being reliant on V-N but still, I'd have assumed that something would exist :)
3. Yeah. But it's hard + often the harder path to do things right in any industry. Such is life, not that that excuses anything.
4. A sort of philosophical point. Feel free to ignore :-).
5. There is stuff out there that already exists too though. Go, OCaml, Haskell, F# are all really interesting languages which in their own ways tackle a lot of the accidental complexity problems out there. Plan 9 + inferno are very interesting OSes, though they are probably a little too niche to be all that useful in the real world. But yeah, understandable, fighting the tide is difficult.
6. Cool will take a look.
Yeah - one of the things that attracts me to software engineering is the relative freedom you get to be fully creative in solving a problem. However that cuts both ways it seems.
Suppose I invented a new chip that was awesome for gaming, spreadsheets, word processing, databases and power consumption.
Who would build PCs with it?
What OS would it run if someone built it?
Who would buy that?
Its not merely a "huge, massively important thing". Its the only thing.
No doubt they made some mistakes, but it wasn't for lack of trying.
(And having debugged code on an ia64 I'm quite happy with the status quo!)
As we move into an era of 'I don't see why I should get a new machine' of growth minimization there is a window for folks like ARM to get in with 'all day computing.' But it will take someone extraordinary to make that happen. Look at the state of Linux on ARM to understand the power of a legacy architecture.
That is, until you do them in parallel...
You're absolutely right: if it could be done much better, there'd be an example in the market to prove it. Yet Intel is walking all over the market, and has been for the last 8 years or so.
X86 is amazing in spite of legacy and poor architectural choices.
Would you also say that there probably couldn't exist a significantly better OS than Mac, Windows, or Linux, or else we'd know about it? I admit "better" can be hard to define; what would make a 10x better car, or IP protocol, for example? It strains the imagination, because what makes these things "good" is complicated. But ask anyone who was around during the early proliferation of computer architectures and operating systems, and they will tell you that what ended up winning out is markedly inferior to other things that could have been. Paths get chosen, decisions get made, billion-dollar fabs get built. The market doesn't pick the best technology.
It's like web standards -- accept them, but only defend them to a point. We may be stuck with CSS, but that doesn't mean it's any good or that it succeeded on its merits.
As to the rest of it, I think you can look at microwaves for a perfect example of terrible software in wide spread use. You need to be able to select cook time and possibly power level or set the clock. Yet, most microwaves have such a terrible interface that few guests get embarrassed asking how to get a new one to work. And as long as it takes more effort to send it back than it takes to figure out the strange design there is little reason to build a better system.
the core dilemma of computer science is this: conceptually simple systems are built on staggeringly complex abstractions; conceptually complex systems are built on simple abstractions. which is to say the more work your system does for you, the harder it was to build.
there are no stacks which are pure from head to toe. I guarantee you, old LISP Machine developers from Symbolics probably had a hard time designing their stuff as well.
And I wouldn't say PPC lost. IBM has competitive CPUs in the market, they're just not in consumer devices. But they're just that: "competetive". They aren't much better (actually pretty much nothing is better than Sandy Bridge right now).
To me TFA is about a reassessment of fundamental assumptions, and it's about exploration. It doesn't suggest concrete solutions because nobody knows what they are, but it does suggest that our efforts to better the art have been short-sighted. Right now the next Intel chip or ARM chip is just another target for compilation, just another language or library fight with instead of solving real problems - solving old problems, not just the latest new/interesting/imagined ones.
(FWIW, this particular example doesn't excite me too much - If the future is DWIM, it almost certainly has to be done first in software, even if it is eventually supported by specialised hardware.)
Also, x86 isn't as bad as it is made out to be either, even if most of that is just how well our compilers can optimise x86 code.
What about IBM's Cell (the one used in PlayStation 3)? Or GPU related technologies, like nVidia CUDA?