I initially disagreed with the author when he said, "we tend to overestimate how much we know about the software we're working on."
I disagreed because that is not my experience; I do know a lot about my software.
However, then he said, "We overemphasise the parts of the system we've personally worked on, particularly those we've most recently worked on. We downplay other parts of the system, including the impact of dependencies (e.g. libraries)."
Oh. Um, well, this is also not my experience.
I work on software alone, and I do not have any dependencies.
So the author is right; I just have weird experience. :)
I agree with the four kinds of optimization, especially since he focuses on optimization humans do. I think you could even shoehorn compiler optimizations into those four. For example, strength reduction could be considered a "better algorithm."
His observations about the differences between best, average, and worst cases are the things programmers need to learn most. At some point, our code will always hit bad cases, and we need to know if it will hit it often.
I love his example of Timsort because I've analyzed and implemented it myself. It ended being more comments than code. (Of course, that includes two license header comments, so slightly less comments than code actually.)
Timsort is pretty complicated, and Tim isn't the best at naming things, so it took a bit for me to understand what was going on.
(The most prominent example is the galloping functions. He named them "gallop_left" and "gallop_right," but they both gallop left and right. I renamed them to "gallop2Leftmost" and "gallop2Rightmost," which is actually what they do.)
The author says he biases to writing algorithms and adopting pre-written data structures. This is a good thing. However, after having written data structures (because I work in C, which doesn't have them), I would encourage every programmer to write some. Best, average, and worst cases are easier to learn on data structures than algorithms. Still use pre-written ones though; they are usually better optimized.
The trick about reducing memory is crucial. I recently spent an entire refactor just reducing the size of some of the most-used structs. I cut the size of one struct in half. `pahole` is great for this.
The part of PyPy is a great example too. A great book to read is Is Parallel Programming Hard, And, If So, What Can You Do About It? The entire point of the book is to make you think, "Okay, I need some concurrency or parallelism; what is the easiest way to get it?" The PyPy example is that same thing with optimization; what's the easiest way to get the performance you need?
Every time you run into an optimization problem, ask that question first, and it will get faster to answer every time. And the author's summary reinforces this.
> However, then he said, "We overemphasise the parts of the system we've personally worked on, particularly those we've most recently worked on. We downplay other parts of the system, including the impact of dependencies (e.g. libraries)."
> Oh. Um, well, this is also not my experience.
> I work on software alone, and I do not have any dependencies.
If you work in a company of 100+ engineers with multiple specialties it's impossible to keep up with everything. At <50 engineers if you collaborate and/or switch teams it's possible to know almost everything, but that's the most I think the majority of us have the internal storage/RAM for.
I suspect your thresholds are far too high, and that some specialties are almost mutually incomprehensible, so rare for one person to properly understand what is going on.
Time is a real factor also. How long has this code base been accumulating...
Early on in the company I work we had a practice of collaborating across teams, sharing members etc. so a lot of us knew a lot of the code base from "different angles". If you implemented a backend feature you might write also write the terraform and the frontend that worked with it, with some input from the people experienced in those things. Kind of like random full-stack development? Slower than normal development at first but it makes future collaboration easier. It was quite great, but unsustainable as the amount of people grows.
edit: I think in general at the early stages of a company you hire a lot of people who are good at adapting and learning new skills since they might need to fill new roles quickly as new needs arise. So those people are good at this kind of full stack development since it's their niche. As you grow, you hire more specialized or junior people, and they can't do this process quickly enough for it to feel fluid anymore.
I initially disagreed with the author when he said, "we tend to overestimate how much we know about the software we're working on."
I disagreed because that is not my experience; I do know a lot about my software.
However, then he said, "We overemphasise the parts of the system we've personally worked on, particularly those we've most recently worked on. We downplay other parts of the system, including the impact of dependencies (e.g. libraries)."
Oh. Um, well, this is also not my experience.
I work on software alone, and I do not have any dependencies.
So the author is right; I just have weird experience. :)
I agree with the four kinds of optimization, especially since he focuses on optimization humans do. I think you could even shoehorn compiler optimizations into those four. For example, strength reduction could be considered a "better algorithm."
His observations about the differences between best, average, and worst cases are the things programmers need to learn most. At some point, our code will always hit bad cases, and we need to know if it will hit it often.
I love his example of Timsort because I've analyzed and implemented it myself. It ended being more comments than code. (Of course, that includes two license header comments, so slightly less comments than code actually.)
Timsort is pretty complicated, and Tim isn't the best at naming things, so it took a bit for me to understand what was going on.
(The most prominent example is the galloping functions. He named them "gallop_left" and "gallop_right," but they both gallop left and right. I renamed them to "gallop2Leftmost" and "gallop2Rightmost," which is actually what they do.)
The author says he biases to writing algorithms and adopting pre-written data structures. This is a good thing. However, after having written data structures (because I work in C, which doesn't have them), I would encourage every programmer to write some. Best, average, and worst cases are easier to learn on data structures than algorithms. Still use pre-written ones though; they are usually better optimized.
The trick about reducing memory is crucial. I recently spent an entire refactor just reducing the size of some of the most-used structs. I cut the size of one struct in half. `pahole` is great for this.
The part of PyPy is a great example too. A great book to read is Is Parallel Programming Hard, And, If So, What Can You Do About It? The entire point of the book is to make you think, "Okay, I need some concurrency or parallelism; what is the easiest way to get it?" The PyPy example is that same thing with optimization; what's the easiest way to get the performance you need?
Every time you run into an optimization problem, ask that question first, and it will get faster to answer every time. And the author's summary reinforces this.
[1]: https://mirrors.edge.kernel.org/pub/linux/kernel/people/paul...