> let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, "I don't see the use of this; let us clear it away." To which the more intelligent type of reformer will do well to answer: "If you don't see the use of it, I certainly won't let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it."
That said, if you know how to refactor safely even without tests, you can make improvements. Since the questioner is working in JavaScript, Martin Fowler came out with a second edition of Refactoring with JS code that might be useful.
I've never read the first edition (in Java) but Michael Feathers' Working Effectively With Legacy Code is a frequent recommendation of mine. Its main goal is to get your mess under test, ideally tests that help you understand the system as well as improve its quality. It's organized as a set of questions one finds oneself asking in these kinds of code bases and some ways to resolve them: https://www.oreilly.com/library/view/working-effectively-wit...
The counterpoint to Chesterton's Fence is the psychology experiment with the monkeys, the stairs and the banana.
This is long but very pertinent to the "that's just how we do things around here" attitude:
> This human behavior of not challenging assumptions reminds me of an experiment psychologists performed years ago. They started with a cage containing five monkeys. Inside the cage, they hung a banana on a string with a set of stairs placed under it. Before long, a monkey went to the stairs and started to climb towards the banana. As soon as he started up the stairs, the psychologists sprayed all of the other monkeys with ice cold water.
> After a while, another monkey made an attempt to obtain the banana. As soon as his foot touched the stairs, all of the other monkeys were sprayed with ice cold water. It's wasn't long before all of the other monkeys would physically prevent any monkey from climbing the stairs.
> Now, the psychologists shut off the cold water, removed one monkey from the cage and replaced it with a new one. The new monkey saw the banana and started to climb the stairs. To his surprise and horror, all of the other monkeys attacked him. After another attempt and attack, he discovered that if he tried to climb the stairs, he would be assaulted.
> Next they removed another of the original five monkeys and replaced it with a new one. The newcomer went to the stairs and was attacked. The previous newcomer took part in the punishment with enthusiasm! Likewise, they replaced a third original monkey with a new one, then a fourth, then the fifth. Every time the newest monkey tried to climb the stairs, he was attacked. The monkeys had no idea why they were not permitted to climb the stairs or why they were beating any monkey that tried.
> After replacing all the original monkeys, none of the remaining monkeys had ever been sprayed with cold water. Nevertheless, no monkey ever again approached the stairs to try for the banana. Why not? Because as far as they know that's the way it's always been around here.
I don't see how that's a counterpoint. Chesterton doesn't say "don't change it, that's how it is and shall be." He's saying, "Question why it's present before you attempt to change it."
Looking around, there were no obvious livestock or other reasons for the fence. But perhaps they're only there six months out of the year. Ok, make the fence a gate, and not a fence, so it can be opened more easily. Maybe it's no longer used for livestock, ok remove it altogether. But if it is used for livestock (but they were just over the hill that day), and the fence is removed, an error has been made (it existed for a reason) and the livestock can now wander beyond the lands they're supposed to be on.
Usually there is nobody to ask though, and if you're refactoring a mess then the code won't give you a clear answer. The only way to find out if the was a reason the fence is there is to remove it and see what goes wrong which is what you were going to do anyway.
Actually, the right place to ask is the unit tests or integration tests. Though there seems to be an inverse relationship between the code that needs refactoring and the prevalence of proper tests.
... which is a form of questioning why it's present. "Attempting to change it" in this situation is more like "submitting the pull request[1]" than "editing the code". The PR tends to only come together after one or more rounds of "git stash, try again".
Short of doing that, talking to the stakeholders could help? I feel like if you understand the actual requirements you can safely make those refactoring decisions
> The counterpoint to Chesterton's Fence is the psychology experiment with the monkeys, the stairs and the banana.
Except that experiment never happened. It appeared in some book without any source (a.k.a. pulled out of the author's ass). Interestingly there was a similar experiment, but with an opposite result; the old monkey re-learned the forbidden behavior from the new one.
Except for most cases, this is the proper behavior. We share experience with each other, and we can learn from it even if we haven't personally experienced it, or even if the person telling us about it didn't experience it personally.
The anecdote says they don't approach the stair because that's the way it has always been, but the monkey might say, "we don't approach the stair because the last monkey in here told us that we would be sprayed with water if we did"
Learning from people who are no longer here is really important. Just because something hasn't happened in your lifetime doesn't mean it won't happen. There are so many stories from Japan after the Tsunami, about places that learned from their ancestors to stay safe from something they themselves had never experienced.
I head a tale that had the same lesson. Not sure where I heard it, but it went something like this:
>There was a Buddhist monastery up in the mountains. Every day at noon the head monk would call everyone together in the main courtyard to meditate.
>One day a cat that lived on the monastery grounds started to come to the courtyard during these meditation sessions. It would screech at, and scratch the monks while they tried to concentrate.
> After a few weeks the head monk got tired of this. He ordered that every day before they meditate, the cat be caught. They tied up the cat away from the main court yard. After meditation the cat was to be released.
> This went on for a few years and everything was fine. Then the head monk passed away. A new monk was appointed.
> A few more after that the cat passed away. The monks retrieved a new cat from a nearby village and they started typing up this new cat; as is tradition.
The monkeys are passing intelligent learned behaviour - the monks are not.
In the monkeys case - it makes sense they 'follow the herd'. The learned behaviour of their ancestors is saying something: "you're gonna get sprayed with cold water if you do this"
I'd imagine a lot of things get passed on this way, maybe some right some wrong, but I don't think this really helps the 'go your own way' case so much. That we generally follow crowds millions of years into our evolution might indicate that there is indeed wisdom in crowds. Though perhaps it's helpful to realize that they are not always right.
That was such an awesome story that I immediately tried to find the source. Unfortunately, all I found was an article and a stack exchange answer arguing that it is made-up by the authors of the book it first appeared in. [0] [1]
Nevertheless, I can come up with a few examples similar behavior in large organizations surrounding processes, workflows and general wisdoms.
It’s an apocryphal story that became popular with “new age” and personal development crowds to encourage you to not be like the masses. It is endearing and illustrative of a useful truism, but sadly made up to be that way.
I think it's always important to lead with an attitude of service and compassion for the labor of others before you and yourself now. Get upside down on that and it makes it really hard to be impactful.
It's easy to get mad at code when you first show up and just apply Chesterton's Fence and move forward. Personally, with legacy systems (and what isn't really?) when I show up to a new project I do my best to grok the internal knowledge and test it. With official tests but also building out an environment around it so I can test conclusions. I document what people tell me, what docs and scripts they link me to, and what my results are. Almost without fail, the institutional knowledge is cargo cult.
People rarely break out of their mold for what works for the thing they need to work on. This isn't a problem that they need to fix, but legacy systems always need someone to take the time to make more clarity. So as a new person I focus on building up dev and deploy environment from zero, over and over. It's kind of an ops process. For me it's about finding multiple/duplicate configuration points, processes, improving VM or container environments, profiling, making sure that if external services are being leveraged that they can be bootstrapped to local dev or at least faked, and confirm tests harnesses if they exist.
With a "big existing code base" I have found that almost universally, the full stack can't be bootstrapped on the devs computers. They rely on tricks to run parts and cloud services and undocumented scripts. It's never because it's impossible and as a new person, I am uniquely qualified to solve that. Once I have a smooth rebuild process I move inward and refactor for clarity and tests, being careful to not disrupt other people's working strategy.
So in my experience, the first 4 weeks are asking questions and documenting the lies while repeated bootstrap. Second 4 weeks is environment cleanup. After that it's learning the system from the outside in by making the entire environment single command bootstrappable. I keep a daily journal of my experience and try to build out docs/readmes to consolidate and correct the practices that were relayed to me in the first 4 weeks as well as new clean up that I'm working on.
Being a new person on a large legacy code base is powerful. You don't have the entrenched survival problems other people have. Clean up and document to explore and try to be an exponential productivity add while you do it. One of my favorite tricks is "adding" features by removing things that people did not remember they have put into production. At my current job, my first three months was -7000 and +500 lines to our master repo. My negative -7000 was more impactful than some people that pushed 100k+/- in commit. Also I find it's useful to wildly speculate about how things work and be willing to be VERY wrong early and often. People are more likely to tell you about how they think something works when they are correcting you than when you ask directly for help. No clue why that's a thing, but it is.
After you get done cleaning things up you can make your own horrible lore and mistakes and start the process all over!
> People are more likely to tell you about how they think something works when they are correcting you than when you ask directly for help. No clue why that's a thing, but it is.
While some of that might just be people's enjoyment at telling other people they're wrong, I think it's actually often something else. It's just easier to _respond_ to something than it is to start with a blank slate.
The question "How does this work?" is a blank slate. Even when asking it for yourself about a gigantic new codebaes, you form hypotheses and then 'test' them in some way, as you say, "testing conclusions". When you give someone a theory, you've just given _them_ a 'conclusion' they can 'test' against their own domain knowledge. :)
I've managed to finish reading the chapter with the example refactoring from Fowler's book before putting it down as parody.
He's part of a design school that likes to pulverise code into tiny functions with the goal of having code that consists only of assignments and function calls, reading like instructions in English to the computer.
Whoever was indoctrinated by Robert Martin, Martin Fowler & co should read Ousterhout's design book for a fresh perspective on this.
The value of a book like "Refactoring" lies in naming the refactoring (e.g. extract function), but please don't listen to the design advice.
Thanks for the pointer to Ousterhout, I don't think I've heard of him before but a lot of the Tcl people have put out good stuff.
Uncle Bob and friends definitely form their own "school". I think they call the "functions and assignment" style "Newspaper style", since for them it's kind of like reading a newspaper. Following it you can actually get a long way towards the dream of "self documenting code". I'm happy when people who believe in some specific school of thought get it to work for them, and even if I disagree on things at least there's consistency.
It's still never as clear as a literate program, though. And while Uncle Bob has been upgrading by learning Clojure, a lot of that school's work has assumed OO-in-the-style-of-Java5/C++98 environments. And to be fair that's a lot of the mass of legacy code we unhappily get saddled with. I think their work is less applicable and there are other schools to listen to when you have different languages or programming methods (FP, declarative, OO with a MOP, etc.).
I read the Clean Code book finally not too long ago, and while I've never argued with a book so much, there is value in it, and some programmers would do much better than their current flailing to follow its approach exactly. But these guys aren't my personal programming heroes, I'd rather be like a Norvig than a Jeffries (even if the latter is a perfectly acceptable engineer). But I can usually find value in all their works; philosophically I'm aligned with Bruce Lee, I'll take what I think is valuable and discard the rest.
Re"pulverise code into tiny functions with the goal of having code that consists only of assignments and function calls"
You dont refactor so its "so small it becomes functional". You remove/refacor code with side effects where possible. So that part of the code becomes functional. It does not matter if a function is a lot of code or a little as long as it does not impact the world outside it.
A small function that adds two numbers and as a side affect assigns some global state is to be avoided however small the function.
"Fuctional programming" is not just about writing lots of functions
I apply Chesterton's Fence to pretty much everything, from code to design to business goals to politics. Back when I worked for a large tech company, I'd get deeply frustrated with their slow decision-making framework. I'd joined the company through their acquisition of our company and it felt like banging my head against a wall all day.
It was only a year after I quit that it began to make sense. Now I try to assume that people had good reasons for their implementations and I try to figure that out first.
Ok, but seriously I am genuinely curious, what is the reason for the fence in the picture? I see fences like this when I am out in the mountains camping or hiking. Is it to prevent some type of animal from roaming down the roads?
Every fence might be different! Clicking on the picture gives a description of that particular one's history. When I see fences/gates on forest service roads or access roads I presume one reason for them is to limit access. Roads require maintenance, having fewer vehicles driving over one means less maintenance. During winter season a fence could also be there to limit access for the purpose of reducing the need for search & rescue or towing services...
With statically typed languages, you can do guaranteed safe, automated refactors. I’ll do those mercilessly without tests. I would be very wary of refactoring dynamically typed language.
I've broken code in Java because I inlined a private method as part of a refactor. I found out later that some other code was accessing it through reflection. Fortunately that other code was an old test, instead of production code, so when it started failing on the test automation servers I found out about it and could update the test to not do that. Then I did what I do frequently anyway, regardless of dynamic or static typing, which is to use ag[0] and increase my confidence there weren't any other surprise references. Of course there's always the possibility of some sadist splitting the method name into two strings and concatenating them later, ag's no guarantee either.
If you've ever done a partial build system you can also break downstream dependencies you didn't know about from even compiling, and not know about it until you try to integrate. (I've done this too, fortunately the integration happens as a gate to checking into the main code branch.) If you at least keep the source of everything locally, even if you don't build it, ag can help again.
My only point here is that static typing isn't enough if you're prone to fear-driven development; you have no guarantees, just things that increase confidence. Static type proofs are but one way to increase confidence. I'm happy Fowler decided to use JS for his second edition, since this particular line of FUD when it comes to there being some impedance mismatch between dynamic languages and refactoring is unmerited. The first auto refactor tools were made for Smalltalk, a dynamic language, after all.
To me the two keys to safely (at high confidence) doing any refactor are to first know what you're doing (and what a tool is doing if you're using one, I'm all for pushing for better tools) and second to do it in small bits with a tight feedback loop. As part of the loop after you make a change you use whatever methods (compiling for static type proofs, running tests, manually testing (REPLs help this a lot), sometimes just pure reason) to become sufficiently confident that you didn't break anything. Sometimes you still break things, as part of a refactor or just as part of regular development -- every bug filed is one that got past all of your reason, your compiler, and your tests, but don't let that fear drive you.
[0] https://github.com/ggreer/the_silver_searcher There are other tools too. Regardless of language, writing "grepable" code promotes a lot of nice qualities, not just easing refactoring.
Even if someone were crazy enough to use reflection to get around accessibility for testing (which you should never do), once I saw that once, I would be judicious about using “Find all references for a class” - something else you can’t do reliably with dynamic languages.
As far as partial build systems, with at least C# that would still be caught when you do an integration build. But that still begs the question, why wouldn’t you use a proper internal package repo and proper versioning?
> let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, "I don't see the use of this; let us clear it away." To which the more intelligent type of reformer will do well to answer: "If you don't see the use of it, I certainly won't let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it."
That said, if you know how to refactor safely even without tests, you can make improvements. Since the questioner is working in JavaScript, Martin Fowler came out with a second edition of Refactoring with JS code that might be useful.
I've never read the first edition (in Java) but Michael Feathers' Working Effectively With Legacy Code is a frequent recommendation of mine. Its main goal is to get your mess under test, ideally tests that help you understand the system as well as improve its quality. It's organized as a set of questions one finds oneself asking in these kinds of code bases and some ways to resolve them: https://www.oreilly.com/library/view/working-effectively-wit...