Hacker News new | past | comments | ask | show | jobs | submit login
Be Kind (briangilham.com)
2475 points by bgilham on Oct 14, 2016 | hide | past | favorite | 434 comments



When he returned to the air field, Bob Hoover walked over to the man who had nearly caused his death and, according to the California Fullerton News-Tribune, said: “There isn’t a man alive who hasn’t made a mistake. But I’m positive you’ll never make this mistake again. That’s why I want to make sure that you’re the only one to refuel my plane tomorrow. I won’t let anyone else on the field touch it.”


There are two schools of thought, especially in stock trading.

1) Reversion to the mean. This is what the above person believes and that this likely won't happen again.

2) Indication of a trend. The pilot is actually incompetent, and this will happen more frequently with this pilot than an average pilot.


I know you're probably not implying regression to the mean is causational, but that was my initial reading so I want to clarify for those who may not be familiar with the concept.

Regression to the mean is simply that any given datapoint is most likely to be the mean, or close to it. This means that any exceptional data point,up or down, can be expected to be followed up by the mean.

The example of this being misinterpreted that I am familiar with is that of a flight instructor's belief on training. When a pilot performed well, they wouldnt comment. If a pilot performed poorly, they would be punished. They believed this was better because when they praised a pilot, they would usually do worse the following run, and when they punish them they do better. This isn't technically wrong, they are just ascribing causation where there is none. I think I saw this example in Signal vs. Noise, but I'm not sure.

Basically, regression to the mean isn't a reason to pick someone who did poorly, it's a reason that that person will do no worse or better than they do normally.


Yeah, I always make an example with coin flips to show how this is true.... lets say heads is success and tails is failure.

Flip 100 coins. Take the ones that 'failed' (landed tails) and scold them. Flip them again. Half improved! Praise the ones that got heads the first time. Flip them again. Half got worse :(

Clearly, scolding is more effective than praising.


That's a brilliant example. I wont forget that now.


Except, coins are not humans with emotions, and neither can they dupe probability to improve their outcome. The 50 heads(success) that you left are not going to do any better than the 50 tails you scolded. You are just changing the sample space in a biased fashion to prove your point.


I think you are totally missing my point. The whole point I am trying to make centers on the fact that the coins are all actually equal. The observed difference in performance is entirely due to chance.

While, of course, human performance is not always equal in the same way our coins are, the fact that performance (or whatever it is you are measuring) will still regress towards the mean still holds true. The coin example just gives an extremely obvious demonstration about what is happening when things regress towards the mean.

https://en.wikipedia.org/wiki/Regression_toward_the_mean


hilarious! thank you!


The pilot performance example is in Thinking Fast and Slow, I think.


I've read that story in "How to win friends and influence people"

https://www.google.com/#&q=%22how+to+win+friends+and+influen...


Ah, yeah. I think that's actually where I saw it. Thanks.


Now if only Intel would add that to their instruction set.


Isn't regression to the mean a phenomenon of random variables?

Humans aren't perfect improvement machines, but they surely beat a random variable?!


Correct. I think reversion to the mean is a non-sequitur here.

Neither the GP's quote, nor the OP, are making the statement "I understand that this mistake was an outlier, you'll probably never make this mistake again if you remain unchanged".

The claim being made is that acknowledging the mistake and learning from it can dramatically reduce the base probability of such mistakes happening again.


Regression to the mean applies any time a process has high variance. If I have an unusually productive week, odds are the next week will be worse, simply because the previous one was an outlier. But people often make the mistake of noticing week X+1 was much worse than week X, and attributing it to some variable that changed between X and X+1.


In my model, humans have an average performance, that increases over time (unless something terrible happens) and a random component, that makes performance fluctuate around the average.

The question is: is the day-to-day random variation much bigger than the day-to-day average increase? If so, regression towards the mean makes total sense.


I disagree with your interpretation. I think this is a case of understanding that the pilot is now motivated to save face, and will be extra careful.

This is psychological insight, not statistical insight.


Wow, you totally missed the point of the story:

3) Out of remorse, pride, being shocked out of complacency or some combination of these, the man would do his absolute best the next day, especially for the very same pilot.


There's also differences in management and industry style.

If labour is so abundant that there's little problem in replacing any given worker, people can and will be fired for trivial offences. If there's a shortage, or there's a considerable on-boarding process, less so.

The threat of a firing-at-any-moment also makes for a more tractable workforce, at least from a management perspective.


3) Experience is a series of non-fatal mistakes.

Not everything is a financial metaphor.


3) You might argue for versions of Bayesian thought as well (full Bayesian, MAP, ML).

I think more appropriate in this context is learning though. And I would suggest here multi-armed bandits (with every arm representing a pilot).

https://en.wikipedia.org/wiki/Multi-armed_bandit

There is tradeoff between exploration and exploitation that has to be made.

The goal in this case corresponds to a particular type of bandit. It is to postpone death for as long as possible by pulling the right arm. I actually didn't find this type yet (a mortal multi-armed bandit has a birth-death process of the arms themselves).

Edit: This is only about the learning process on the non-pilot side, as one of the other commentators already articulated.


Totally ruined that warm feeling I got from the anecdote with your damn logic.


There is a perhaps apocryphal story about the time when Mizuho Securities lost ~$225 million by attempting to sell 610k shares at 1 yen instead of 1 share at 610k yen. When asked whether he intended to fire the person who physically keyed in the order, the relevant department head stated that he did not intend to waste the firm's substantial investment in teaching them to act with due deliberation.


I wish that story ended with the developer of the trading software learning a lesson about user interfaces


This is a good point--I remember even the online game "Runescape" had a warning that would pop up if you tried to sell an item worth X for a very low price.

It would seem to me (somebody who knows squat about stock trading) that a warning screen would be much more beneficial for stock trading software than it is for an online video game.


You'd think that, but then what happens is users get used to the warnings coming up and consciously/unconsciously adjust to click by it without reading.

I'd imagine in the case of the stock trader, where quick reaction is valued, he'd have quickly entered whatever key combo is necessary to dismiss it the warning and made the same mistake.


Only if the warnings come up too often. I imagine it should be quite possible to define criteria that discriminate selling shares at a price of 0.0000016 yen from normal prices, by comparing with the market price, or the range of prices ever seen by the software, for example.

Alarm fatigue is an important design consideration, but not a reason to not put warnings at all (given that this isn't amenable to an "Undo" button, which is preferable when possible).



Alarm fatigue is indeed an issue -- but if the trader is repeatedly trying to sell for way below market price that often, then perhaps there is a much bigger problem...with the trader.


It ended up with order sanity checks being implemented for all manual trading systems at Mizuho. Things like blocking an order if it's more than 10% away from the market mid, etc.

You shouldn't fix an issue like this with UI validation alone - it needs to go between the component that creates orders and the stock exchange, so that it also protects against software bugs. For instance, a possible bug is a developer multiplying the order size by the lot size in the backend, when it has already been multiplied in the frontend, causing huge orders to be sent. Sanity checks can catch this.

Automated trading systems have traditionally been under a lot of scrutiny, and nobody in their right mind would run one without sanity checks and a kill switch. That incident taught Mizuho that manual trading can, in fact, also be quite dangerous :)


I've seen several versions of that story, often with the quote "Why would we fire you? We just spent $X million educating you." Probably some combination of truth and exaggeration, but it makes for a memorable story regardless.


I was not familiar with this story; that prompted me to research it. I found it interesting that this aviation incident had a positive outcome, in the form of safety innovations: the Hoover Nozzle and the Hoover Ring. Wikipedia states:

"A perhaps-undesired recognition is the Hoover Nozzle used on jet fuel pumps. The Hoover Nozzle is designed with a flattened bell shape. The Hoover Nozzle cannot be inserted in the filler neck of a plane with the Hoover Ring installed, thus preventing the tank from accidentally being filled with jet fuel.

This system was given this name following an accident in which Hoover was seriously injured, when both engines on his Shrike Commander failed during takeoff. Investigators found that the plane had just been fueled by line personnel who mistook the piston-engine Shrike for a similar turboprop model, filling the tanks with jet fuel instead of avgas (aviation gasoline). There was enough avgas in the fuel system to taxi to the runway and take off, but then the jet fuel was drawn into the engines, causing them to stop.

Once Hoover recovered, he widely promoted the use of the new type of nozzle with the support and funding of the National Air Transportation Association, General Aviation Manufacturers Association and various other aviation groups (the nozzle is now required by Federal regulation on jet fuel pumps)." [1]

[1] https://en.wikipedia.org/wiki/Bob_Hoover#Hoover_Nozzle_and_H...


In general, this design principle is known as poka yoke:

https://en.wikipedia.org/wiki/Poka-yoke


In terms of physical connectors, I've always referred to this as polarisation. The connector is polarised so that it can only be inserted in one orientation, generally by use of a slot and key.


And the problem it solves is Murphy's Law: https://en.wikipedia.org/wiki/Murphy%27s_law


This is a great design concept to know the name of! Thanks!


this is a great anectdote, but the reason i am replying to your comment is because of your user name

i am a huge fan of the turn of the century dancer isadora duncan and her lover who she first had a child with, the theatre set design theorist, edward gordon craig who she affectionately called endymion

a complete aside, but if you have, or anyone reading this has, yet to read duncan's autobiography 'my life' i highly recommend it to anyone and everyone

she is a brilliant writer, lived an eccentric life, and she was and her writing is imbued with a mad passion for expression and both life's hardships and joys


Seems like something my wife would like to read. Just bought her a copy. Thanks for the tip.


Good idea. I've just done the same. Thanks both


I can't resist chiming in with another Endymion reference: the Hyperion Cantos by Dan Simmons. Fantastic series.


That is some really quality nerd stuff right there. Thanks!


I'm now reading My Life because of this comment. I'm greatly enjoying it so far -- thanks!


In my experience, while someone might be extra careful not to repeat a mistake that has burned them in the past, making a serious mistake is often a sign that someone is careless, and more likely to make other, different mistakes in the future.


This doesn't jibe with my experience. As long as the person doesn't evaluate their mistakes in a vacuum they become more careful in general because they learn that things can bite you in the ass in totally unexpected ways.

So I'd say it depends on your environment combined with the individual. Someone who is apathetic and/or lacks critical thinking skills will probably learn very little beyond avoiding that specific mistake again.


*jibe (jive is a dance, jibe is agree)


http://www.dailywritingtips.com/gibe-gybe-jibe-and-jive/

Shows the other meanings of the homophones - English huh!


Right you are. Thanks, I was able to edit it in time.


One mistake, even one very serious mistake, means nothing. People are human, even the best of them are going to make mistakes occasionally. Be as careful as you want, it doesn't matter, literally nobody is perfect.

A pattern of making mistakes can tell you somebody is careless, or sloppy, or in over their head. But a single mistake? That's just inevitable.


> A pattern of making mistakes can tell you somebody is careless, or sloppy, or in over their head.

Or that you need to fix the process.


We all make serious mistakes from time to time. Yes, you too. If you haven't yet, you will - don't worry. Nobody gets an exemption from that.

When that happens, I hope the people around you are more forgiving than you are.


> making a serious mistake is often a sign that someone is careless, and more likely to make other, different mistakes in the future.

Or under external pressures they have no control over.


Right. I've been in a situation where I attained a huge amount of responsibility and authority in a short amount of time. I was learning a dozen new tasks simultaneously and getting something like 4 hours of sleep a night (military). I made some pretty big mistakes, and I think it's fair to assume that anyone would under those circumstances.


Agree. Some people simply have unrealistic expectations.


Well that's not the experience I've had at all. I bet you if you ask any senior engineer they'd tell you the numerous different ways they have really messed over their careers.

I think it's especially true in tech jobs where there is a lot learned on the job. I would be deeply skeptical of any tech company that takes a fire first attitude. That just tells me they 1) treat devs as disposable and 2) they get rid of everyone that has a chance to learn from their mistakes


It's probably not unrelated that American air travel is very safe because there's a no-blame, learn-from-the-issue attitude to problems.

More generally it is very difficult to build highly reliable systems of any kind without having a very open culture where you focus on exposing problems and fixing them rather than blaming the message.


I'm reminded of the time Tony Blair visited Silicon Valley to figure out how to replicate it back home. He was at a round table with tech royalty (Gates, The Sun guy, Schmidt etc etc). Steve Jobs was there too.

Everyone was chipping in with their theories about why the U.S and Silicon Valley were so good at what they do when Jobs lost his patience and butted in in true Jobsian fashion.

"Listen! Take a look around this table. Everyone here has a massive failure in their past. Big, epic failures. In the US, we think thats a good thing. In the UK you think it's a bad thing. Thats it."

Or soemthing like that, you get the idea.

Anyway, i've always liked that outlook regardless of wether it's true or not.


Link to the Bob Hoover story: http://www.squawkpoint.com/2014/01/criticism/


Nice!


This was so nice to read. I think back to my very first coding internship after my freshman year of college when I messed up big time. I made a bad mistake that ended up forcing my supervisor to put down everything he was working on for a full afternoon and do an emergency fix. I so vividly remember sitting down in his office, trying to just calm down and keep it together. He never got upset or annoyed (at least he never showed it), instead he listened to me explain what I had done, figured out what was wrong, showed me what my mistake was, fixed everything up, and calmly walked me through the entire process, turning it into a learning experience. At the end he asked me some questions to make sure I understood everything that happened and told me "everyone makes mistakes, don't sweat it, just make sure you learn from it" and that was that. I remember leaving his office that day thinking "if I ever make it to the point where I have my own interns, I'm going to do everything I can to support them and treat them as well as he treated me." A little kindness really does go a long way!


To follow up, have you had interns yet and how did you treat them?


I had an intern for the first time last summer! I tried my best to focus on being as encouraging as I could and helping him whenever he ran into trouble. He clearly was working very hard and doing his best, so I focused on being as supportive as possible and praising the work he did (which was very good!). If I'm being honest though, we was a really terrific intern and we never ran into a situation like I did, so I wouldn't say it was much of a test :)


Mistakes happen. I can deal with that.

People lying about their progress however, that is hard to deal with.


Having been in this situation and been supervisor to people in this situation, it's possible to not be lying but also to not have complete understood the task or have it mis-spec'd.

Having someone rip into for lying when the spec was terrible to begin with is massively discouraging and the OP lesson applies just the same there.


Oh no, absolutely. There is no doubt this happens in the majority of cases, so when two things don't pair up when you look at them you don't accuse people of lying. Of course this does land you in an incredibly tricky situation when you do find out there is a habit of systematic lying.

Thankfully that situation is resolved now, though it lasted for too long, and miraculously didn't fall off the cliff it teetered on.


when a supervisor gives a freshman work which can mess up erverything it is the supervisor's fault and not the fault of the freshman


Why do you need to assign blame?

My opinion is that trusting people and giving them responsibility is one of the best ways to let them learn and grow. The bigger the mistake, the bigger the lesson. As long as it is possible to recover, it's a calculated risk which may well be worth it.


Exactly! A sane supervisor should have put a rule that disallows deploying on Fridays. End of story.


Maybe he made the same promise you did back in the day?


As software engineers:

As beginners, we're over-confident in our ability, even if we actually suck and make lots of mistakes: https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect

The opposite seems to become true - experienced engineers (who have learned from their mistakes) seem to be extra paranoid. I've seen also older engineers that seem to be confident still, talk a bit game, but they just never learned. It seems paranoia is a great indicator of experience, and over-confidence/arrogance is an excellent indicator of a lack of learning. Not to say those are the rules, as there are certainly arrogant engineers that are excellent but I'd rather work with the softer one personally.

I know as I have grown I have become softer, not harder, as I realize my humanness.


Overconfidence is a huge benefit in playing the political game - getting people to take you seriously, managing up, interfacing with the outside world. Like it or not, humans are primed to respond to confidence; for most people, listing the dozen ways a system may fail signals that you shouldn't use the system, while for engineers, it signals that these are a dozen things to be fixed.

The trick - when getting to higher levels of technical management - is learning how to context-switch between being overconfident for the benefit of non-technical stakeholders and being paranoid with the technology so that the things that could go wrong don't actually go wrong.


Right, this is really a tricky game. I help run a nonprofit wiki farm, so a lot of our business is out in the open. We lost one of our clients because they were hanging out in the IRC while we were discussing the cost of an upgrade and the technical issues facing it, as the exit question revealed that they thought we were running out of money.

Now of course donations in the past few months have been roughly double our expenses -- information which is also public. But it's very hard to project confidence during a problem solving session, because it gets in the way of actual problem solving.

I wish I knew where the balance point for secretiveness and openness was for public organizations. But given that all of the candidates for U.S. President seem to be at a bad place, I'm starting to wonder if a perfect balance point even exists.


This is why I don't really want to get back into management, even though sometimes I think at some point it will be the only thing I'll have the patience and mental capacity to do. I saw the best and worst of myself when I was a manager- the worst is what scared me. But moreover, I learned after I stopped being a manager that I didn't deserve the accolades I got. If you were to take the best manager in the world, give him/her a shitty team, a hell of shitty code, shitily designed application and infrastructure, a shitty relationship with the customers/users, shitty support team, and shitty project/product/upper management, and you don't let that manager work to fix these things, they'll quit or fail, guaranteed. If you give them the best of all of these things, they will succeed, guaranteed. Being a good manager- knowing how to manage well and doing it, can be critical. However, it is nothing on its own.


One big difference is that a good manager can take an average team and build a good team over time. If you can't do that, you aren't a good manager.

A manager that can't tell who on their team is good will (in the long run) take an average team and turn it into a bad team.


>Like it or not, humans are primed to respond to confidence

Exactly. My feeling is that while perhaps many of us learned this the hard way, well at least I did--from experience--in hindsight it seems obvious that our emotional impulses and intuitions, while albeit valuable, are not fine-tuned to success in technical careers.


I think intuition and emotional impulses is immensely important for communicating with non-technical stakeholders. They haven't learned the art of emulating a computer so we've got to be a human being. IMHO.


Yea, I agree with that actually.


Fun fact: Dunning & Kruger measured perceived vs actual performance on very simple, and somewhat subjective tasks. Ability to recognize humor was one of them.

It's strangely difficult to find this important piece of the methodology. Wikipedia's article doesn't even mention it, and this context is fairly critical to understanding the effect.

I think you're right that some amount of inexperience and youth makes the kids overconfident, but what has been demonstrated is that the task difficulty is a large determinant in how good people are at estimating their own abilities, and Dunning Kruger only applies to easy tasks [1]. For very difficult tasks, the Dunning-Kruger effect actually reverses and becomes Imposter Syndrome [1][2]. Software engineering may fall into the latter category -- something that is difficult enough that, statistically, on average, beginners are actually pretty good at knowing they can't do it.

[1] http://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-k... [2] https://en.m.wikipedia.org/wiki/Impostor_syndrome


What do you mean "reversed"? DK says that both low and high skilled people estimate themselves as close to average, due to the obvious bias in one's available data


I mean that for difficult tasks, people who are unskilled rate themselves as unskilled, and people that are skilled rate themselves as skilled.

"Our studies replicate, eliminate, or reverse the association between task performance and judgment accuracy reported by Kruger and Dunning (1999) as a function of task difficulty. On easy tasks, where there is a positive bias, the best performers are also the most accurate in estimating their standing, but on difficult tasks, where there is a negative bias, the worst performers are the most accurate. This pattern is consistent with a combination of noisy estimates and overall bias, with no need to invoke differences in metacognitive abilities. In this regard, our findings support Krueger and Mueller’s (2002) reinterpretation of Kruger and Dunning’s (1999) findings. An association between task-related skills and metacognitive insight may indeed exist, and later we offer some suggestions for ways to test for it. However, our analyses indicate that the primary drivers of errors in judging relative standing are general inaccuracy and overall biases tied to task difficulty. Thus, it is important to know more about those sources of error in order to better understand and ameliorate them."

Burson KA, Larrick RP, & Klayman J (2006). Skilled or unskilled, but still unaware of it: how perceptions of difficulty drive miscalibration in relative comparisons. Journal of personality and social psychology, 90 (1), 60-77 PMID: 16448310


To clarify, I think what he is saying is that the effect exists on both ends. On the low skill side it's over confidence and on the high skill side it's imposter syndrome. It's the same effect. You are both saying the same thing.


Yes, and that makes "being kind" a lot harder.

Many people reading the article think of a nice person that humbly tries their best but makes a bad mistake. But what if they're a hugely over-confident asshole?

I mean, not everyone likes everyone, so when someone you don't like to begin with is arrogant, you might easily label them as an asshole.

I think the deeper lesson of the article is not to consider other people assholes, and to be kind to them even if you don't like them at all.


>But what if they're a hugely over-confident asshole?

Then they have absolutely no business working on a team effort.

They never should have been hired in the first place.


I don't agree. I've seen some very junior people change their attitudes rapidly (for the better) when working on a team; if we wrote them off immediately we'd lose out on a lot of promising people.

I was also one of those developers and have since changed my ways :)


Our definitions of "over confident assholes" may differ, perhaps? :)

Absolutely however I do agree with you. If a young developer doesn't integrate into a team role immediately, that's no reason to not hire him.

A lot of great developers are so great because they've spent years alone with their computers just cranking out code. So the best ones won't have the greatest social skills.

But if the kid's still having trouble integrating into their team role after their 6 month probationary period, for example, I personally wouldn't vouch for them any longer.


If there are five equally inexperienced applicants in the wings that will get along with everyone without having an attitude-smackdown first, then that's absolutely a reason not to hire them.

The worst problems I've ever seen on teams spring from people that are arrogant and standoffish, followed closely by people that are too dogmatic about technical minutiae and best practices in the face of situations that require flexibility (my classic example is the prototyping team that spent two weeks setting up an automated testing and deployment infrastructure, even though they only had six weeks to do their game prototype).

I'd take a pleasant junior that's really green but is happy to learn how to code better over a more technically proficient asshole that needs to have manners jammed down their throat any day - tech skills can be taught to a newbie, personality cannot. People have had a couple decades to cement personality habits as junior programmers, so those are mostly fixed, whereas tech skills are super fresh and malleable.


Yes, we might always change someone: a friend, a spouse, an employee.

But that's a risk; depending on that is ill-advised.


Well, part of my point is that assholeness is subjective. Moreover, a certain group of people seem to think there are a lot of assholes, while the majority gets along with most people.

Of course, there are genuinely poisonous people, and I agree on your points with regard to them.


And yet here they are.

With respect, this doesn't tell us what to do if we run into a hugely over-confident asshole.


>With respect, this doesn't tell us what to do if we run into a hugely over-confident asshole.

Fire them, if you can. Before they poison your entire talent pool with negativity.


When I was 19 I started working as a video conference support person for Europe in a multinational. This meant interfacing with the highest people in the company, where a failure could make an insecure CEO or similar complain to the IT manager screaming how this was shit etc. this was using multiple ISDN lines making international calls, so it was prone to failure once in a while. After the first instance, the manager told me to not worry, but to just let him know when something failed so he had answers when the angry user came complaining. This was my first learning experience in this.

A few years later, in the same company, I was managing the middleware servers that handled all communications for the sales people in Europe. Here, when there was a failure, the sales people would have to fax in their copies of the data if there was a failure, so even more pressure. This is where I really learned to both appreciate the criticality of my mistakes as it could make several hundred people redo their work. However, the most important thing I learned with this was to slow down, and pause for a few minutes so that I could explain what was going on to the non technical managers and team leads. This was more important sometimes then actually solve the problem.

Both of these instances and a lot of others that happened over 11 years in that company thought me ot really double, triple and quadrouple check any change. It created a healthy paranoia, but also taught me that mistakes are ok, as long as you learn from them, but also, lean to communicate with higher ups that could not follow the technical jargo. I think the communication part is something that a lot of very confident people have problems with. However, it's very hard to teach this to younger techs, without letting them fail, so as a senior tech, I have seen problems from new juniors, but I have let them actually make the mistake, and feel the stress and pressure that they have messed up. I have found this to be the quickest method to teach responsibility, while standing by them and their mistakes.


Just as a note about detecting mistakes and not correcting them, you choose mistakes that does not hurt the company, and of course you try to show them the mistakes before the cause harm. However, sometimes you catch the mistakes late, and that is what I was referring too.


I think that I have become softer and harder. Harder in the sense that I have no tolerance for bullshit.

Softer in the sense that I want to foster an environment of respect. As a lot of the senior leadership that I've worked with has retired, I've noticed that the 30-40 year old crowd now is more brutal.


The self-confidence of amateurs is the envy of professionals


>It seems paranoia is a great indicator of experience, and over-confidence/arrogance is an excellent indicator of a lack of learning.

This is certainly true, and it isn't limited to the field of engineering.


This was literally the biggest issue for when I started at my first job. We had a team member who was terribly condescending and talked down to everyone but especially me. You could tell that he hated the fact that he was on a team with a junior developer and took every chance he had to made sure I knew I wasn't as good as him. It makes a terrifying environment to ask questions because who knows what kind of response you're going to get.


Maybe I have just had bad experiences but a lot of developers do this thing where they answer a question with a snarky question. Sometimes it's annoying to be asked basic questions but behaving in a condescending way lends to people not asking questions at all and not communicating issues. It doesn't matter if the person is Jr. or Sr. I've seen good capable Sr. developers miss stories because they would rather be silent and struggle trying to research and figure something out instead of asking questions due to the snarky competitive nature of the team. At my last company I became the default mentor of a non Jr. team member because his real mentor, who was also my mentor, was so condescending the guy was terrified to approach him. Having been in his shoes I had no issue repeatedly helping him get his build working or helping him understand the undocumented proprietary framework etc. A little empathy goes a long way


I think it comes down to trust. In a lot of teams where the trust is poor, one can't ask a question without your competency coming into question in a subtle way: many developers will latch onto it and try to exhibit their business-specific or technical knowledge in a form of one-upmanship. So they have turned it into a competition instead of a conversation.

A good way to detect these environments is when you observe those who have been there longer not asking obvious questions. That usually means the trust issue is there. When you start asking those obvious questions, at first you get that feedback loop of slight condescension. But then others start asking questions and you often get a fruitful conversation.

Having worked outside of the startup world and in the startup world, I think this is a little more prevalent in the startup world because there is another axis besides experience involved in these conversations: how long the person has been with the startup. It's common to have an official or unofficial hierarchy based on experience but in the startup world, there is another hierarchy based on how long you have been at the startup. That additional axis means it comes up a little more in the startup world (in my experience so far).


> Maybe I have just had bad experiences but a lot of developers do this thing where they answer a question with a snarky question.

Extends far further than the workplace. A few minutes in some of the more popular IRC channels on Freenode, you'll get the same experience. It's extremely frustrating, especially seeing it happen as an outsider to communities you like.

I feel like, in the same way that power generally corrupts people, "feeling smart" can corrupt the same way, coming down to the same attitude of feeling like you have an advantage over other people. Regardless of whether you actually are smart or not.


On one hand, I get it. Especially the popular channels on Freenode or places like the Arch Linux forums can get swamped with people who could not care less for rtfm'ing. Ungrateful, entitled, obnoxious folk, robbing people of resources and time who're putting in work for free, at least in a lot of cases. Sucks. I usually get splendid answers from these sources because I don't even bother unless I have a specific question or error, logs to back it up maybe. Yesterday I had somebody on #Openvpn help me with an issue in three lines. Nice.

But if the snark is heavy without being warranted at all, it more than pisses me off. It reflects poorly on whatever you're representing and it's just bad communication. If all you have to offer is a lmgtfy.com Link, don't bother. Those immune to that train of thought won't, the others already have.


And that means the developers will all try to fix their own mistakes without bringing issues to the wider team, leading to a worse solution to problems in general. Premadonna "10X" developers might write great code themselves, but if they're pushing the quality of everyone else's code down then the overall effect of having them on the team is a negative.

Anyone who treats someone else on the team in that way, regardless of seniority, is demonstrating that they're not really interested in the project as a whole. It takes a group of people to build something complicated, and someone who is antagonistic towards other people holds back the effort of the group. You're better off without them.



Maybe he meant pre-1958 developers, in the same way we used to use "BC" as "Before Christ"


"Madonna" is the Italian name of Christ's mother. The singer chose her name for that reason


I had a similar experience at my first job. I was given a lot of access to important, core functionality like server configurations (e.g. using staging vs. production API servers and DB config), but mistakes weren't well tolerated. I've always been rather frustrated that I was given this unsupervised leeway yet judged when something inevitably went wrong - what did you expect would happen with a junior developer given master keys to all of your stuff? I still don't understand the thinking behind it.


I've always felt this attitude towards giving trust to the new hire is a good thing. It remembers me to something I read not so long ago where a Perl core contributor makes the same thing with collaborators.

What I don't find ok is to blame a person for an error going to production. Specially when the error is caused by the lack of supervision to the new hire code (code reviews) and probably QA.


As a junior, I've found a similar experience as well. Programming environments seem to have a really unprofessional attitude when it comes to this.


Oh his behalf, I apologize. I literally started laughing at a junior's code once but in my defense, he had spent 3 years at Amazon.


I'm sorry, but you have no defense. Laughing at a junior developer'a code is completely inexcusable.

Simple fact is, if you worked for me, I would have fired you for that. Junior devs are supposed to do bad things - that's why they aren't senior devs.


"that's why they aren't senior devs"

Amen. I don't know about firing, but I'd definitely be having a conversation about proper mentorship. We hire juniors because they show promise, but need to learn. Being snarky and shutting them down doesn't just hurt the individual dev – it hurts the entire company.


Firing for laughing? Did you forget the point of the article? Be kind. Don't hand out harsh punishments for mistakes. Teach instead. (Only fire if the person is immune to teaching.)


Let me relay a story about people who laugh at and mock other people's work.

My wife used to work as a copy editor at a major newspaper and worked with one of those "laugh at and make rude comments about other people's work" kind of people. The copy editing checks had to be checked by another copy editor before it was allowed to be forwarded to the layout team for placement in the news paper. How the story checkin and checkout system worked was that your submitted checks were anonymous to other team members. So, there was this one guy on copy editing team who openly mocked and call other people's work "stupid" and "idiotic". He could never say the person's name that he was talking about, but since every copy editor was in the same room, you knew he might be talking about your work. Everyone hated the guy and everyone complained about the guy. My wife would say how much everyone just hated working with the guy and he was just one of those people that made the job unpleasant.

Then layoffs came around (Newspaper in the Internet Age). When this guy was laid off, champagne bottle were opened up and people celebrated. You know when someone is bad when there is a "sorry to see you go" bar get together and they are not invited. A strange thing happened though, productivity went up because people felt better about submitting their work for final approval. Less copy editor mistakes, less stress, more learning, more openness and more engaging with other team members happend. Barriers between teams fell and even with layoffs, people felt better.

It turns out that the person who mocks other people's work, make for a crappy work environment. You can be tolerant of failures on technical levels, but failure in personal level should not be tolerated. Be pleasant to work with, or should you should be fired.


hmm, seems a bit harsh. For instance if your guy had done that once and been taken to the side and told "stop, shut up, never do that again, next time you're fired" then (a) the work environment would have improved much faster and (b) he might have never done it again. Firing instantly for one harsh comment/action seems like foolishness, just like keeping a known problem person around forever (as in your story) seems like foolishness.


Fireable offense? Software Development is a profession, it's not coding school. Junior Devs are expected to have have an education, written code before, and know the basics. If they write something so crazy that it induces a chuckle, it's probably pretty bad. By all means tell them how to fix it, but don't mask the fact that the job they are getting paid to do expects them to know how to do this.

Maybe an analogy would do better. In the Army, Soldiers go through basic training and are taught just that - the basics. Once they show up at a unit, they will have lots of questions and lots to learn. But if they show up with their name patch velcroed upside-down on their uniform, I guarantee their Sergeant is going to chuckle before they tell the Soldier to turn it right-side up.


Yeah, there's an obvious lack of details in the story that should preclude judgement of the poster.

I've seen people write things like: if (true == true)

I won't be mean to you about it, but I can't promise I won't chuckle a little before explaining why that's unnecessary. If you've been working in a professional setting for 3 years and are writing code like that, then yeah, I might struggle to be empathetic.


> I've seen people write things like: if (true == true)

Which, if you make that an === in JavaScript, can actually make sense in some situations :)


Can you give an example? I can't think of any situation to literally write if (true === true) rather than just if (true). (If anything.)


Sorry I not only mis-read but mistyped and I can't edit or modify my original post. Please ignore it :(


I've never waded into JavaScript, so I'll take your word for it. I've heard some...surprising...things about JavaScript, so I'm not really taken aback that this might exist.

I'm referencing a time when I was in school, and most everything was taught in Java, where that doesn't make sense. This was written by someone who was almost finished their degree and had already started at their professional job. Definitely junior, but I still feel like that's stretching the acceptable level of noobishness. That said, I have an EE degree, so maybe had to solve / reduce a lot more boolean algebra expressions than a straight CS major?


Your comparison is apple and orange if you think a person's education in software is comparable to an army. in army, the training is real hands on. rigid. while learning how to code is never a real hands on, always partly figuring out on your own, thus each person having more various understanding/competency on how to do the job.


I think you're quite possibly judging the guy too much based on a simple internet comment.

His statement could mean anything from cruelly laughing in the face of the junior dev during a face to face code review to having a quick heh under his breath from the privacy of his office at the developer's use of "if (foo === true)". Or anything in between.


Perhaps, but the "but in my defense, he had spent 3 years at Amazon" quip seems to confirm that this guy has a poor attitude.


> I think you're quite possibly judging the guy too much based on a simple internet comment.

There are such judgements in every thread.

It's even better when it is paired with the famous "I would not hire you" (or its variation "I would fire you") although the conversation does relate to this at all, the OP has shown no intent no be hired in any company, let alone the answerer's company, and, icing on the cake, the answerer is not in a position to hire anyone.


If it was a completely private laugh, seems weird that they would apologize for the previously mentioned jerk.


Due to open office layouts there is no such thing as a private laugh.


Again, the guy spent 3 years at Amazon and was talking himself up big. I looked at the code and started laughing. He wasn't a junior in title, in fact, he was on his way to being senior and had his eyes on an architect position.

That is my defense and the only time I've laughed at someone's code. I guess the combination of him talking himself up big about all the amazing things he did at Amazon and the code vomit that he actually produced was too much for me.

I regret it, but he shouldn't really have been that junior after 3 years?

Though I guess I still write pretty shitty code now too on occasion.

Alright, I'll take the karma L in this case.


You are an asshole if all it takes to fire someone is some laugh. Everyone shoukd be able to laugh at everyone's code. It's just a code.


> It's just a code.

Well, no. It is the product of someone's work. Someone who is trying to learn and should be supported and taught how to make it better and why it is not as good as it should be.


Laughing at someone's code is like laughing at someone's painting or their novel or really any other creative endeavor. It's not ok period.


My team playfully makes fun of each other's code all the time. Every once in a while, someone does something stupid in code and we playfully make fun of that person, everyone on the team takes it in stride and our code is better for it.

Hell, every now and then I stumble across some idiocy in the code base and think "LOL, whoever did it this way is an idiot". git blame. "OH, hahaha, it was me, what an idiot past me was, amirite guys?" Everyone on the team takes levity about mistakes very seriously ;) It's healthy.

Mocking condescension on the other hand is a different thing entirely.


Self-deprecation is classy. Other-deprecation is pathetic


If someone's work is poor enough that a reasonable person might laugh at it, then there's nothing wrong with that. Hence the phrase "laugh at our mistakes". It doesn't mean you are a bad coder/painter, it just means you made mistakes that were a little funny and can learn from them like anyone else.


There's a difference between laughing at your own mistakes and laughing at someone else's. People who are ridiculed for making mistakes are incentivized to hide their mistakes rather than learning from them.


We "laugh at our mistakes" not "laugh at other people while they are making the mistake".


Is it? Or is it more like laughing at someone who built a table with 3 legs and wonder why it fell over? Or someone who fixed a car but forgot to put the engine back and wonders why they can't go anywhere?


Senior devs make mistakes too. I think your tone is rather inappropriate given that the poster seems aware of the inappropriateness of his response and is apologizing for making that sort of mistake.


While I admire the sentiment, I've NEVER seen an asshole fired for belittling juniors/co-workers.

Harsh, withering treatment is quite common not just with developers but in all technical fields. I wish things were different, but that's the reality as I've seen it.


At more than one place where I've worked with at least semi-functional management, people with attitudes like that tend to get moved to special "one person" teams and then are given a long stream of crap work (or at least work they feel is beneath them) until they get bored and quit.


I've seen it once, but the guy cursed and laughed at the team architect when we had a meeting instructing us to start writing tests and adhering to a certain level of code coverage. Seeing the guy literally begging for his job after witnessing him demean and belittle people for months was kind of eye opening to me.


Hopefully after you make the mistake of firing someone for laughing, your boss is kind to you (as in the OP) and lets you keep your job


Trading one form of radicalism for another. That's the spirit...


Maybe I don't get the joke, but why is this a defense?

(I had a collegue at my last company who worked at Amazon before. His code was just fine.)


He said "In my defense", justifying laughing at the junior dev because he (the junior dev), had 3 years of experience at Amazon.


Right, which suggests that coders from Amazon generally have rubbish code, excusing the laughter.


No, it means he expected that particular person's code quality to be better than that, because he lasted at a 'Big Four' company for three years and thus should have decent coding chops, but apparently didn't, thus the incongruence made him laugh. It doesn't mean all Amazon coders are shit.


The original wording is ambiguous. I too read it as if OP had some issues with people coming from there.


I like figuring out how to look at a sentence how others do (like those images that can be viewed two ways). But I'm having trouble with this one. Amazon devs being generally bad would not be suitable for use as a defense ("in my defense") for his behavior. I think it would require the assumption that he thinks that Amazon devs are generally bad as well as the assumption that he thinks that laughing at them is something that's generally permissible.


I too originally read it as saying that it was expected to be bad coming from Amazon. And the "in my defense" part is that, if you know the coder is from Amazon, then you understand why the code might actually be truly bad enough to elicit a chuckle (as opposed to just chuckling at normal bad code). In other words, it's like saying "in my defense, it was super bad code".

That said, I think the interpretation that "he had 3 years experience at a big company, he should have been better" is probably the correct one.


> I think it would require the assumption that he thinks that Amazon devs are generally bad as well as the assumption that he thinks that laughing at them is something that's generally permissible.

They are so bad that laughing is permissible.

Maybe you just don't spend enough time around arrogant people to interpret this sentence this way :)


I had expected better code to result. It was borderline comical.


That is completely unacceptable behavior. If you were in my company I would petition to have you fired. I hold a strict no asshole rule, life is too short to work with people who would do something like that.


Sounds like I wouldn't want to work with you anyway. Life isn't rainbows and unicorns.


Then you're as bad as he is. Especially given that he's already acknowledged he shouldn't have done it. Indignation doesn't make you right, it just makes you stubborn.


Wheaton's Law violator here. Don't be a dick.


Laughing at a junior's work when you are their mentor is a firable offense in my eyes.


People ready to fire someone for laughing should not be allowed to work with other human beings at all, in my opinion. What's going on with this ready to be offended for wind blowing the wrong way culture and the fragile egos?


It's not just about laughing. It's about laughing at someone's work in a derogatory manner. I hope you can tell the difference and understand the extremely negative impact the latter can have on culture.


I hope you can understand the extremely negative impact firing someone over a simple mistake can have on culture.

Laughing in this situation is merely inappropriate. Firing someone for being inappropriate once or twice is incredibly toxic behaviour.


I disagree. Firing assholes is never bad for culture. On the contrary, it increases morale and makes the workplace better.

Keep in mind that I don't categorize laughing at someone's work as a "simple mistake." Bugs can be simple mistakes. Offensive jokes can be simple mistakes. Laughing at someone's work however is deeply troubling behavior that actively undermines trust and discourages cooperation in the workplace. That's why you have to kill it with fire.


> Firing assholes is never bad for culture.

Firing somebody based on your own subjective opinions is toxic for culture.

I've worked on amazing teams with plenty of good natured ribbing and I've worked on great high performing teams where you could say "this code is rubbish, you can do better". I've also worked on teams where saying that would really hurt people's feelings and impact moral.

Put aside your own pre-conceptions and look at how your team responds to an event/situation. That's the only way to build a high performing team.

Sometimes that will mean laughing at someone's code is toxic and needs to be addressed, other times it will be a non-issue and addressing it creates an issue, and other times it can even be a good bonding exercise.


I already specified that this isn't just about laughing, but about laughing at someone's work in a derogatory manner. You seem to be saying "but a laugh can mean other things!" and it's kind of besides the point.


No. I'm not. I'm saying people, cultures, and teams operate differently and assuming that laughing at someone's work is toxic rather than looking at the actual impact to the team is unprofessional and a sign of poor leadership.

I worked in a team where if you broke the build you had to put on a clown nose for the rest of the day. It worked well and was a bit of fun. We still catch up a couple of times a year even though I left that team more than 5 years ago.

I've worked in other teams where pressuring a team member to wear a clown nose would be harassment and deeply unsettling.


Everyone seems to be taking this too far--parent just mentioned a laugh. It could be anything from a well meaning jab to a mean put-down, and I'm reading it as just something funny for the junior to learn.


Wasn't his mentor. Just a guy reviewing code.


Nice little read. There is that little answer in the back of my head of:

"It is a good question why I caused this problem. It is weird to think there is a competent company that has been around for so many years, yet they have no procedures in place to stop this from happening. You would think that any changes that could cause downtime on a clients website would go through an automatic test suite and only after passing all tests would it be tested by human QA and finally made live. In this particular case I am the one who has made a mistake, and being a sensible person who is eager to constantly better myself, I will try and learn from this error, however I work alongside dozens of other people on my team who like me, have every chance of making a mistake at some point. It seems like a bad policy for us as a company to say that we should expect every person on the team to break a clients site and then rely on overtime from other people to fix it. So of course I will try and do better, but this is not fixing the root of the problem, the root of the problem is something that needs to be fixed at a much higher level. Have you, as my boss, not considered this problem already? What has the company learned from this? I would be happy to be part of the team that solves this problem by creating unit tests and creating policy to avoid this."

Of course I purposely make this slightly stand-off-ish, over the top, and written from a very specific perspective, to illustrate a point. But the reason I do this is I absolutely agree a single developer shouldn't feel bad about making a mistake (they should try not to, but these things happen), but the company should put measures in place to minimise the impact of mistakes. When the company fails to do this, the company has screwed up a lot more than the developer.


Think of what you're saying on a meta-level here. You would basically be telling this guy, "I fucked up, but I'm blaming you for treating me like an adult instead of a child. You screwed up by giving me enough autonomy that I could make a mistake." That might not exactly be a smart career move..

On a more personal level, if someone shows you kindness in the face of a mistake, the last thing you want to do is throw it back in their face and go on a tirade against them. That's a very quick way to make sure nobody ever wants to work with you again.


I disagree. If you have something that is of significant value to the company, you need to hedge your risk. Automation is a particular hedge. People fail, and while processes do too, they often fail less.

Would you be comfortable moving something fragile by hand that is worth a lot of money? Maybe. Would you prefer if the fragile item being moved was done with an automated process that was shown to best protect fragile items? I would.

"I'm really sorry I caused that car accident." "That's okay, next time take Driver's Ed and don't drive on sidewalks." "Thanks, I'll make sure to do that... by the way, why can I even drive on the sidewalk in the first place? Why wasn't it required to take Driver's Ed?"

I'm very obviously using hyperbole here, but questioning a process, to me, often shows a level of maturity in engineering. It's not blame-shifting - you clearly still made the mistake - but it's fixing a root cause.

I do think it has to be brought up carefully and with the right tone at the right time though.


I think you've hit the nail on the head at the end with "right tone at the right time" - the right time is almost certainly not when one has screwed up.


Or if you think it can and should be fixed at that institutional level tell them what you've learned to do better on a personal level and then volunteer to help lead the effort to create the automation/systems to prevent anyone else from making the same mistake in the future.


Well, "process" and bureaucracy tend to march hand in hand. I think it's easy to get carried away with creating "processes" (i.e., additional bureaucracy) when sometimes people just need to show good judgement. Pushing code right before you leave for the weekend is just bad judgement, and the guy learned his lesson. Heavily bureaucratic institutions laden with processes are not exactly known for their efficiency or pleasant work force.


OTOH, at the most sophisticated organizations, being afraid to push on Friday demonstrates an embarrassing lack of QA infrastructure.


Reminds me of an old VP of mine who came thundering out of his office, absolutely livid, shouting, "What idiot gave me permissions to delete the source backups???"


From a security perspective, there is the least privilege concept.

Why does the VP have that kind of access by default? I understand having a separate account if needs are there, of having some sort of priv-escalation.

In my new job as network engineer, all the RH boxes have SElinux off. I don't get it, nor can anyone give a cogent answer. And up 4 floors, we have a selinux kernel Dev.

I'd greatly like to even limit roots potential damage against users' data (in this case, DBs).


> Why does the VP have that kind of access by default?

Two common reasons. First, because the VP or someone in a similar position resented not having such access; it's sometimes hard for people to accept that people who work "for" them hierarchically have permissions they don't. And second, because either they or the person who previously held their position had a legitimate reason for such access in the past, and didn't drop it when they no longer had such a reason.

But yes, even if they have a legitimate reason for such access, they shouldn't have that permission all the time, only when they intentionally don that hat.


I read what he was saying differently. When errors like this occur it's very much worth evaluating what could be done to avoid it in the future beside me just learning my lesson. There are many general team behaviors (e.g. Automated functional testing, automated load testing, static analysis, code reviews, non-production environments, etc.) and things specific for the problem at hand (what test is missing that would have caught this?) that must be considered after a major issue. If these aren't evaluated then there it is likely they will happen again. I don't believe these are being treated like a child. Instead these are the team deciding to be adults by avoiding emergencies and being more confident when releasing.


Why not both? Of course you can accept responsibility for this mistake but then suggest creating a testing suite to run commits through before making them live?


Yip, that's exactly what I did today. I made a mistake which borked layout on one page. I thought I'd looked at it, didn't, and it got deployed. I'm now suggesting the company use continuous integration and some automated screenshot type tests for catching these issues.

I hate making mistakes, but if more robust procedures are the result, well, it's an overall win!


Good point. It's important to take responsibility even if you think there should have been guardrails.


I worked for a boss once who essentially engineered this kind of problem to exist: he knew not having automated testing etc. was a problem, and instead of asking his (too junior) programmers to build up a test suite, he decided to let them fail and discover for themselves why writing up test suites is a good idea.

His reasoning (later revealed) was that you can't give people fish and expect them to become expert fishermen. They have to experience hunger and ask to be taught to fish. If people don't have the soul-crushing experience that teaches them why something is really important, they never really internalize why it's important. That the reason why we appreciate that this stuff is so important is BECAUSE we suffered through something that taught us that it was important, and that if we don't give junior folks that same kind of experience, then good practice just becomes something on the list of priorities and not a moral imperative.

There's something to that, but at the same time, it's basically a defense of institutional hazing on the investor's dime. So take it for what it is.


> His reasoning (later revealed) was that you can't give people fish and expect them to become expert fishermen.

I think you can, it's just not nearly as efficient. College and trade schools are essentially about mixing a very small amount of actual necessity (deadlines, tests, etc) with copious amounts of being told what's best and how to accomplish something. This works well because it's both more palatable to a wider audience of people, and it accounts for people's different interpretation of events. There may be multiple things to learn from any particular failure, and without some guidance you may come away having learned few or none of them the first time. Being told what to expect before hand, or what you could have done to mitigate problems afterward, goes a long way towards making sure you consider all the useful aspects of the problem.

Will you learn any one single lesson as thoroughly or as fast as you would in the real world when it affects you so much? Likely not, but then again, that assumes you actually saw a solution to the problem, and that still might just be one aspect.


There is a cost in not having certain procedures in place. On the other hand the red tape has a cost too.

A single mistake shouldn't lead unconditionally to a mandatory multi-step process that involves multiple people and may take several months to finish. The examples of the "cover your ass" decision making overpowering the common sense engineering are abundant.


That's true, you can definitely over-correct. I guess if I was the supervisor in this case, I'd be doing some analysis with the developer after the fact. Were procedures correctly followed? If not, use this event as an example of why the procedures are in place. If they were followed, was it a brain-dead simple error? If so, then obviously there is something wrong with our test plans. If it was one of those errors that turned out to be a perfect storm of environment differences, unexpected inputs, code side-effects or what-have-you, then there probably is no quick solution and the imposition of some overbearing system to try to limit something that will probably not happen again would be overkill.

It's always best to do an after action analysis to see where things broke down, I think. Changing procedures or adding protections might be warranted, as long as it's not the "get my permission before ever putting a Y in that field again" kind of enhancement.


It depends on the project. Is it a web application for keeping track of daily tasks? You probably don't need that much rigor. Are you building something that if it fails will harm/kill/cost people a lot of money? More rigor is needed.


I feel where you're coming from, but in reality it's not like any test suite can guarantee to prevent downtime caused by bad deploys. All it takes is a little networking related bug, something that differs ever so slightly between environments, whatever, and your site could be down.

I agree with the other replies that while you may be the 'technically right dev' kind of person, it still comes of as being a dick and you will likely not get a positive response from something like this, especially if you caused it.

What makes it more reasonable to say it's the company's responsibility to protect you from fucking up the site instead of you just being a good developer, and not fucking up the site?


It's not about being a good cowboy developer, it's about beijg a professional engineer who builds systems that work.

I don't expect the folks who built my house to detach and reattach the door pefectly every day, I expect them to build a house that works.


Exactly. If you're one mistake away from a disaster, you already lost.

Ideally you would have either test gates, or a staged rollout with automated rollback.


One way to avoid Friday deployment issues is to go to the pub. Obviously you need to spend all afternoon there and not be tempted to go back and deploy, otherwise issues may be compounded! It seems to be a common mitigation technique in some shops I've worked at ;)


I can confirm it is an approach taken by different companies out there.

I used to work for this company where every single Friday we would go out for lunch and get tipsy (wine with the food, some digestive liquor shots just at the end, then maybe a mixed drink or a beer at another bar) because we were the only team not allowed to go home early on Friday... for no reason. Mind you, we were some kind of internal IT team and there would be no one to request anything from us, so we never had urgent stuff to do.

Back in the office we would enable the "fire extinguisher mode" which meant "only move if there's a fire" and watch silly videos in Youtube, have some coffee with Baileys because why not...


I like the way you drink ... wooops I mean 'think', I like the way you think.


I hope Kevin learned not to let a junior dev deploy to production on Friday.


Well, he'd have to sooner or later. Better start learning as soon as possible :)


Nobody should be deploying to production on Friday... Nobody.


My company just had to do it today in response to a critical security issue we identified in production.

Saying that nobody should ever do it isn't realistic, sometimes things go wrong on a Friday, sometimes you can't afford to wait 3 days to watch them become even worse.


It's a good rule to live by, and like all rules it can be overridden when the circumstances require.

It's good to instill in your team that Friday deployments should be done only in special circumstances, or they become a habit (and people inevitably end up working Saturday). But yes, sometimes it's necessary.


IMO, if you deploy on Friday, you are promising to be available for fixing it on Saturday... which makes it not really a Friday in the sense in which it's meant.

Don't deploy when you won't be around to support it is a better description; but it doesn't roll off the tongue as easily.


So I should just leave a security vulnerability rather than fix it on a Friday, because I won't be available Saturday?


If it has the potential to take down your site? Unless you have someone who can fill in for you, absolutely.

Really, it will come down to a cost/benefit analysis. Are your chances of the site going down due to being hacked over the weekend higher than the chances of a last minute update taking down the site?

The answer is almost always no (much to a sysadmin's chagrin). If the answer is yes (i.e. another heartbleed), then you are probably going to be working through the weekend anyways.


Read to comprehend, not to argue.


Great, now what am I going to do with my life?


This should on the top of every page here.


See if you like this version better:

"Only deploy on Friday if you like working weekends"


Yay pedantry!


I don't think I was being pedantic. The parent poster was very clear, there was nothing there for me to overstate.


I'd rather have a robust infrastructure and enough confidence that deploys won't break anything that we simple stop paying attention to the day or time!


And you don't get to that ideal state by breaking production on Fridays.


Friday is the best day to deploy to production. If something screws up, you still have the weekend to solve the issue before the big bosses are there.

On weekdays you have the added pressure of other work.

(Of course depends if your business is mostly or evenly loaded on weekends as on weekdays. In most businesses I've know it's usually the lower load/customer visits period).


No, the best time to deploy is when everyone you need to fix the issue is in the (metaphorical) building already.

You should not be pulling people in from their time off to fix shit.


And if they've done their job correctly, you wouldn't have to -- even if you deploy on Friday.


+1. Some systems aren't used heavily on weekends so it acts like a beta test.


In my experience this strategy has not worked out very well. When we tried to deploy risky changes during low traffic times, we would end up creating time bombs, scale issues being masked until a high traffic time rolled around.

This also creates like a kind "brain latency" for developers I think. I'm coming at this from the operations side of things, so maybe my observations are a little biased here. I have observed that if people deploy changes and it breaks something immediately, it's a very clear correlation and they can fix the bug generally pretty quick. If they deploy and then it breaks 72 hours later, any number of things could be the culprit, especially in a fast moving environment (times 1 billion percent if it's a microservices architecture without a strong devops culture, which most of 'em are). Debugging then takes much much longer. This is made worse if the person who deployed the change is not quickly available when their thing breaks, and it makes being on call for someone else's unproven feature very stressful.

So instead I think it's better to make sure deployment and build systems are rock solid, and deploys are as accessible and as idempotent as possible. Chatops type systems are good here. Then you can roll out big changes during peak traffic and be confident that you can quickly revert if it goes bad, and that the changes were reliable under load if it goes good. I also think it's critically important that big changes are behind rollout flags, such that you can dial up or dial down traffic at will. This is also useful when introducing new cache systems or something like CDN if you need to warm up slowly.

This is a better approach I think than trying to use the time of day to modulate user traffic. I would rather developers can control traffic to their feature themselves and have the person deploying the change with their hands on the wheel until they are confident they can take them off. That way people can do stuff independently, and everyone can trust everyone to deploy and yet still feel safe.


[Client, Friday 4PM]"Hello, client here. I know it's Friday 4pm but we messed up and did X. Could you deploy Y fix, thanks!"

[Client, Friday 4PM] "We are having a big sale this weekend we told nobody about. Could you quickly deploy a fix where all the product's prices are red and bold? That shouldn't take you long, right?"

[Project manager, Friday 4:45PM] "Hey team, X just released an important security fix for Y platform. I need you to deploy it right now or the client could get hacked."


The appropriate response to Client is "Great, but it will cost $Z in Friday fees"


In my opinion, only the last one may be justified.


And my own darn fault for using platform Y from team X that would release a security update on a Friday. Yes, fix it. But if this isn't a one-off, get off of platform Y.


Are you suggesting delaying security updates because it's Friday? I mean, you can update only on Monday if you so wish, you're unprotected either way.


If platform Y regularly releases security updates on Friday, that's poor management by X and I would reconsider using platform Y. 0-day issues of high severity should be the exception.


Timezones can make it so. It's not necessarily their fault.


Time zones do make things interesting, but weekends are more or less synchronized. (Holidays are not, notably.)


Reminds me when my boss got pissed at me because we were discussing moving all deployments to production after 5pm and I replied something along the line "I'm not going to be on the hook after hours to fix problems other people caused".


At one place I worked I put in a 4pm deadline on deploys, because I was tired of the devs seeing the end of the day as their deadline, tossing it over the fence to me, and then sodding off at 5. Invariably, it led to me trying to hunt someone down at a time when everyone was commuting home or similar. 4pm was enough time to do the deploy determine if something went wrong, and get the relevant dev started on the fix.


Easier said than done. It's best avoided, of course. But when on a startup you generally cannot afford to be an idealist.


Because missing three days of velocity will definitely cause your growth hacking to fall off the hockey stick and reduce the tempo of your disruption of unique hackathons for people with a left little toe deficiency?


Nothing of that. The reason has nothing to do with "startup culture", head in the clouds buzzword crap, quite the opposite.

The changes are pushed and working in staging, the tests are passing, Q&A is done. Why hold off? So that you get your Good Practices™ badge? I'd say it's better that you don't have to deal with last week's work on a Monday if at all possible.

I think being down to earth, and keeping your good judgement is key. I don't recommend making world-shattering changes on a Friday, but even then, well, it really depends on the circumstances.


If it's a startup the more likely scenario is no QA employees and minimal test coverage.


Surely you can envision some scenario where it makes sense?

* It's the holiday weekend and your new website with its curated comparison feature needs to go live

* It's the end of the quarter and having this is the only way you can sign a deal now (enough of your partners will be bound to quarters that this is possible)

* You're in the business of live sentiment analysis from TV video and a critical bug needs to be fixed before this weekend's Presidential Debate or your news channel partner will be pissed

Reduce the tempo? Some of these can kill.


Well, when do you do if you have a submission of your project on Saturday morning, and someone finds a bug on Friday afternoon? You fix it and hope it doesn't break anything else, that's what :P


"Never deploy on a Friday" assumes that you don't work weekends.


I can see not PLANNING to deploy to prod on a Friday, but rules were meant to be broken. When a change is needed, a change is needed.


Trial by fire. Unavoidable in battle, untenable in surgery.


He did :-) It's a lesson I also learned that day.


Definitely a lesson that was re-learned that day Some lessons only come from pain.


Kevin might have learned more than Brian that day. It goes both ways :)


The reverse takeaway is perhaps even more valuable. Most are not going to be as angry as you expect them to be. When you mess up don't hesitate to tell people, it's going to be okay.


This is pretty much rule #1 for my team. If you make a mistake, I'm not going to yell at you, but

1) If you try to hide the mistake I'll be mad. As soon as you know there's a problem, we can mitigate the damage by addressing it immediately.

2) I want to see that you're learning from it. A pattern of repeated mistakes may take some explaining.


This should be upvoted to the top! Definitely more than some story about turn of the century dancers :)


Most of my career I've been the guy people come to when they get stuck and need a second pair of eyes or just advice. I enjoy it, we all have to start somewhere. For the most part this has been considered a positive by my bosses, because they see the value this creates even if it's not technically my core job.

I did however once work at a company that was very metric-focused. Turns out that the metric they used for our team was closed tickets, and they felt that given my salary I didn't close enough of them. No understanding whatsoever that others on the team were able to close more tickets because of my help (let alone that the tickets that made their way to me tended to be the complicated ones that others couldn't solve). After the first talking to, I stopped helping out my teammembers (I explained why) - and jumped on the first opportunity to get out of there.


Mentoring is hard. It makes me question everything that I know, and worry about what this guy's code will look like in a year if I criticise this, or praise that. I wish there was some way we could all just work together, for real, in real time.

I miss construction. Back then, I could just tell someone "Hey! You! Don't fuck that up, I'm pouring concrete around it tomorrow!" And we would still be cool at lunch break.


Ha, it's true. I grew up around lots of construction and farming. Those people just aren't as sensitive as white collar workers. Probably why I'm not working at a place like that.


If you have the culture you can make similar jokes with people who write code.


I run a number of side project sites and I do much of my work late at night.

I have one simple rule for myself - which is to never deploy anything at night before I go to bed. I've made several critical mistakes which I deployed to production and then went to bed only to wake up and find out that users couldn't use my products.

Thesedays, I do all deployment in the morning - that way, even if there is a critical bug, I am awake to catch it and fix it quickly.


Generally a good idea, if you don't have a setup where everything is validated end-to-end before you go to production - which is most software, and even if you have end-to-end automated verifications there's still risks outside of that scope / vision. (a server may run out of hard disk space from the new code, to name something random).


Definitely. I have a similar policy about deploying late in the day on a Friday. Rushing to get something deployed before an arbitrary deadline is generally a bad idea. By all means, push hard to finish the feature and qa it thoroughly, etc., by whatever cutoff, but wait to hit that launch button until the next morning. Saves so much stress.


This reminds me of a similar story about being nice. It has nothing to do with software development but I want to share this anyway.

I was in Seattle and I took a public bus. The fee was $1. It was my second day in the US and I didn't have a dollar bill in my wallet (nor a transit card). I was assuming that I can pay with a larger bill just like Japan. Big mistake. The driver yelled at me but nonetheless she let me ride for one time. Then the guy sitting next to me suddenly offered a dollar bill and said "take this, so that you can pay for it by yourself." I probably looked at his face with amazement. "Thank you very much, sir." The guy got off the bus in a few stops later and I never had a chance to say this, but I know it's my turn now. I want to thank the nameless person who slightly changed me. Kindness is contagious.


I had a similar one. I was working in Rio de Janeiro, here in Brazil and money was really really tight back then. I would fly from Rio back to my city where my girlfriend is every other week.

So sure enough I got into a taxi cab and asked him to drive me to the airport, but make a stop at an ATM. When I went to withdraw the money for the cab, I noticed my payment didn't go through, it was still pending. I was desperate, didn't know what to do.

I kid you not, the guy withdrawing money from the ATM next to me somehow managed to notice my despair and got a chunk of money bills out of his pocket and said: "how much do you need?". I think I just stared back at him for a minute or so. "What?", I replied.

He told me that one day I'd have the opportunity to do the same for another person. I always remember this and always help strangers any time I can. And I realized that we are the ones that gain the most when we help.

EDIT: somehow somehow


You are allowed to overpay. The drivers don't like it because they are Seattle nice and would prefer you just not pay at all.


I wish my first development job was like this place. Instead there was a blame culture and I'd get told off when I made mistakes, eventually getting fired for not progressing quick enough.

It's now 4 years since I was fired from that job and I'm still in development, despite that incident nearly causing me to decide it wasn't for me.

I was extremely happy to read this post, it gives me hope. I try and take a similar approach when working with newer developers, or those that are inexperienced in areas.


This is underrated for managers and leaders trying to develop young talent. The takeaways:

  - With a system outage, don't lose your cool. Rationality is what's needed at the time.
  - Elevating "perfection" at the expense of potential breakage will ensure nothing ever moves forward and your team doesn't grow.
Some might believe that a serious ass-chewing is needed in this scenario; that would be a decision that's simple, quick and stupid. An emotionally-charged ass-chewing is almost always a sign of weakness in the ass-chewer. The only thing anyone ever learned about a beating is how to avoid it in the future.

Instead, this manager leaves his engineer with thoughts about how to avoid that problem in the future. Always remember -- an outage is eventually fixed, but a damaged relationship continues forward.


I have found that difficult people are a function of both themselves and the environment they’re in, and you can’t necessarily say “be kind” without fixing the 2nd part.

The level of stress matters a lot. If a team is being run in a standard “everyday crisis” mode, they quickly reach the point where there is very little they can take. Every tiny mistake wears people down, reminding them of how much more there is to do, and turning them sour. Managers seem to panic and cut into their team’s time even more by starting to have long, daily meetings to “fix” things.

If you want “nice” people, you have to set them up for success. Reward completing the whole chain, not just hacking away (e.g. for software, not just coding but also testing, documentation, and seeking peer review). Keep meetings to a minimum. No overtime. When short-cuts were taken to meet hard deadlines, open up your schedule and scribble in the exact window after the deadline where you will stop everything and clean up the mess that the short-cut created. Give your people the best equipment that money can buy. And so on.


Seems like this story is being received pretty well here.

Though, in my opinion I think it's kind of insulting to be asked something like "What did you learn?". The question isn't really necessary. You know that you screwed up.

That kind of dynamic between an engineer and a team lead is off-putting to me.

I think the proper way for a team lead to handle it is to instead work with the engineer to help find ways to eliminate the human error by implementing tooling or processes. The conversation should go something like this:

Lead: "I wonder how we can make sure none of us break XYZ widget again"

Engineer: "We can build out ABC and run that, also generally just test better before pushing to prod".

Lead: "Cool, do you want to go ahead and take care of that?"


This statement wraps it up:

“Great. It sounds like you get it. I know that you can do better.”

Giving folks a chance to communicate what they learned, and then encouraging them to "do better," is the best way to lead.


As long as it's not overdone. If every review ends in "you can do better" it just becomes the new "must apply themselves more in class".


So long as you're not having to "you can do better" for the same issue every time, I think it's fine. There's a lot of minutia to learn in software development; a lot of opportunity to screw up.


I don't want to ever work at a job where to can't get better.


I've worked with dozens of companies of various sizes over my career, and I've always found that this type of attitude starts at the top of the company. If managers are kind, curious, and tolerant of risks that might reap rewards, it is an indication the person at the top is running it that way.

If people are "blame-first, ask questions later" types, the person at the top is inevitably running the show by focusing on mistakes and miss-steps and their people follow suit. As result, people become wary of taking risk and run to assign blame, even before existence of an issue can be validated.

As a result, I don't think you can change these organizations. If their temperament doesn't match your own, you have to live with it or craft an escape plan.


Also, never deploy and go camping. Not a knock on camping. You should also not deploy and head immediately to the hospital for scheduled surgery. The point is that if you deploy, on any day, you need to be available in case something is amiss. The only advantage to deploying Mon - Thur is that you're probably going to be available anyway.


I have the similar problem with code reviews. It is really hard to not sound harsh when giving a code review, especially in ones from junior developers where a whole laundry list of fixes comes out.


Best way not to sound harsh is to ask questions. "What are your thoughts on ...?", "Is this really what you meant to do?", "Do you think there is a better way to handle...?". It puts the power and learning opportunity back to the other person and lets them feel the accomplishment of improving.

To the junior devs, when you screw up (and you will) own it and learn from it. Like public scandals, the cover up is almost always worse than original issue.


I find that tone works when the developer has clearly thought things through.

Other times, though, some developers need a very stern review. Things like: "Don't name your tests test1, test2, test3. Give them descriptive names," "Follow style," and "this does not belong in the dependency injector," and "don't screw with event publishing logic, filter this out in the event handler in the UI" are warranted when a developer isn't taking the time to think through his/her changes.


A lot of that can be softened with "This is the project's style. I don't agree with everything in the style guide, but consistency is important." You can say that, even if you agree wholeheartedly with the part you're calling out.


Never say "do it this way". Try to explain why "this way" is better so they learn. If you don't they'll just think you're saying "my way is better" and resent you for it.


Stern really is the last resort mainly because the things that people really argue about are often just opinions. No one is going to argue that test1, test2, etc... are good test names. It is certainly better to let someone realize that on their own with a question than sternly tell them they are bad.

The very rare times when I have to be stern is with consistency issues. Usually though, that's even softened with a "Yes this might be a better way to do it, but it is not consistent with how it has been done so far. We'll make some time in the future to change it everywhere, but for now stay consistent." Often times the biggest issue with juniors or even mid level people is they are only looking at their immediate piece, and not thinking about the larger application or system level picture.


Or... bring the team in to review some old code that no one has touched in a while and give everyone the opportunity to critique it. Set those critiques as the bar for everyone's code review. Then, when you review the JD, if he has made those mistakes, you don't have to be stern, you just have to remind him (or her).


Does being stern actually produce better results?


Yes, much better. It establishes that we are a "clean code" shop, and that sloppiness isn't tolerated.

This is important when dealing with junior contractors.


How about...

"Can you think of names for these tests that would be more helpful when we need to debug why they're failing?"

"Check out the styleguide for indentation like this; consistent code is easier to maintain"

"Here's a place where we did something similar. See how this logic goes over here instead? That helps us keep x, y, modular"

Phrasing things a little differently can make feedback feel like a learning opportunity instead of a criticism.


I was subjected to this "Socratic" type of code review when I was younger and I didn't like it. I felt like I had to worry about what I thought the reviewer might be thinking as well as what was actually going wrong with the code.

I think just saying what you think but with a bit of humility and the attitude that you need to justify yourself is best. Especially since even the best seniors often get hung up on pointless crap during code reviews.


Yeah I feel like a lot of advice is describing what constructive critics do, rather than the attitudes that lead to it.

If someone is humble, they'll naturally tend to ask questions as described. What you've written isn't what they expect, but they assume you're not an idiot and there's a reason for it, so they ask why.

But if someone assumes they know better, the socratic method is likely to be just as condescending as just saying you're wrong.


> What you've written isn't what they expect, but they assume you're not an idiot and there's a reason for it, so they ask why.

This is exactly why I ask questions. It's important in a code review to understand the frame of mind of the code writer. I presume the person is not an idiot and did things for a reason or will respond with a doh! it was late/that was careless/thanks I'll fix it.


Also, "why not X?". It puts the assumption up front that the programmer has already thought it through, which is often the case.


When it comes to code reviews; judge the code, not the developer. It's ok to be harsh to the code, but always make it a learning experience that people can take something away from -- and as a lead or manager, it's important to give your engineers the understanding that code reviews can be harsh, but it's meant to help improve everyone on the team, and to learn from anything someone says.


My only advice is to not worry too much about it. One of the first things a new dev needs to learn is how to separate critique of the wok from critique of them as an individual.

IMHO, code reviews should be clear and concise. They aren't a place to go out of your way to soften blows. I expect the same when my code is reviewed.

Edit:

It occurs to me the parent may have been referring to an informal review or one with the intent of mentoring. Asking questions, like the sibling mentioned is a great way. My comment is geared more towards a formal review.


I'm currently working as a junior dev. Not by title, but by experience. Honestly, I appreciate getting a laundry list of fixes over none. I know either way I can be doing better. As for sounding harsh, I feel it's a very subjective thing.


There's a developer where I work who has an amazing ability to make you feel good about code reviews --- I come out of one feeling pumped and enthusiastic even though they've just shredded my code apart and now I'm going to have to do all that work again.

Conversely, I've come out of reviews from other people, where they've been fundamentally happy with my code, feeling depressed and miserable.

Communication skills are vitally important. I've been trying to learn from the first reviewer; next time they visit I should actually grab them for coffee and talk about it...


I work with a couple of particularly arrogant junior devs. In code reviews I will tell them things like "the way you're doing this works, but it's better practice to do ..." and they just shrug it off, saying they don't really want to change it, they just want their code merged. Drives me crazy. So junior devs, and really ALL devs, please be humble about your abilities or you will be terrible to work with.


It sounds like your org is lacking in some basic leadership. If issues are brought up in the code it either needs to be corrected or justified with the team.


It is hard, particularly when management has put someone in a position they don't have the training for. In a junior developer that's understandable, but still painful when you have deadlines. Luckily, most developers are willing to learn and pretty good at seeing the patterns[1] of things done by the team.

To me, the biggest thing is don't lie to save feelings. You need to tell the truth or they will feel a whole lot worse later. Loosen yourself up. Don't go into that ridged pose. Talk evenly, and if you can manage it act in a jovial mood as you walk in. For the love of all you hold holy, know what they did code-wise, and take some damn notes before hand. Winging it will be the death of their trust. Teachers you don't trust aren't going to teach you anything.

I had a job where I was the C code reviewer, but was not allowed to program in C[2]. One developer, a fairly senior one, wrote some of the worst code I've ever seen. My code review was basically C 101 and it really was a tough chore to keep myself following good body language and words while still indicating the code needed a lot of changes to be acceptable. I really do blame the employer for setting the developer up to fail (no C training, just read a book and wing it). I did it ok, but I really wish I had a mentor of some sort in the room to give me a critique of my critique.

1) I really don't mean design patterns, more like the general pattern of code with that project / workplace.

2) Well, obviously I couldn't write C code since there would be no one to code review mine. Not obvious? Wasn't to me either to tell the truth. I got to spend my days programming in a reporting language and AWK. I only lasted 9 months (my 3 seasons in heck).


This is a matter of being friendly with the team on a daily basis. If you are, then you can change a sentence that could sound harsh without the context into friendly conversation which may teach both of you something new. If you have found anything bigger then one line, sit together, have a laugh or joke (friendly one), chat about the issue, think (both) about improvement. Not so difficult, really.


I find it's a good idea to throw in some positive comments as well along with indicating the severity of different comments (e.g. must fix, nice to have).

A far worse situation though is when a mid-level or senior developer is defensive about code reviews and won't cooperate in the code review process.


This is one of the main reasons I prefer pairing w/junior developers over code reviews. Giving lots of small pieces of feedback feels a lot less harsh than a monolithic dump of everything that's wrong.


Be kind, but don't be passive either. Talk about it, be kind, but don't glaze over the learning experience.


I heard a story about Pepsi (dont recall the names), where an executive made a decisiom that lost them 10 Million dollars. The CEO called the executive into his office... The executive shuffled into the office, head down, and meekly said, I guess you called me in here to fire me." To which the CEO replied, "FIRE YOU?! I just spent $10,000,000 educating you!


I recall that from Thomas Watson Jr's autobiography about his life working for his father at IBM.

https://www.amazon.com/Father-Son-Co-Life-Beyond-ebook/dp/B0...


I was just thinking about this today. Why is it that software engineers have a supreme mind set...

    "Oh you don't know that?"
    "Oh you don't know about that tool?"
    "Oh you use PHP?"
We categorize our peers into two buckets. One with whom you respect, and unfortunately the other as newbies, rookies, not worthy of our time and energy.

Listen, I am completely guilty of this behavior myself. Howerver, after reflecting I am going to try to be more helpful to peers. Remember that we are all constantly learning, and the knowledge and experience that you have took you time. That newbie was once you.

Finally, let's think about this in terms of rapport. If your dismissive and arrogant toward a peer when they ask a question, they are probably going to think you're a total tool and jerk. Howerver, instead if you have the knowledge and experience and show and teach them, they are going to admire and respect you. It is really the best option.


That sort of dismissive, arrogant behavior is my number one red flag during interviews whether I'm the one interviewing a candidate or I'm being interviewed.

The only time I would allow myself to react that way, and even that's a stretch, is if I was interviewing someone who claimed to have worked with something for several years, hands-on, and would've indicated to be a guru level expert on whatever it is, and then over the course of the interview couldn't answer even basic questions. You've just flat out lied to me and my team in order to waste everyone's time in an interview.


If anything, acting arrogant helps maintain any habitual problems you have. If you teach someone something, you have to reword it - which teaches it back you you most of the time, and offers a good third-person view of your work habits.


> One Friday afternoon (...) [Kevin:] what did you learn? / [OP:] I talked about the need for proper QA. About thoroughly testing my changes. About taking the time to make sure the job gets done right.

Kevin sure sounds like a great guy. But there was a simpler lesson to be learned: never, ever push changes on a Friday.


never, ever push changes on a Friday.

I think this goes a little too far.

I personally push code every Friday morning, and keep my Friday schedule relatively open so I have time to deal with any surprises.

Many successful companies brag (justifiably) about their continuous deployment systems and speed of deployment. I can't believe they suspend all of that every Friday.

http://product.hubspot.com/blog/how-we-deploy-300-times-a-da...

https://github.com/blog/1241-deploying-at-github

https://www.wired.com/2013/04/linkedin-software-revolution/

https://www.facebook.com/notes/facebook-engineering/ship-ear...


Pushing twice a day in that Facebook story gave me a shiver.


If they were deploying to their entire fleet all at the same time, I would also shiver.

However, most likely that have many stages in their deployment with testing and monitoring that exponentially deploys and if any of those stages don't pass, they auto-rollback.

The End game: minimizing risk with code changes.

1. Deploying more often allows easier root cause analysis to identify the code breaking change.

2. exponential deployment allows developers to monitor their services as it partakes in more variation (more users = greater chance of edge case breaking bug)

3. proper monitoring and test coverage while using auto-rollbacks allows developers to deploy without fear as any unintended effects will be detected at the soonest possible moment while minimizing cost to the users of that breaking change. Then, auto-rollbacks will redeploy code changes that previously worked which should automatically mitigate that code breaking change.


Facebook legitimized the "move fast and break things" attitude by demonstrating that users don't care about reliability, at least nowhere near as much as some of us wish they did.


Junior devs should also remember that whilst they might be trying to be kind, sometimes the support person you are trying to instruct probably knows more than you do.

I was a bit bemused when a young guy who had just gone into devops and sysadmin gave me a mini lecture on the importance of open source and contributing to it if you have the skills, and that I really should give it a go one day, because open content is amazing.

I didn't have the heart to tell him I'd gotten into a few of the LibreOffice release notes for work I'd done on the project. Nor did I feel like telling him I'd made significant contributions to Wikipedia. I think I sort of weakly smiled and tried to change the subject.


Juniors in particular are guilty of this, I've found - spending the last few years solving problems, they often come into the real world looking at most problems as 'simple' or 'a waste of time', and tend to be dismissive of non-programmers or domain experts. Some young people can also be surprisingly arrogant. College tends to be a very different experience from the working world.

Good on you for being patient and not letting them drag you into a place that would have humiliated them, though.


Thanks a lot for sharing this. I've been in 3 different companies now where I learned a lot but the biggest reason I left my first job was because I was afraid to ask questions because my boss did not know how to respond to a simple question without being snarky or rude. I learned a lot from that job but I knew that I wanted to be in a better environment and I am thankful that I made that decision. Working in an environment where everyone is encouraged after making mistakes and learn from it is where I want to work for.


The virtues of being nice are often extolled, but it's kindness that's truly valuable. You can do nothing and be "nice", but only actions can make you kind. When you strive to be kind, you'll also start noticing who around also tries to be kind and who is merely nice.


Also, don't ship on a Friday unless you expect to work on Saturday.


#FirstJobsMatter! Based on my subjective, statistically unsound observations, when it comes to bad, obnoxious, and intimidating work place behavior there are people who cannot be helped and are equal opportunity offenders, and then the vast majority who appear to have picked up poor behavioral habits from their old bosses, peers and companies that they were unlucky to be associated with early on in their careers. I would assert that very few of us are resilient enough to truly differentiate between good or bad workplace behavior [in an early stages] of our careers - especially when you scored a job in one of the many respected companies with a track record of success. No matter how toxic the culture - it must be right since company is doing good, right? And then as we move forward in our careers we take these ‘learnings’ with us. As a recent observation, this seems to apply to all phases of peoples careers where I witnessed an outstanding R&D manager / director turned into monstrous VP, who in reflection is just putting into action same management practices that were used by a pre-acquisition start up CEO he used to work for. And then of course there are those golden people that really make it worthwhile to come to the office and that you just love working with/for - sort of like the project lead described in the story here.


Situations like this really let you know the true merit of your leadership.

I had a similar situation - where alerts based on bad data went out to customers, and hundreds of customer support tickets started flooding in. The VP, Director, and my manager all came to my desk.

Instead of chewing me out, or telling me how much I had messed up, they calmly figured out the issue, told me that mistakes happen, and thanked me for helping them figure out how unreliable the underlying data we were using was.


Junior developers who make a mistake? Cut them some slack and teach them.

Senior developers who make the same mistakes over and over and never learn? I've kindly pointed out the mistake and invited them to avoid it in future. After a few more failures, I sometimes talk to them sternly, but the sad thing is that they won't learn from either kindness or harshness. I'm not in a position to fire them. Does anyone have an idea better than "Just suck it up?"


Talk to your manager. That is their job to make sure they are doing what they're supposed to in a more formal setting if informal coaching does not work.


Yes, I suppose this is the "correct" way. If my manager doesn't have a problem with my coworkers failures, then I'll just have to tolerate it too.


Aren't you interested to find out why they're making those mistakes?


Of course. I guess I can give some examples.

One developer writes huge methods or functions with a tangled mass of if-else. The inevitable result is that an else case missed and the whole thing fails spectacularly in some situations. Discussions with the developer are basically "That's how I've always done it and I don't want to change. I'll just add that missing else".

Another developer constantly forgets to terminate servers that he started in our cloud hosting provider. This costs the company money and thus should be avoided. When its noticed that they're still running, he gets asked to terminate them but he doesn't do it right away and forgets again. Someone else ends up doing it. He's been asked repeatedly to make sure to terminate the servers. Suggestions to help include setting a timer, writing a script that will check for unused servers, etc. Those suggestions aren't heeded or I guess he forgets to remember to set the timer.

Another developer says in a meeting that he'll perform a task, but forgets before he gets out of the meeting. When he's asked to write it down, he does so, but never looks at the paper later.

There are more cases, but it seems to boil down to people not learning. You need to learn how to structure your code, you need to learn how to manage your time, you need to learn how to remember to do something.

But maybe I'm the slow learner and I'm missing something. Thus I'm open to suggestions. So far my strategy is to accept that these other developers behave this way, and setup ways to protect myself from their failures.


Teams are made of people, and people are made of animals stretched to purposes far beyond any resemblance to the environment they were designed for / by.

Animals will have ingrained habits, many of these will be maladaptive. People though can create systems and systems can adapt for those habits. Do some cost/benefit analysis to enacting policies within the system that is your software development lifecycle. Assign people tickets, tag machines with their desired longevity and kill machines past their designated life span (or a default minimum), hold code reviews for new highly conditional methods and include drawing out truth tables and checking the desired behavior for each row.

Lots of teams either expect everyone to be be an ideal spherical engineer and stumble when that proves not to be, or expect nothing of anyone and are doomed to achieve that expectation. The middle path of expecting excellence but tolerating failure is a lot of work but has plentiful advantages. How much failure you have to tolerate and how complex the systems designed to provide that tolerance are depend on the makeup of your team and its size (the larger your team the closer to the mean you'll have to be, and with more individuals a wider variety of behaviors will be represented).


Well put. I prefer enforcing things by technical means when possible and via policy and process at other times. Sadly I see a lot of policy and process that isn't enforced, thus it might as well not be there. Nevertheless, I'll just attempt to improve that which is within my sphere of influence.

I suppose I'll just have to do my best to expect excellence and tolerate failure. I appreciate the insight.


This demonstrates two principles from "How to Win Friends and Influence People":

- Don't criticize, condemn, or complain. - If you're wrong, admit it quickly and emphatically.

There are many good anecdotes like this in the book, including endymi0n's example.


I have witnessed the same thing at my current startup and this is the reason why I excel at my job and why my coworkers are some of my dearest friends. It is this sense of camaraderie that brings people together. We've all been through a lot, late nights of frantically pulling together an entire feature, reading scores of hibernate exceptions, debugging through IE bugs and jumping to help another developer with no questions asked. It is this through this journey that I learned how to provide safety nets for my junior developers and find ways to lead them to success just like my predecessors have done for me.

This is why I love being a developer.


Great anecdote. We've all done it. Another thing I learned really early on - never deploy code on a Friday. Especially not Friday afternoon/evening. And definitely not if you're leaving for the weekend :)


I had a similar experience when I first started as a junior engineer at a financial company. By accident, I dropped a database in production... immediately, people found out and the head of production support came to me and said "This is strike 1".

Terrified, I ran to a senior engineer, who calmly and efficiently proceeded to restore everything in a few minutes. The only thing he said to me was "happens to everyone, don't worry about it, just be more careful next time."

I was extremely thankful and a LOT more careful next time, and each time after that.


This reminds me of one of my favorite blog posts: https://circleci.com/blog/kindness-is-underrated/

> The choice to be kind can be hard, especially when you’re frustrated or when, thanks to your gifts, you know that someone has made the sort of error you could easily have avoided. It’s easier to be direct; to not concern yourself too heavily with the emotions of others; to want to be unequivocal in your rejection.

Being kind is hard, and being an asshole is easy. People have a natural tendency gravitate towards the easy choice, the path of least resistance. Making the more difficult choice to be kind takes time, effort, and lots of empathy, but I firmly believe what you get in return is worth all that.

By being kind, you help create a more trusting, inclusive, and generally more pleasant atmosphere for you and your team to work and grow in. People also tend to respond to kindness in kind (bad pun intended, sorry!), and kind people tend to prefer working with other kind people, so in many ways kindness literally begets more kindness. Being an asshole also comes with similar feedback loops, but I know which one I want working for my team!

Disclaimer: I'm working at CircleCI now. :)


I think there are few professions where people screw up a little bit (sometimes literally) and it gets logged, measured and escalated so frequently than in SW.

Computers are unforgiving. Humans err. This really needs to be at the forefront when selecting personal - developers and managers. The answer can't be all forgiving. It can't be you are fired. And it probably should not be ad-hoc all the time. Company values matter if they are lived by.


The best teams and organizations are the ones that are constantly learning, and foster an environment that encourages and supports that.

And, can take those lessons learned, and establish practices to help prevent and minimize such mistakes in the future.

I've seen lots of teams totally embrace the first, but inexplicably, don't follow through with the second step. They go through the motions of "learning" without actually learning.


Is that so uncommon? I mean, maybe I've been particularly lucky in my career but that sounds like standard behavior to me. I mean you already feel bad enough for blewing it that it's no use to add some more to it. And as the article says it happens literally to every one of us at some point. So unless your boss is a genuine bag, that's the expected way they should react


Unfortunately there are a non-negligible number of managers that take to the 'stick' approach and will yell at people, etc. Many people seem to be okay working in that environment so there doesn't seem to be pressure to stop that behavior.


I have been a graduate dev for four months now and i already feel like im not good enough on almost a daily bases. Though I do enjoy my code reviews because i know it will improve me as i go along.


I've worked at a few different places now and have never seen junior team members treated badly either.


I think you've been lucky. I've seen some really awful senior devs/team leads in my time.


While it's nice that we know that the protagonist is agonizing over his mistake, in reality, that's not often the case.

Since I don't have insight into their heads, and I can't tell if something was a sloppy mistake because they were inexperienced, or a sloppy mistake because they were trying to get out the door to a camping trip and prioritized their personal life over the team or client.

Clear example of why it's bad to deploy on Fridays, why it's bad to have single-person deployments, why it's bad to leave your laptop at home, etc.

But how would the kindly Senior know the Junior dev just made a mistake vs. was a fuckup and was thinking, "I don't give a shit, Senior will catch it and I just want to be on a beach now. Fuck it, I'm 'done' -- now to just sneak out and put my phone in the glove compartment for the next 3 hours while this blows over..."

The answer is he wouldn't.

So here's the other issue. Let's say I trust one person more than another. And Junior A -- who I believe to be a superior talent -- gets the benefit of the doubt, but Junior B gets scolded.

Now let's say there's even the slightest difference in hair length or eye color between A and B... and presto that's an HR discrimination complaint I now have to deal with. As it turns out, Juniors don't have insight into the minds of Seniors either.

So, some good advice... Be humble, do what you can to not make things worse for others, set up good practices that let everyone thrive (the best way you can do this is to avoid failure opportunities -- i.e. don't deploy on Fridays), and do what you can to treat everyone fairly and equally based on perceived actions and offenses.

We can't know what goes on in someone's head.


All this talk about being kind (or staying calm) in tense situations reminds me of this video I caught on reddit last year. [0]

There is a dash-cam filming the inside cockpit of a plane when the pilot abruptly finds himself in one of those "death spirals" or "flat spins".

The pilot doesn't say a word.

He immediately reverts to his training, rights the plane, glides it towards the ground, and only grunts a bit when he crash-lands.

I found that absolutely amazing. To have that level of poise when you are literally facing death is unbelievable.

It is very similar to the simple advice given in the linked blog post.

Staying calm in difficult situations is the most efficient course of action.

Stressing out, raising your voice, yelling about things, or blaming others isn't going to do a damned thing when your plane is literally spinning out of control and going to kill you in a few minutes.

[0] https://www.youtube.com/watch?v=bvbS-oHi9ro


  > "death spirals" or "flat spins".
Just an aside, for the record, obviously not important but since the topic came up...

A spiral and a spin are two very different things, and a spin and a flat spin again are very different.

In a spiral you actually still fly, in a spin you are not, you are in a continuous stall situation. That means from a spiral you can recover simply by "flying out of it", for a spin you have to do stuff you would not do when "flying", like full(!) and sudden rudder.

This heavily depends on the airplane, but a flat spin can be really dangerous, getting out of one often is much harder than from a "normal" spin. No problem in an aerobatic airplane, but in a jet it can be an unrecoverable situation.


Approaching problems with a calm (and positive!) mental attitude is a tough life skill, even more so when it's become a life-or-death skill.

This is why I really dislike road rage - people take dangerous situations and make them worse, usually saved by instincts alone.


Also as a senior developer you carry some of the responsibility for the junior's ability to make that mistake. You probably set up and maintained the development environment where such an error could slip through into production.

So in case of a critical error/melt-down you should first of all blame yourself and think about why you set up a process and an environment that let this happen in the first place.

Think about what went wrong and how you can prevent it from happening again. When I started at my current place of work, they had a horrible manual deployment process that was the cause of 99% of all problems they experienced. Rarely critical problems, but still annoying and unprofessional. So we implemented automated build, test and deployment process that got rid of almost all of these problems.

Now, whenever something bad happens, we realize it is a problem with the process (not the individual) and correct/improve it as needed.


This reminds of important lessons I've learned from the classic How to Win Friends and Influence People - never ever criticize people and you will get the best out of them. There are countless other great lessons in this book about being a leader and dealing with people.


I agree except for the word "never". In this story he was able to say “Great. It sounds like you get it." If you can't honestly say that, then it's time for some leading questions possibly followed by criticism.


No, definitely criticize people, but do it in a constructive and positive manner so it doesn't seem like it is criticism or coming from a negative place.

You cannot run a team without some measure of criticism or feedback.


When I started of as a junior developer it was in a team of 3. 2 Senior developers and me.

One of them was really kind and a nice guy that was getting along with everyone else in the office (other teams etc). Also he was a very good dev. Nice personal projects, knew what he was talking about, humble and hardworking.

The other developer was aweful. He was slacking a lot, all day on facebook and doing his personal home-owned business over his actual work, not caring to help me at all on any request. We didn't get along, and at first I thought it was my own issue. Later on I realized and having a talk with my manager that noone really was getting along with him. At some point he thought it was a great idea to make an education session, I was like ok he is putting time into it. He then proceedeed telling me that he is gonna teach me the very basics of programming as "we" didnt go to uni, I told him I went to uni and I finished with a Distinction but sure he could tell me whatever. He was trying to teach me things he didnt even understand and I was taught about them at uni. (I understand this is a basic mistake junior devs make think that they know it all, this wasn't the case here, he was trying to teach me some basic CS concepts, I went to uni for it he never went to Uni for it, he was just a programmer, nor he had the interest or hobby to go more intellectual on the matter) This lead me into not trusting him at all, in anything, while the other developer was really kind and he let me on his own to understand that he was a really good developer and I should listen to him.

A year later my performance was really good in the yes of the company, I was producing a lot of stuff etc. The developer that didn't like me was clearly angry at me. He was producing things that could be done in a week in like 3 months. Well long story sort, the company wanted to fire him, although it wasn't very viable at that moment as they would have to pay him cause he was with them for a very long time, so they started a formal process against him by keeping schedule giving him a formal warning etc. He improved etc.

I stayed at the company till I became a developer and a year after. I was working alone on mobile apps, and I remember having a chat about an issue I had for the app and some ios apis and how I tackled it and I remember that dev still trying to tell me what to do and how although he never ever working in a mobile app in his life or any experience in that field whatsoever.

Anyways, please don't be like the 2nd developer, be nice and let junior developers believe in you and understand your actual skills and capacity on their own. You don't need to brag or try to showoff because you might end up making them not trusting you.


I'm re-reading Peopleware this week, and this is one of the core points it makes. You need to give the team the flexibility to make mistakes. I'm not sure I've ever met a manager who has read that book and disagrees with that principle.


While I fully appreciate the point of the post, the junior dev didn't screw up. The team screwed up for not having a process and practices designed to avoid these mistakes. Unless they do have those, and Brian decided to ignore them.


101: Never deploy on Fridays.


> It’s okay,” he replied. “We’ve all done it.” He paused for a moment. “But what did you learn?”

Don't deploy to production on Friday! Queue it up in staging for Monday morning. Make a team policy for it!


This is the Kaizen/Deming principle #8 of "Drive Out Fear". Here's a good in depth blog article about this as it's a difficult to explain topic that, especially for managers trained in the older methods of management, deserves a long form treatment:

http://michelbaudin.com/2012/10/27/deming_8_of_14_drive_out_...


Had similar experiences in my first job with one of the smartest engineers I ever worked with (Claudio Marcio da Silveira). He early-retired to work on his family business (something related to flowers). But I can still remember all the times he let me learn by doing small mistakes, covered my bigger mistakes, and also educated me.

Trying to become that same engineer in my own way, but I definitely own a lot to him. And to other professors from uni/school that were extremely kind as well.


> Jr Dev

> Access to production

> With clients that need the service running 24/7

Oh boy... is he used to work on facebook? when i was a jr dev, i didn't have access to production everything went to source control and then tested by my project technical leader and qa. This ensure that i won't do this things since my changes getting reviewed and tested plus we have reproducible builds that anyone can verify and continue working.

Case in point: No JR Dev ever need access to production ever, period.


I disagree. I work for a big company with a big product that shouldn't have a single minute of downtime. We let new devs deploy to production from day 1. We have ways of mitigating the risks, like a very sophisticated monitoring and the ability to rollback changes in seconds. Yes we sometimes have outages, and we lose money. But in our experience, empowering developers is much more effective than handcuffing them. They learn to be careful and freedom allows for creativity to flourish resulting in amazing solutions.


I think in your release model are forgetting about development and the staging areas that most devs can run or well use freely before ever going to production. Those are the places where the developers not only have access but are also encouraged to disrupt, destroy and let their creativity run before going to production. Giving access to production from day 1 its a disaster waiting to happen and m pretty sure stoling ip or databases aren't part of your monitoring software scope nor being able to detecting malice.


Good to know there are people looking out for junior devs :-)


Wow, this really resonates with me.

The very first day of my first 'proper' job, I made a total mess of something that cost the company around $5000. But my manager was like 'don't worry, we'll sort it, just learn to ask if you don't know how to do something'.

Many years on and I lead a development team. Now I always try my best to 'be kind', to coach and mentor, and it almost always pays back.

That one guy's kindness had a lasting impact.


The hardest thing for me when I was junior dev was simply being noticed. I'd rather someone be hard on me than to pretend I don't exist.


This advice is obvious. How can you not be kind with a junior person that makes a junior mistake? It's human. An angry phone call would not have made any difference (except damaging the relationship). But what if Brian deployed on Friday night without testing 4 times in two years? And 3 times it caused trouble for the business. How do you stay kind then?


You can be kind while still being critical. Even if it's kindly letting the person go with as soft of a landing as you can provide rather than the walk of shame to the lobby with security.


A mistake is an education. If you fire someone after they make a mistake then you are throwing away your investment in their growth.


For actionable advice on how to be kind while still capturing the lesson. I recommend this article by John Allspaw on blameless postmortems.

https://codeascraft.com/2012/05/22/blameless-postmortems/


The mistake was launching something on a Friday before vacation. Wed or Thursday would solve this. Launching on Mondays or Fridays, not a good idea.

So that is another lesson learned in addition to being cool when someone messes up once in a while. In ego driven teams sometimes it turns into the blame game and throwing under the bus, that is not kind.


Cmon. An article about kindness as top post on HN. This is taking the piss. This is a place that specializes in snark and dismissive discourse.

Everyone is a peacock here. The bumbling programmer should thank his stars his boss does not appear to be the typical agitatedly infallible HN commentator or monday would be a different day.


proggit has a better title: [Mentoring Junior Devs: Be Kind](https://www.reddit.com/r/programming/comments/57g3x7/mentori...)


Maybe im just lucky but this is generally how any manger I've ever had normally would have handled the situation. Not every single time, of course, since they were all humans and got frustrated sometimes, especially after a few of these in a row, but for the most part, they weren't dicks.


My first job was at an aerospace company in Florida. My boss was so nice.

I was really surprised to get the coveted desk in the 60° server room. Coming in off those 90° 100% humidity streets to put on a parka and sit at my workstation wearing fingerless gloves.

Man, anybody should be so lucky to apprentice with that guy.


For some reason I am imagining you working in a server room shaped like an equilateral triangle (i.e. the walls of the room meet at a 60° angle)


And he came out of an aquarium (i.e. 90° walls + 100% humidity)


Florida native here. You're not really doing yourself a favor by sitting in the A/C all day. You aren't allowing your body to accommodate to the heat.


I have seen people that react positively to feedback.

But some people take things very personally, even if you try to be very polite and respectful.

Sometimes people, especially people with narcissist traits or people obsessed with rank, will not accept any feedback coming from a someone from a "lower rank".


If someone asked, "what I'd learned" in the above context, I'd assume it was more to satisfy their needs of feeling important and superior than genuinely caring.

Much better is: "Shit happens." That's an empathetic attitude that reflects experience in our field.


Well, this rule applies to basically every field of life. Most (if not all) religions and moral systems have rules that say to respect the others and be kind, even if they are not as we want them to be. But I'm not surprised that it's rediscovered over and over.


I dont think "not getting angry" qualifies as "being kind" unless anger is the default state.

kindness doesnt automatically emerge to replace a lack of negative outcome.

"My father didnt hit me with his belt" is not kindness. "My boss didnt fire me" is not kindness.


"One Friday afternoon, [...] I deployed the changes"

Good god, this should be taught in high school.


I thought this article was fantastic. We could use more kindness at home, at work, and in general.

I find it really unfortunate when an engineer is fired because of a mistake. All the company is really doing is giving a more experienced engineer to someone else.


Personally i prefer the type of mentor/boss that comes and says things straight, surgarcoating everything isn't good once you realize that everyone could think you are stupid or well starting to find your replacement.


Stuff like this always boggles my mind - anyone can mess up. Mistakes happen, but if you didnt learn from it, history will repeat itself. I can't help to feel that common sense is sometimes lacking.. Why is that?


Be kind, but also be honest.


The key sentence: "Kevin gave me the space to screw up, as long as I learned from it." I like that concept of space to screw up. I definitely recognize the need for that.


"Blame is not for failure, it is for failing to help or ask for help." It changes everything. Usually attributed to the CEO of The Lego Group, Jorgen Vig Knudstorp.


I only had to rad the first three words "One Friday afternoon". I learned early that I was doing everyone a favour if no updates went out on Friday afternoon.


I have wept in the night For the shortness of sight That to somebody’s need made me blind; But I never have yet Felt a tinge of regret For being a little too kind.


Being founder of a startup, I often get into position as this. I think there is alot for founders too to learn form this small story. Thanks for sharing.


From my experience as a manager, being kind works very well. This is especially important in the stressful start-up world where tensions are high anyways.


I really liked that article was short and to the point!


> “It’s okay,” he replied. “We’ve all done it.” He paused for a moment. “But what did you learn?”

Never push changes on a Friday PM. Ever.


Can't believe you could get fired for something like that. Do you work for an American company or something? Jeeze...


> I walked into the office certain I was about to be fired. Wait, can you be fired that easily ?


Many US states are "At Will" employment. You can be fired for almost any reason, at any time. At the same time, you can quit at any time. Giving notice is a professional courtesy, but not strictly required.


Right now, this post is 4 votes away from being in the top 10 most popular HN posts of all time.


Every child should be so lucky to have this experience as well. Empathy goes a long way.


Simple but highly effective writing . How can I learn to write like this ?


Here's a list of online writing communities slanted toward young writers but not exclusively: http://study.com/articles/40_of_the_Best_Websites_for_Young_...


Rewind.


Came here for that :)


No. 1 rule for deployment: Don't deploy on Friday.

Let it be a deployment free day!


That's why I never deploy on Fridays. For me, for everyone else.


This goes for everyone on your team, not just junior devs.


Very good advice. Your good deed rewards you over time.


Isn't there supposed to be a code review before devs are allowed to deploy something to production?

Yeah, Brian screwed up. But the company has at least some fault for letting him get that far unchecked.


This varies highly depending on the organization. I've worked at places that do it both ways. Particularly in small places when resources are tight, teams are distributed, or there is a non-code safety measure in place (e.g. Business decisions are made ex-post of the code running dependent on the quality of its output), code reviews pre-deployment may not be necessary and can even be harmful (due to wasted resources).


This was far enough in the past that code reviews and "Don't deploy on Friday" weren't necessarily taken as standard practice at a lot of companies yet. But yes, we all learned :-)


Sure, and I don't see them blaming Brian in any way.


Refreshing...


Thanks!


No problem..


Don't tell me what to do. ;)


Be kind. Rewind. :)


Amazing!


We're an industry filled with people who have social disorders. Some people are just angry little people. Misspell something in a config file and they just ... go ... ballistic... It's not enough to just try and tell these people to be nice. They don't know HOW to be nice. Their brain isn't wired to be nice.

Edit: spelling.


I hate to be this guy... but 'ballistic' has two l's.


Correct, and thanks.


You're fired.


interesting


cool story bro.


Life happens. Being kind shouldn't just be restricted to junior devs, or even just devs for that matter. I've seen many folks be flatout mean and unrealistic to adult freelance developers from low-income-countries. It's a serious problem.


Except in real life people sometimes make malicious "mistakes" deliberately and when caught and called out on them either completely dismiss them or even gloat.

This reads almost like a fairy tale.


We detached this subthread from https://news.ycombinator.com/item?id=12707881 and marked it off-topic.


I understand that this thread might seem off-topic, but if that is the case it's due to the fact that the parent thread (regarding a Bob Hoover anecdote) was already off-topic.

You are obviously free to decide to detach the threads that you see fit, but not detaching at the parent level strikes me as incomprehensible and discretionary.


> Except in real life people sometimes

A minority of people are toxic, you start by cutting them slack as you'd do anyone else, then you cut them off. It's not really a problem.


[flagged]


In my experience, most of the time those people in positions of authority aren't actually malicious. They're either misinformed (the higher you get, the less you're able to have both depth and breadth of knowledge), or they have competing priorities.


What you describe certainly happens and is probably the most common scenario. Many times there is no malice, just some very worldly constraints that result in undesired consequences.

However, rising through the ranks of social structures is facilitated by machiavellianism and this results in an over representation of the kind of people I was talking about the higher up you go. It's a numbers game.

This isn't even very controversial and academics studying psychopathy will even acknowledge it.


That seems to be a scenario more or less completely unrelated to TFA or the discussion.


[flagged]


Pssst you're the problem


In my experience very few serious mistakes are deliberate and there is a huge gain to find a way to not make the same mistake again. Which include avoiding blame, but rather go looking for the root cause.

People who can handle serious mistakes in others are some of the best partners you can have in your efforts. IMHO they aren't fairy tales.


There's a story that I love about Pixar, a Black Swan event, and Toy Story 2 [0]. The gist was, the movie's files were accidentally deleted while still in production, and the backup was corrupted.

Instead of blasting people for mistakes they set about fixing the immediate problem, and then instead of demanding "accountability" from those responsible, they set about fixing the root cause.

A few excerpts from the linked article:

"Aside from the fact that there was immediate chatter about who might have made such a dumb move, the discussion quickly moved right on to how to fix the problem."

"Instead of dwelling on pinning the blame or lamenting the loss of time and effort, the team made sure to alter the backup strategy so that something like that didn’t happen again, and it went about making up for lost time."

[0] http://thenextweb.com/media/2012/05/21/how-pixars-toy-story-...


Never attribute to malice that which can be adequately explained by stupidity.


In which case angrily confronting that person would further justify their ill will towards you and strengthen their resolve to sabotage you. How is that productive or tactful?


And becoming passive implies the message that you're a pushover to whom anything can be done because there will never be nefarious consequences.

To those people the mindset is:

If I do X I might gain something (or feels good) and no mater what I'll never suffer any retaliation for it. Why shouldn't I do it?


I think perhaps your mindset, and possibly your situation, might be improved if you assumed less about other people's thought process and motivation, or at least if you didn't assume you were correct. It's very easy to come up with ideas about what someone is thinking and why, it's much, much harder to be correct about that.

By assuming you know why people are doing things, you've already biased yourself against future information that may contradict that. Maybe sometimes they aren't doing it to be a dick to you, and maybe sometimes what you view as gloating isn't directed at you, or isn't actually gloating.


I've had many of these people explicitly state that this is their state of mind after they realize they can't get their way.

They'll do it in private so that their social standing doesn't suffer but they are very explicit about it.

Not a lot of room left for assumptions after this.

Downvote away.


> many of these people

How many, out of all the people you interact with? It's safe to make your assumption the people who revealed their workings to you, but not with everyone else.


You're talking about sociopaths. Who are estimated to make up 4% of the population. So you should be prepared to deal with them and cut them off as needed, but you shouldn't base your average strategy on assuming everyone you deal with is one.


No. I'm taking about usual human behavior.

Those figures of 4% are fairytales, not unlike what most psychologists said before knowing the outcomes of the Milgram experiment.

Almost no one believed that the vast majority of subjects would take the experiment to its end.


You sound like the one who'd have called BG back from his camping trip to fix up his code - a 3 hours drive and a ruined weekend for him, instead of maybe a 5 minute rollback to the latest stable version for you.

So with your mindset, it's no wonder people react to you the way you describe it. But believe it or not - most probably you are the problem, not them.


Practical action can be taken without having to angrily scold the person.

Giving into anger is the easy thing to do, but in doing so we become emotionally blinded to more practical courses of action which we may have realized if we had kept our cool.


Are you suggesting that man purposely tried to kill Bob Hoover?


No.


>That’s why I want to make sure that you’re the only one to refuel my plane tomorrow. I won’t let anyone else on the field touch it.

...he said, before his final fatal flight. Apparently some people DO do the same mistake twice.

([Edit] to avoid confusion: My addition is about giving a different twist to the story, to show that it's not a given that people who made a mistake will not be prone to repeat it. Some people seem to be extra prone in repeating certain mistakes. It's not saying Hoover actually died or anything... I mean, isn't it obvious?)


We detached this subthread from https://news.ycombinator.com/item?id=12707881 and marked it off-topic.


Just to avert anyone else's confusion: Bob Hoover didn't die. Not sure what coldtea is referring to.


I was giving a different twist to the story to point that things can (and do) go differently to the rosy picture painted in it ("oh, you did a mistake once == you'll never make it again").

That's why in the except I kept, I didn't keep any reference to Bob Hoover. It's about the idea in the story, not some particular pilot.


This is one of those situations where "trust but verify" takes on a whole new importance.

There's a world of difference in how you should treat someone you know almost nothing about that makes an mistake that is almost fatal for you and someone you know to be fairly responsible. That doesn't mean don't be nice if you can, relative to the situation ("nice" when someone has almost killed you doesn't necessarily mean be cordial, it may mean cool down for a sec before attempting to get them fired), but it would be stupid to assume the same mistake won't happen again without knowing a bit more about how it happened in the first place.


What did you learn?


That people need things spelt out for them even if the meaning is obvious?


that's a twisted way to twist a story.


Aren't those the best?


I don't know if you're being snarky or not, but Bob Hoover is still alive



Maybe tell a true story that helps your point, rather than a blatantly fictitious one.

I could tell a story about purple moon dragons, but it wouldn't "show" anything.



>Maybe tell a true story that helps your point, rather than a blatantly fictitious one.

The point wasn't to give a true example, it was to add a twist to the ending of the story told by the parent to make those reading it fathom an alternative way things could have gone that goes contrary to its message.

>I could tell a story about purple moon dragons, but it wouldn't "show" anything.

Why not? That's what allegories, parables, zen koans, fables and 200 other forms of fictional stories used in making a point do.


Wait, who? Bob Hoover is still alive.


> I talked about the need for proper QA. About thoroughly testing my changes. About taking the time to make sure the job gets done right.

Oh good, a bunch of useless platitudes.


Honestly, I came to this page expecting some really intriguing story, from which I could learn a lot. Its been voted to 2000+ points, so must be it hit a nerve of a lot of people. But to me it seemed like a nice but very simple story.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: