* SQLite has 1177 times as many tests as code (not a typo)
* Live television is broadcast with a 5-10 second delay
* IMVU automatically reverts commits pushed to the site if regressions are detected
* Netflix implemented a system called the Chaos Monkey, which randomly shuts
down processes. Ensures the system can survive any failure
* VLC, Unity3D, Windows, 3D Studio Max and many more applications phone
home crashes, which allows developers to quickly patch frequent issues
* Code reviews and pair programming ensure no one person's mistake
can break critical code sections
* Similarly, multiple people should sign off on copy written. For newsletters
and press releases the whole team should, since they can't be withdrawn
* Well designed systems automatically backup, and those backups are
automatically tested, so nothing is deleted forever
Ultimately it's up to the company to decide where the balance lies, and to live with the consequences. Startups will accept a drastically different risk profile to banks and Fortune 500 companies.
I have a feeling it's more like: <git annotate>; "Oh, it's Bob's fault."
I was going to go off on a tangent about the importance of good management, but I haven't ever had a boss (self employed then startups), so by missing half the picture I didn't really feel qualified.
"person to blame" pre-supposes that the root cause is a person, which is not true in most cases. While programmers do introduce bugs due to oversight, I have seen that systemic issues (build environment doesn't allow for tests to be written, missing patches in legacy architecture, bad staffing decisions etc...) introduce a large percentage of defects, especially in larger projects.
If your manager is really interested in reducing defect injection, you will want to champion root cause analysis (read up on it - Karl Wiegers and others have some some great work on this).
...and who reviewed the bad design
...and who reviewed the bad code
...and who wrote the bad tests
...and who built the spaghetti system that can't isolate failures
...and who pushed new code to production without smoke-testing
...and who demanded the product ship before it was ready
* Well designed systems automatically backup, and those backups
are automatically tested, so nothing is deleted forever
Eventually though after he turned over project management of the software to the employees, they responded to the constant bug reports about the lack of this feature and added the ability to have this field. I recall he made a note about it on his blog, but that the project belonged to them and it wasn't his project to micromanage, so now they have the feature whether he likes it or not.
Caution: I probably have half the details wrong, I'll try to find some links.
Instead, filing a bug for Firefox, Gnome or the majority of popular open source projects requires you to provide even the color of your socks.
Sometimes this will go as far as establishing a "war room", possibly for weeks, dragging in people from a number of different areas to address a particular issue.
It's not about pointing the finger or otherwise apportioning blame. It's about learning from mistakes and preventing them from happening again.
That being said, I've worked at other places where shame is attempted to be used in such situations (which is what this is). In my experience it just fosters a hostile environment.
When done right, RCA is a great thing.
A bad programmer shouldn't work there in the first place, and if they need to be blamed, they're probably doing other things to destroy morale.
Recently, a piece of code that I wrote crashed after running for over a month because I forgot to lock one of several methods that was concurrently accessing a collection. I was very ashamed of this - this was an amateur mistake, and one that someone with the amount of experience that I had should not have made. I didn't get any any trouble for this, but I was pretty ashamed. "I am better than this, and I should not be making this sort of mistake," I thought and have been very careful to avoid this sort of thing since.
I don't write perfect software, and I never will. I'm choosing to not feel bad about that. It's all matter of perspective and expectations, isn't it? My colleagues can think less of me if they like - that's their baggage, though, and I won't pick it up. Shame is dangerous, and leads to a lot of the communications errors that developers make, which are often worse than the bug itself, or even the cause of it.
If you can't handle having bugs in your code without getting all defensive, you probably shouldn't be coding. The only way to get better is to get feedback.
We have good stability (we hit four 9s recently), but we're always pushing for me. The post-mortem process is very positive and affirming. That's because it's transparent, as it should be.
(1) The IDF is "ineffective" on an absolute scale.
(2) The IDF is significantly more effective than most other organizations.
While better isn't perfect, you can learn from both.
National will is important, but you don't rank on that list by being ineffective.
I didn't claim that the IDF was more effective than all organizations, I said that it was more effective than most.
For example, I'd claim that the IDF is more effective than almost every failed startup.... (You can argue that many fail for reasons outside their control, but I'll respond that such vulnerabilities are a failure.)
As a veteran I can tell you that everything you do is critiqued, and everything your boss does is critiqued, and so on.
At certain levels it's peer review, at others, hierarchtical. The whole point of it all is to increase your chances of survival.
It's a great process, and I've noticed that most military folk are great at speaking plainly and accepting critique, much more so than civilians. I beleive these processes are a huge part of the reason.
And even if there is justification for additional process, they should let the ops/dev teams come up with ideas/solutions themselves: we know our tools and teams best.
First, hating your job and not quitting for three years speaks A LOT louder than any badmouthing you can do in a 45 minute interview. You're much better off pretending you actually mostly like the place, but you're just not getting challenged like you used to do and it's time to move on.
Second, setting a positive tone is pretty important for any interaction you want to fall out in your favour. "I want this new job for this positive reason" is better than "You guys seem OK, no way you're as bad as the idiots I'm currently working for".
Someone filling a grunt position in a sweatshop mostly cares if you can show up on time - the ass-kissing in interviews is a nice perk. Someone who likes his job wants to hire someone who will like the job (and be good at it) - not someone who will just not hate it.
Technical skills can only go so far. It is rare that a person who is a technical master but doesn't appear to be able to participate in a design will succeed. Fortunately, that is easily fixed by getting out there and interviewing (with a bonus of maybe finding a job you love in the interim).
1. Get a github account, publish code, keep a social media profile etc. At the very least, this gets you interview calls.
2. People are not that creative with tech questions. Even Google (at least these days) asks you questions that come out of a textbook. So go through one, say like Skiena's Algorithm Design Manual and solve all the problems on a white board. This sucks a ton because when I sketch out code for a project, I never go do it on a whiteboard. The trick is to get to the stage where you can mentally compose the main blocks of the problem in your head and then writing down the answer on the whiteboard. This means that when you hit the whiteboard with the engineer staring at you from back, you know pretty much what way you want to go at the problem.
2.1. Statistically speaking of the countless interviews I have done, I think a large proportion of questions have been on trees, graphs and dynamic programming. I feel like this is because "some people" think that these are exotic enough that solving these problems have some correlation with smartness. Again, solve a few problems from any textbook on these or maybe go through the TopCoder tutorials on the matter and they start getting as easy as a day to day programming problem.
3. Accept the fact that people are paranoid about false positives and don't really care about false negatives. Sometimes if you have a bad day, it is not you.
4. Someone mentioned this before but interview more. Talk to people you like, you hate and people whom you have no intention of working with. This will help you get over your insecurities and make you realize that interviews are just a game: You will get more confident, and believe me, that helps. (This is just like dating, the more you do of it the more you get better ;-)
4.1 Organize your interviews so that your dream bets are towards the end of your process by which time you will have done so many problems, attended so many behavioral interviews, talked to so many people that you will be absolutely amazing!
Many people can't complete the first item, something that would take me 5-10 minutes. I feel like it's about the equivalent of fizz buzz, but scalable. It also respects their time, since I don't ask them to sacrifice a weekend day to make an app for me or similar and by the end of the interview series and a huddle, they'll have an offer or not.
2. Where did you get the idea that Google asks questions out of a text book? But yes, practicing your whiteboarding skills is a good idea. It comes in handy once you actually work as a team on designs. (I love to sit down and sketch out things for myself, but from a certain problem size on that's largely practical)
2.1 It's a basic filter (and should be part of the phone screening, before you waste a day on interview). A surprising number of people really has trouble understanding even mildly advanced data structures. If (and only if!) those matter in what your company is doing, screen for that as early as possible. Trees do the job.
4. Be careful about that. If you keep interviewing successfully and turning down offers because you like your current place, you are (at least temporarily) closing doors. So be sure to know at what conditions you'd be willing to switch, because you just might get an offer that's hard to refuse.
4. Well, I am not suggesting interviewing if you are happily married to your current job. I thought OP was not happy and wanted to interview to get a new one.
You can start to overcome this by telling yourself that your are amazing at interviews. Eventually you will start to believe it.
At the end of the day an interview is just explaining why you are awesome, and then getting the interviewer to tell you why their company is awesome.
I hope you can get out of this toxic environment. No one deserves that kind of disrespect.
Also, interviewing is a skill that gets better with practice.
Now, others already advised you to quit the job — but that's obviously not so easy. But be careful, and if you can, find another outlet for creativity, because this can seriously damage both what you can offer to other companies, and how much you'll be able to enjoy any job.
In the end I decided that I would do the best job that I could possibly do with the knowledge I had and to hell with the over-inflated opinions of my line manager.
The best seniors are those that answer questions you have, steer you in the right direction and are open to new ideas and suggestions.
The way that you defeat a system like this is to use it. Be humble, honest and calm, and go out of your way take the hits. But refuse to be blamed for things that aren't your responsibility. Force the problem people to do the same.
That undermines the system, as your putting the problem person in the uncomfortable position of taking responsibility for his actions. The whole point of "blame assignment" is to deflect blame from the golden children.
In my case, the blame regime lasted a few months. It collapses when the person pushing the regime is on the defensive too often.
And in development there is always somebody else to blame if you look hard enough (because every time a choice is made, it has costs).
But if you work in a shop that values covering you ass, then you should cover you ass and not care too much about getting stuff done (and then make a plan to get out of there).
If the poster really wants to keep that particular job, but wants to also improve the environment, they need to start following all of the rules they don't agree with to the letter. The worst thing that can happen is that it actually improves the team somehow.
It's not always the right way to make change, but in these situations it really can be the best route. Bad rules and procedures can only be (officially) recognized as such when they're actually being used. There's no need to make a mockery of this stuff, either, because time will tell whether the new rule is really helping or hurting.
Like you say, humble, honest, and calm.
This is one of those solutions that causes more of the problem its trying to solve. Management will love it.
I was sitting down with a team getting started on Agile/Lean, and I was trying to lighten things up.
"Well, the first thing to do is pick somebody to blame when it all goes south," I paused for effect, "and that's usually me, the consultant."
It was dead silent. Everyone in the room was looking at me seriously.
It's good advice, because criticizing people in public (i.e. in front of their peers) always backfires -- it just makes the criticized person defensive, so they stop being open to learning from their mistake.
People filled it mostly with our boss (we agreed with him, that when it's hard to say whose bug it's - we assign it to him), or old workers, that work here no longer, that first made commits to the feature where bug is. Or people solving the bug assigned it to themselves. Only sometimes somebody assigned a bug to another programmer who still works here.
Now the field is optional again.
As it is, bug trackers are good at collecting information and letting you create a process that is as convoluted as you'd like to track the work towards coming up with and releasing a fix, but leave much desired when it comes to trying to reduce your overall defect rate.
That said, trying to pin the blame on someone is the absolute worst way to accomplish any of this, and virtually guarantees that any "root cause" information you try to gather is going to be completely useless.
Some of that comes back to the concept of requirements traceability. Unfortunately it didn't quite make the agile cut. So it's languished in the pre-agile SEng world, locked up in clunky, heinously expensive tools.
The European Directive 94/56/EC  "on the investigation and prevention of accidents and incidents in civil aviation and repealing" states that:
> ‘causes’ means actions, omissions, events, conditions, or a combination thereof, which led to the accident or incident; the identification of causes does not imply the assignment of fault or the determination of administrative, civil or criminal liability;_
(Art. 2 part 4).
This is said again in the preamble of the same directive, paragraph 4:
> The sole objective of safety investigations should be the prevention of future accidents and incidents without apportioning blame or liability.
I'm sure there are many other legal texts (including in the US) that state the same thing, as it's plain common sense: assigning blame and getting to the root cause of a problem are two very different and separate businesses; both are necessary but shouldn't be conducted by the same personnel and not at the same time.
(Why HN doesn't support Markdown, we'll never know.)
If you have an idiot coder that has been there forever and is hard to get rid of then this field would come up in their evaluation. Needless to say it may get overlooked on other peoples evaluation. It may be used as a way to manage some muppet out the company for wasting everyone's time.
But I don't know the full story, the boss may be a control freak and an idiot himself but he may be actually trying to improve working conditions, he just chose the wrong terminology for the field.
That the records are personal -- they belong to the programmer -- is repeated again and again.
Because when you keep management-accessible records like this, three stages occur:
1. "We won't use this to judge your performance and it will not be connected to reward or punishment".
2. It is eventually connected, directly or indirectly, to reward or punishment.
3. The indicator is gamed.
PS. Your boss is stupid.
and if he'd rather check it out in his car, have him listen to John Allspaw and Mike Rembetsy talking about blameless postmortem on the food fight podcast (the relevant part starts around the middle of the show):
At one of my past jobs, we worked like consultants (we quoted hours that a task would take to get done, tested etc). We filled daily timesheets to ensure that our hours did not exceed our estimates. In fact, the time on those timesheets had to total 40 hours every week. So we worked the full 40 hours. And we billed those 40 hours to the client. In those few months, I was as productive as I've ever been.
Unfortunately, we weren't paid like consultants.. we got a regular salary. Management was doing nothing wrong. They failed to realize that the 'consultant lifestyle' of billing and working in hours only works if there is an equally motivating paycheck to go with it. I've known multiple people to leave because of how hard it was.
You shouldn't need a person to blame field. Good responsible employees will know what they did wrong and work to fix it. And if they don't, having the field in there will only increase finger pointing and people trying to protect their own jobs. And your bottom line will still not improve. So why do it?
> Put your bosses name in that field for every bug. Convince your teammates to follow that practice, then wait how long this nonsense will last.
> Quit. No, don't look back. Just... quit. In your resignation letter, put Person to Blame: [Boss's Name]
What I would do is try to explain to them calmly and patiently why this is a bad idea. If this does not work, then just refuse putting blame on other people, and encourage others to do the same. If you really must do something childish to protest, put your own name on every bug that you file together with a nonsensical reason why are to blame for it.
Unless you're also awarding gold stars for every bug-free line of code then this is just an unproductive disincentive.
In a culture of blame anything new or different is seen as bad, and to be avoided at all costs, because the unknown nature of the new might also bring blame with it.
They expected a long report with the root causes behind the delay, like unclear specifications, wrong estimation scope, unforeseen technical issues etc. Instead they got back a very short and concise reply: "We will find the guilty, and we will punish them"
Hiring a bunch of shit programmers and putting up a blame field is like trying to patch the Titanic with a band-aid. If your programmers are shit, your boss is the one to blame and should be fired. Adding a blame field will just incite demoralizing outcomes for those who make regular human mistakes.
A root cause analysis should ask "five whys" as to why this happened. Tight schedules, bad specs, improper training? Blaiming everything on the individual who checked the code in is not going to be very accurate, and probably very bad for morale.
I do agree a "person to blame" field was obviously implemented by a "management only" oriented person to probably cover his/her own ass. Essentially this is a scorecard to give to his higher-ups to say "Hey look Bob is the one with all the issues, no me".
But yes, as mentioned, "Root cause" is a better approach.
You actually have to a analysis to determine this though, and not just load it off on someone.
Just note the broken revision and `git blame` or `svn annotate` or whatever equivalent you have. :)
It let's us focus on fixing stuff.
Boss to 'A': what happened ? why the bug ?
'A' to boss: I just checked in the code. It was done by 'B'.
Boss to 'B': hey what happened ? Why the bug ?
'B' to boss: I just merged the code from 'C'.
boss to 'C': hey what happened ? Why the bug ?
'C' to boss: My code worked fine when I unit tested. I gave it over to 'D' to integrate.
boss to 'D': hey what happened ?
'D' to boss: I integrated fine when I did. It is the damn software that might have crashed at the time and code was not integrated properly.
boss to himeself: Fuck it was the damn software.