> I drastically reduced the time developers spent repairing those failures, but there were no metrics that tracked developer time.
For several jobs in a row, I've felt that helping others on a team is undervalued and under-recorded. I've been planning to implement the "assist" metric, similar to basketball, on my own team for a while.
The idea would be something along the lines of everyone gets a set of assist points they must distribute to people who help them the most.
While I don't love the idea of gamifying it, all the places I've ever worked have had too strong of a reverse incentive towards individual achievement, especially when it comes to promotion, and not nearly enough support for teamwork in specific and measurable ways.
Anybody have experience with teamwork metrics like this, or others that worked or didn't work?
I worked in a remote team once helping a company in Seattle (that perhaps had a bit of a jock culture problem). I could be stuck for days because following the README in a repo just wasn't enough to get the project to compile and run. Every standup I was telling "I'm completely blocked for the last two days because I cannot run the repo, so I cannot run any tests and at the moment I'm doing absolutely nothing.". To be sure to repeat a message like that every four hours in channels like Slack.
Managers read it, my teammates in Seattle read it. Nobody ever helped us (because you know pair programming is so much more efficient if you can have two programmers blocked when you could just have one staring out of the window and typing "help" to invisible people). If at all I would be unblocked after a few days usually the problems was some kind of unguessable piece of configuration that should have been part of the README and/or code, a problem completely unsolvable without knowing what to do. Only then I could continue doing a minor change like updating a CSS file for the next Safari or IE quirk, usually just 5 minutes of work.
Apparently helping remote teammates to get unstuck in their work just wasn't an interesting performance metric.
Until this day I still can't believe that everybody was just throwing tens of thousands of dollars of work out the window because helping stuck teammates wasn't counting towards some kind of bonus.
On the one hand it is indeed disappointing that nobody would help you with this. On the other hand, there's a more pro-active role you could have played yourself in this as well. "When everybody is responsible, nobody is" - what often helps is specifically asking people, by name, to sit down with you for a bit and help you get through it.
In fact, there are many social group situations in which asking specific people is far more effective than asking questions "to the group".
If a manager of some kind were to notice the Slack messages and ping someone specific to help, that would be a perfect example of the kind of management that is so sorely lacking at so many companies. Many employees are too burned, or proud, to even put out an SOS... so a cry for help is actually a gift to management. If only it were someone’s job to think for a few minutes each day about each employee and notice if there is some small action that would help them.
Nah, let’s just alternate between micromanaging and being totally absent; fret about lack of productivity or lack of progress; maybe institute a daily stand-up.
Why not both? I cringed when I read OP's comment where he spammed "I'm completely blocked" in the chat without asking for specifics. What a complete lack of individual ownership. Even the best manager can't help somebody who is unwilling to help themselves.
> I cringed when I read OP's comment where he spammed "I'm completely blocked" in the chat without asking for specifics.
You actually expected that what I wrote was a literal quote from the only thing I ever said during those days? It probably was something more like:
"Tried to run project X but kept getting error Y. I tried to figure out where it comes from but it seems to missing some kind of value it pulls from somewhere. Tried every step in the README and installed twice. I think the information in the README is incomplete and there's another step. Nobody else had to use this repo before here so the guys here can't help me out either. Can you help me out @colleagueFromTheUS1 or @colleagueFromTheUS2?"
After 48 hours I get very specific about being completely blocked and that I have exhausted every way thinkable to get it running. So nobody would be living in the fantasy the problem was going away by just ignoring it long enough.
One of your jobs as a manager is to debug these people issues, right? So when you get a "bug report" like "I'm stuck", you should investigate the bug and look for a solution, which might be something like "Hey Bob, Jim is having trouble getting the Frobnicator running on his machine. Can you please give him a hand with this later today? Thanks". (This assumes that Jim's SOS was issued in the first half of the day; want to make sure the issue gets resolved promptly while also not interrupting Bob's own productivity/flow.)
Yes that's true, which is why I said "why not both?" The manager should do more, but so should the employee. Sitting there and saying "I'm blocked, woe is me" is garbage compared to saying "I'm blocked. Can you please help me Mr/Ms XYZ?"
I think your specification of a "correct way to report a problem" is company specific, or your personal preference, or the way that you think an ideal problem report would be stated.
We're humans. Unless there's been a training day or a Readme file explaining HOW to report a bug, I think it's perfectly reasonable to expect this protocol to work:
"I'm blocked!"
"Why?"
"{reason}"
If nobody ACKs the "I'm blocked" message, it's reasonable for someone to think that nobody cares or nobody is listening. This is a normal human protocol, as used in most human environments.
"{problem}!"
"What's happening? How can I help?" (any considerate person nearby responds)
You will get a lot of downvotes for making that statement here... but let me just say I agree with you 100%. The people I see that get the most work done aren’t the ones crying publicly for help on a nebulous problem, but asking very specific questions about specific problems. In cases one doesn’t make progress, escalate to the manager giving details about exactly what is going on, how many times you reached out to another team and got no answers. That gives the manager ammunition to talk to their peers on other teams or escalate to higher ups.
It’s rather unrealistic for the manager to be understanding all the time and coax a team member to give all details of every problem...
That's how I was taught (via managers on reddit actually, not irl).
Dig as far as you can, record it all, then escalate. You also have to gauge how critical it is and how much of your time (esp. if other things are building up) to spend on it as a factor of when to take it up the line.
Tbh, I haven't had any managers lately that have been that great but I'm still holding out hope.
I agree with you that employees that you cannot trust should be fired. Skepticism doesn’t necessarily mean lack of trust. It’s possible to trust dishonest people and without a skeptical view to be aware that people may be dishonest, even in small ways, it is easy to fall into that trap of trusting good liars.
> As a manager, are you prepared to put your real name to these comments?
This is an anonymous public forum with a massive audience and great content. Let's keep it civil and not target individuals. Please don't try and start witch hunts to ruin someone's career just because you do not agree with him.
A new repo, likely a new employee on a new project. I have to disagree with your statement since you don't know the context at all. If all they're given is a repo with inadequate instructions, there's not much more you can do than keep asking for general help.
Comment sounds like typical 'pull yourself up by the bootstraps', meant to put down new talent, ignoring that complex, modern dev environments have many parts that are completely out of reach for newcomers without adequate onboarding.
Like you said, why not both? The employee is already asking for help, that's their part. The manager needs to come in and assist, unless he enjoys churning his dev team.
> I'm completely blocked for the last two days because I cannot run the repo, so I cannot run any tests and at the moment I'm doing absolutely nothing.
That is not asking for help. Asking for help would be "Can somebody please help me? ... rest of message here...". Even better is asking an individual (or individuals) directly.
Nonsense, this is asking for help. As a former manager of a tech team, if I had seen this I would've taken action, and as a team player individual contributor I would've taken action even if I were not a manager. I appreciate you sharing your perspective, but in my opinion you're leaning too far to try to find some way to blame this employee for processes that an employee should reasonably expect a competent management team to have solved already.
Say what now? Mentioning you are blocked, in a stand-up, multiple times is a cry for help. What is the point of having stand-ups if you're not assisting blocked team members?
He mentioned that he brought it up in the standup. The standup is when you announce blockers. Was no one else in the standup listening when he said he was blocked? That's when someone goes oh here is what you do.
The standup is for blockers to be announced. It's the 3 stage of the standup.
Sounds like poor management, don't blame the new guy, he did say he was blocked in standup. How is he to know who to ask. Whose time can he impinge upon. It might be he asks someone who helps him but he gets in trouble because he now took up that persons time from something else.
The manager should see oh he is blocked, I'll assign so and so to help unblock him.
This person probably DID ask more specific questions. They were just describing the gist if the interaction. Not a literal copy paste of the exact messages.
Having my blockers completely ignored for a significant amount of time was a major factor in deciding to find a new job. There are only so many times you can beg for libraries, source code, permissions and other essential tools before coming to a conclusion that maybe somewhere else there is a place where you don't have to deal with this bullshit.
When someone under-performs, I always look at who they report to. Nobody wants to perform poorly. Typically the lead / manager has given inadequate guidance, or there is a serious skill set mismatch, both of which are on the lead / manager.
You should stop being a manager. Or get training to become a better one. Seriously.
When a member of your team can't perform because he's not given the proper
resources(access, help, info, etc), it should be your number 1 job to find out
why is that the case and do everything in your power to rectify that.
I don’t see the reason to assume that was litterally the entire message posted to the channel, it seems more likely OP just pharaphrased what they actually said when the made that commen on a public discussion board.
Nitpick s/he said s/he couldn’t run the repo. If someone posted that in our slack someone would jump in and ask what errors s/he was getting. Once we found a solution we would get the docs updated, usually by the person who found the problem since they have the most context.
People ask for help in different ways based on personality, skill set and experience. It’s more important to meet them where they are than sit back and smugly suggest they lack individual ownership.
Could you please give an example of a message with more ownership? Keep in mind that he didn't know at the time that he was missing some undocumented configuration step.
What kind of company or culture is that, when someone asks for help from a group
and no one volunteers to even suggest to them a person that would be the best candidate to help them?
I've never been at such a company and I hope I never will.
I've worked at different companies that just culturally have very different approaches. Where I am now, I will basically corner people until they help me or provide a better option on where to seek the help I need. At my previous employer, there were certain mailing lists where you could ask questions and if your questions reflected real effort, you could expect helpful responses.
This is similar in emergency situations. When you need somebody to call 911, don't ever say "Somebody please call 911!". You have to look at someone, point at them, and tell them "I need you to call 911". Most people will either stand in shock or think that someone else is on it.
> Most people will either stand in shock or think that someone else is on it.
The bystander effect is real, but I wouldn't expect to see it in organisations with explicit hierarchies. Can you imagine this happening in the armed forces? If this happens under your watch as a leader, you're not good at leading.
It occurs in explicit hierarchies all the time. Watch the opening scene in "Saving Private Ryan." Most of the troops are scared shitless, waiting for someone to tell them what the hell to do. And these were troops who had been trained for 3 years before hitting Normandy.
I like that the problem is remote specific, and the first image you have is “asking people, by name, to sit down with you”. Your image is nit wrong, but a symptom of how we can be hardwired for physical proximity.
Otherwise, a common issue for a team with colocated workers mixed with remote workers is a facility to ignore calls and messages.
Central people get easily overworked, interrupted and drown into tsunamis of requests. One of the coping mechanisms in these situations is to just ignore most requests, and only deal with people getting loud enough to be heard. In a mixed environment, the coworker siting 10m away has an easier time grabbing attention than the icon in the chat room with a unread badge.
Even simply knowing when someone is taking breaks can help to time requests to get them at the right moment.
I have remote days every now and then, and it’s interesting to see how the game is completely different, and how often I end up asking someone physically there to go talk to people on my behalf.
I agree that remote is more difficult than physical, but it's still much easier to ignore a general comment in a chatroom than one that's specifically addressed to you. Most chat applications will even highlight lines mentioning your name.
(Note that being remote in an otherwise physical team makes this significantly more complicated as well.)
It's good you pointed this out because many people don't know. I took a first aid class and they taught us in an emergency situation, never say "somebody call 911!" Because everybody will think someone else will do it. Point to a specific person and say "you, call 911!"
This is a big challenge for remote employees. At some point when do you just give up and settle for what you can get done and not worry about what you cannot change. Its a poor attitude but it happens at a lot of companies.
What makes my stomach turn is the interpersonal aspect. How can all your teammates see you fail through no fault of your own for days and pretend they don't notice. Unfortunately in my experience, that's not a rare occurrence.
I can give some insight from my own experience. In my current position, there are a large number of "things" that I've had to figure out, either by reverse engineering, reading code, or intuition. So I get constantly bombarded with helping others get unstuck. Well, when it comes time for a project meeting, those on the meeting are far removed from the other groups that I constantly have to help out. And it sounds like I'm just giving excuses of why my work isn't done.
So if there is a micro-crunch coming up (i.e., I need to have an SOP drafted, tested, etc by this afternoon), the entire morning I tend to blow off other peoples issues. Until I can get a chance to come up for air.
But that makes me feel bad, so what I end up doing instead is I go home at the end of the day and work till past midnight just to get caught up on what I couldn't do in the daytime. For any outside-the-box work I tend to pull 20-30 hours stints over the weekend. After 25 years of this, and as I need more sleep since I'm getting older, I can be a bit snippy when someone breaks me out of my concentration because they can't figure out something, and they have a down customer that needs service "right now".
It is also easy to get irked when people are seemingly unwilling to do any digging or reverse engineering on their own and go straight to asking for help. Allowing someone to search for an answer for several hours may seem like a waste of their time, but it also could allow them to learn how to figure out a problem on their own. Of course their is a fine line between being a jerk and not helping someone because you want to maintain your technical superiority and not helping someone because you want them to learn how to problem solve. Clearly you do not want to leave someone roadblocked for 48 hours when they are trying hard and you know the fix is some stupid bug. But, it's not always super straightforward either.
Yes. Unfortunately, by the time you start to think people are asking for fishing lessons just to get lunch, it can become difficult to recognize a true student.
I started out as the eager mentor, even when it was among my peers in university. But somewhere in my ~25 years of R&D, I started to notice how coworkers would become dependent. They would ask for help on types of things that I knew I had figured out myself, even when I was a novice. They develop the plea for help as a first reflex to avoid any learning tangential to what they consider the task at hand.
So, I found myself swearing under my breath and refusing to be their search engine or man-page. Depending on my mood that day, I might tell them I have no time and they should google it themselves, or I might pretend I don't know the answer either. Sometimes I relished the time to sarcastically school them on the mundane deductions it would take to answer their question from the very screen full of error messages they pasted at me. But, I already knew they would miss or ignore the hint, and be back again next time they stumbled.
Sadly, I think this cultivated impatience leaked out into my attempts to mentor a very junior staff member we had a couple years ago. I could no longer really distinguish "too lazy to try" and "drowning". They found a new job after a year or so, and it was probably wise for them. I am still trying to visualize how I can do better the next time we try to onboard somebody far junior to our current team...
I struggling with this constantly. If I have to fix another VPN issue caused by weird DNS server entries in OS X I’m going to scream. “Anybody else getting this error?” No, figure it out. I’m at a loss to explain how so many people wind up in software dev roles while lacking the most basic of problem-solving skills.
Capability and responsibility become conflated very easily. You start off as the superhero who can fix problems, but that gradually creates dependency.
1st year
'Thanks for your help finding that error that was holding us up.'
2nd year
'We are all finished now. Just sent it over to you for debugging.'
3rd year
'Hey we are all waiting on you to fix. Big deadline for big customer.' CC: management
4th year
Management: 'we need to talk a about the regular delays here. We have several teams waiting on you. Also your own code checkin's are much lower than theirs. Seems like you are only making changes to what others write.'
Managers inadvertently create this dynamic when they are focused on attacking obstacles rather than creating infrastructure. If <employee> knows how to do something, and that thing is not getting done, then <employee> is the obstacle. The discussion often doesn't make it to the point of 'why doesn't anyone else know how to do this?' Management needs to think more like logisticians rather than tacticians.
From an employee standpoint, when helping your peers ask them to take notes. Any additional requests -> ask them to refer to their notes. This sets a clear boundary and gives you a built-in defense while still helping the team achieve goals. Now you are no longer the obstacle, because you provided training. The obstacle is now <Peer's> inability to properly take notes or retain information. <Peer> knows this and is much more likely to attempt solutions rather than nag you into doing their job, or escalate to management. Additionally <Peer> will have better knowledge retention by synthesizing their own notes.
Documentation is a good barometer of this behavior (it's the first casualty). If your environment has little documentation, then you are probably looking at a group that will punish responsibility. No one documents anything if all that earns them is 'John's instructions don't work, I've asked him several times to fix it'.
Start by actually talking to your management instead of just trying to be a rock star. "Hey, I'm spending a lot of time doing Team X's job and it's impacting on my assigned work, if you want me to keep doing this then put me in charge of Team X, otherwise let's all sit down together and clarify who is responsible for what."
So I'm asking this not to move the goalposts, but because it's the situation I'm in - what do if your manager is too weak/distracted/uncaring to actually act on this information and/or the projects in wait state on you are not actually within your manager's sphere of influence?
Hmm, tough one. The default answer is "follow it up the chain" - if your boss isn't doing the right thing, talk to their boss, and so on. Another approach is to talk to the manager of the team who's leaning on you, and let them know (if they don't already) how much they're depending on you and that you should really be in that team. This can cause drama with your immediate boss (they're seen as 'poaching' you) but if said boss doesn't care then this might not be a problem.
Of course, by the time you have upper management yelling at you because "several teams are waiting on you", the response becomes "why have you allowed several teams to become completely dependent on one overworked person?"
The other, more drastic solution (as someone else said) is simply to move jobs. This can be the only way to fix the issue if the entire management structure is corrupted and you don't think it's fixable from within. You could even just leave, start a consultancy company, and let the second team know that you're available for hire when they need you.
Unless something is keeping you there, just find a new job, here are some reasons:
- If you reach this situation, you're more competent than the rest of the group. Nobody to learn from, time to go.
- "Management" is a big machine, with its own insider culture and politics. It does not change overnight, do you want to wait 6 months minimum for things to get better?
- Changing job is good for just about everything: career, money, knowledge... as long as you don't do it too often.
- Meta-bonus: getting into a job with better peers means tighter work-friends. It's not fun to be around people overdesigning object-oriented projects when you understand functionnal programming and low-level debugging.
That's neither healthy nor sustainable. You're the one that sees a problem; you absolutely need to make your boss aware of that situation and that the company needs to arrange for better utilization of man-hours. If management doesn't fix that to be healthy, consider moving to a healthier position.
Just to try to put this in a (unfair to you) constructive context:
> I could be stuck for days because following the README in a repo just wasn't enough to get the project to compile and run.
>> I can give some insight from my own experience. In my current position, there are a large number of "things" that I've had to figure out, either by reverse engineering, reading code, or intuition. So I get constantly bombarded with helping others get unstuck.
This is why I always try to follow readmes, log my progress and fix errors as I find them. Also document hidden assumptions etc.
That way others can help themselves - follow the readme, look at the up to date network/architecture diagram etc.
Partly this is because I really cannot be bothered to keep such details in my head, anyway.
> Well, when it comes time for a project meeting, those on the meeting are far removed from the other groups that I constantly have to help out. And it sounds like I'm just giving excuses of why my work isn't done.
Honestly, there's not much you can do if you feel like your company is not acknowledging work that you believe is a valuable use of your clock time.
What I notice more senior people do is 1) expose clear SLAs to require their help and 2) get in the habit of producing more artifacts while they concentrate so they can refocus faster after an interruption.
I'd best describe myself as a "computer person". Other than the first few years when I first started at around 15, my job has been Unix Systems Administrator / Engineer, but I've always thrown a lot of programming at my job too. Not just systems automation programming, but various utilities, apps, etc.
For example, back in 89 (I was 18 - 19 then) the company I was at had a need to access multiple screens at the same time in their green-screen (serial terminal) application. So I wrote a multi-session terminal emulator. Ended up being a lot like what GNU screen is today (not sure if screen existed then, this was pre-internet days for me).
Other times I'd write up software distribution systems, monitoring systems, etc. Or work on creating things like a custom interpreted language, just because an app developer at my company needed that. Or write hand-coded postscript, because someone's app needed that.
In my current company, I'm still a Linux systems engineer / Lead, but I've been on-loan to our app development team to help with C-based libraries that they need (this code is a couple decades old, and they don't have any C hackers that can work on it). This is software that interfaces with medical testing equipment, so it feels like I'm making some difference.
Lately I've been brushing up on my web development skills, becoming buzzword-compliant, and putting together web apps both for internal use, and related to maintaining our product.
I guess I should have went into full time development years ago, but I was always worried that I'd get shoved into boring business applications instead of working on really interesting stuff. So I've stayed in the Unix/Linux ops role, found out that a lot of what I've been doing is now called DevOps, and doing open source work on the side.
As for the trajectory, I haven't changed jobs that often. I've had a couple of dud-type jobs where I lasted 1 - 3 years (they didn't want someone who could do "extra" stuff, because that "made the rest of the team look lazy" as I was told in a performance review once). But when I start at a good place I try to make a good first impression, then a better second impression, and try to work and support as many people as I can (while doing my best to make them look good, but making sure I get recognized too). Helps to have a number of people that "have my back".
I've actually slowed down a bit in recent years, however the main driver is that the work is fun. During the day I do the necessary (but not necessarily fun) work. After hours, I work on stuff that is mentally challenging and needs concentration, but only if I find it fun to work on. I do this instead of watching TV for example. As long as I get in enough time for other activities (hiking / bicycling, woodworking, designing my own creations, time with family) then hopefully I have a balance. But I have always had a slightly unhealthy fixation on sitting in front of a computer screen and creating (whether I'm doing it for work, or personal projects).
Having been the 'goto build guy', I've had to cut people off who were seeking me out for very self explanatory issues. For example, when the build threw an error because it couldn't find a file, pointing out where in the error to get the file, but the person seeking help didn't even try reading the error. Basically I limit my intervention to prevent enabling helplessness. The amount of help given also depends on the role of the person as well, junior devs get more hand holding then senior devs.
This to me sounds like a complete lack of self-ownership on your part. You literally sat blocked doing nothing for days, without reaching out to an individual to help with this? Instead you just spammed Slack saying you're blocked?
While I agree that it would have been nice if somebody helped you, you are in control of your own destiny. If nobody is helping you, you need to personally escalate it and reach out until you get the job done. If somebody did this on my team, the first question I would ask them was why the hell they let themselves sit blocked for days.
...and of course my team would have been more proactive on helping you because one of our core values is help your co-worker, but that is besides the point here.
And even if everyone completely ignored them when they direct messaged their teammates... why do nothing. Why not dig in and figure out how to get it to build and update the Readme? I don't understand why someone would just sit there and do nothing instead of trying to fix the problem.
I did that once and then rewrote the repo. Everything was fine for months on our team until another team merged their exact copy with new changes that broke what I was doing. Everyone senior left my team and it looked like I was doing nothing for 4 months. Communication should come first and the manager needs to step up.
I agree that communications is important and was the primary thing wrong here. It was more that even on top of that, they just sat there doing nothing vs. at least trying to be productive and figure it out for themselves.
Sure I have backups. The other group took a copy of the repo and changed it over a three month period and never checked back in. The versions were very different and merging my copy in was difficult and required a rethinking and rewrite.. in the end it became three separate components. All of this happened during a management change so the timing wasn't perfect.
It also appears to assume that someone could help them. There are plenty of times where knowledge of a project or component is simply lost, either the developers don't remember or are gone, it's one of the costs of employee turn over.
OP said slack. DM is as simple as doing "/msg @joe.blow I'm blocked, can you please help me?". If no response, go to @joe.blow's colleagues. If still no response, go to their boss. If still no response, go to their bosses's boss. Repeat until you run out of people to escalate to, at which point you find another job.
If they are willing to pay a blocked remote worker for sitting on their ass doing nothing, why should the worker rock the boat so much that they burn all bridges at the company and finally have to get a different job? Remote jobs seem somewhat hard to come by and can be a sweet deal.
The company needs to take at least partial blame here.
No need for worker to do that. But that's then their conscious decision and they shouldn't express their frustrations over it. On the other hand, if they want to make a progress, they should escalate appropriately.
Note that I am not convinced that this is the case here - I simply don't have enough information to make the judgement.
So someone wanting the system they are a part of to work appropriately without them having to be an excessively aggressive asshat about it needs to swallow their frustration entirely and not even vent about it on an online forum?
Even!more than that, at some point you start including "You are paying me $xxx/day to do nothing productive because I'm unable to get 15 minutes of time or even acknowledgement of requests from persons X, Y and Z."
My other thought was to file blocking bug/defect issues on the relevant software because obviously if it can't build in the first place it's certainly not passing QA.
Some of this may get to the level of passive-aggressive or aggressive-aggressive bridge burning.
> Every standup I was telling "I'm completely blocked for the last two days because I cannot run the repo, so I cannot run any tests and at the moment I'm doing absolutely nothing.".
This happened to me once a few years ago, once I finally figured it out I put all this silliness into one single bootstrapping function you could call to get the pig of an application to run, and asked the lead dev to add it to the core library. Later that day I got called into my manager's office as apparently my behavior was disruptive and problematic.
Maybe it's not the initiative per se. Did you name the function silliness(), runPig(), or give any other blatant display of contempt? It's dripping from your comment.
(not saying it was necessarily about something real, just that it could be)
Nope, any contempt would have only been after the other developer displayed his contempt for productive use of shareholders money. I always find it funny how so many people seem to think that is a matter of opinion rather than a matter of fact.
This is endemic to our industry. The problem really is short-sighted management. They only value something that shows themselves in good light with their superiors.
Managers absolutely don't seem to care about actually building a team and cultivating a culture of success.
Engineering managers are way way worse than Soccer managers in this regard.
I would put this all on the hiring processes right from how we hire engineers and managers and what gets a promotion. But unfortunately, our industry would rather remain blind.
The people usually tasked with figuring these things out are new to the team. There can be all kinds of politics and interpersonal baggage with helping the new person with old configuration/ compilation/ deployment stuff.
Often, it'll take one of the legacy leads to step in and clean things up. By then, everyone's annoyed at the new person, and the failure of this effort will get saddled on their back for some time more.
I had this exact issue at a recent gig, and the application architect spent months poking at jenkins to get a clean build. Once he quit in frustration, one of the legacy leads re-engineered the whole thing, in a different language, in a few days.
Legacy lead was one of those "untouchables" in the eyes of the CEO, so there was no influencing him to help earlier, and his reticence cost us a good application architect.
To my mind, this is often a sign of a poisoned, political company culture. The work can be done, but there are non-technical reasons why progress can't be made, even by earnest effort.
Often the non-technical reasons are "management doesn't want to assign the super-lead to the work, but also doesn't want to let the new hire have free reign to rewrite a legacy product. Just fix it."
I've often wondered whether this dynamic is exactly the reason tech culture is ageist. If we hired older developers, the culturally-assumed "gravitas" of age would make their relative social status to more-"senior"-but-younger devs illegible. We might actually have to sometimes give them the right-of-way on making the changes they want to make, even though they're new!
Every company I've worked at for software development has had review / feedback opportunities for my manager. I can't imagine working at a company where feedback from direct reports wasn't a major factor in someone's job performance evaluation.
Then again, I've been very selective in where I work. People like to bash the "culture" stereotype, but it's definitely something I consider when I'm looking for a job... you know, how people are expected to interact with other people, not whether there are ball pits and the like.
I don't think it's specific to our industry. I think it's specific to the corporate system we have, and that it correlates with the sense of ownership and freedom that we give people. When it's "every man for himself", you're going to see that optimization toward individual metrics and protection of personal reputation, even if it means that a colleague's reputation is deprived.
When you have a recognized long-term leader that doesn't need to constantly re-justify their involvement and existence (something akin to a BDFL), more collaborative group efforts can be fostered, and individual gamification is much less useful, because the people who know your behavior are going to be there for a long time and there is a lot of personal continuity. This system is, of course, not without its own type of flaws, but it doesn't have the same negative feedback incentives for individual contributors.
It's hard to get that long-term involvement in the tech industry specifically because the rapid rate of technical change limits career length and portability. Skills in language/system/framework X rarely retain marketability for more than 3-5 years. Most people don't have the stamina, interest, or whatever it is that's needed to keep up with the constant re-education and re-justification cycle into their 40s and 50s. They look for an out: early retirement, promotions into non-coding management positions, etc.
To the extent that individual metric gamification is higher-than-normal in tech, I would attribute most of it to the effect of shorter career tenures, both overall and at specific companies.
I think short-term thinking by superiors causes short-term behavior by reports.
If the management pays fairly, with fair raises, fair balance, fair acknowledgement of all kinds of work and not just the flashy stuff, most reports would focus on the right things, do their jobs well and not look to switch jobs all the time.
But if the management is always trying to escape giving promotions/raises, does not acknowledge the underlying effort to get something to production, only thinks about their own empires (basically short-term selfish thinking) it will cause a disproportionately high number of reports to behave the same aka look out for their selfish interests.
I am absolutely certain that top companies look to retain good players (top != stock price or market cap).
I can't say for sure but your mention of 'CSS' makes me guess this is a web application, in which case you should be able to read the source code and figure out how to build it. If people aren't helping you then according to their cost benefit matrix it isn't worth it. Maybe they are supposed to be doing other work and they can't calculate how many hours they would have to spend helping you. Maybe they are just afraid of not knowing how to get it running either and will get entangled and implicated in what is currently YOUR ignorance alone. (I've worked with plenty of these crappy apps with undocumented dependencies where if you get the AUTHOR of the repo to try to build THEIR project on a machine other than their own it craps out on them in which case they fucking bail or defiantly say 'it works on my machine'. These same people will snort derisively if any junior tells them of problems building their project, with a line like 'its not rocket science you know' -- until you confront those mofos with the demand to actually build their own fucking project. At that point you discover that not only did they not document all their dependencies, but they can't even read a simple node error message complaining about how node-sass is not defined, because their project depends on node-sass being globally installed and they didn't realize it themselves.)
>your mention of 'CSS' makes me guess this is a web application, in which case you should be able to read the source code and figure out how to build it
pfft. If it's a web app (from 2015-2018) I have even more sympathy for the OP than on any other platform.
Here's an example from last year-ish:
* Get repo from external contractor (let's ignore the 1+ weeks trying to get an install of Git approved)
* Try to get npm working - can't get through our firewall
* Figure out the right chaining proxy to use
* Spend a non-zero amount of time trying to work out all the right bits to flip to turn off ALL SSL verification from npm because interception proxy
* Hit the Windows MAX_PATH limit and have to spend ages trying to delete files that NPM's created (original developer used Linux)
* After npm starts downloading stuff correctly, suddenly it needs a binary dependency (and thus more build tools)
* Give up and check in said binary dependency from elsewhere
* Finally build all the CSS and minified Javascript and check it all into source control with a warning to other devs not to rebuild it unless absolutely necessary
> I can't say for sure but your mention of 'CSS' makes me guess this is a web application, in which case you should be able to read the source code and figure out how to build it.
Not necessarily. In my case, we have a web application where it eventually turned out that the variable was read from an environment variable. On a server where I didn't have access.
Isn't the Toyota's idea of Kanban supposed to alleviate this? As in if there is some problem with the production line, anyone can push the stop button, and everyone assists?
Just to clarify your point, I believe you are referring to andon[0], the system of alerting people to an issue, rather than kanban[1], a system for inventory management.
In a manner of speaking he did pull the andon cord by speaking up in his team meetings and alerting people in Slack, but the cord means nothing if there is not a culture supporting the alert.
No, the equivalent would be as soon as he raises the issue, no one can type even one more line of code, not even a character, and their editor would be completely blocked.
That’s what these companies need. The entire business comes to a screeching halt.
And if they really needed more motivation, production servers could be shut down if the issue was not resolved in 24 hours. That will really hold their feet to the fire.
There are other issues at play when it comes to helping out remote team members.
When you're co-located with someone you develop better comradery with that person. You'll know them better and they'll come right to your desk when they ask for help so you're more likely to help them. And you sit with them as they execute commands so you can see exactly what's going on.
remote coworkers are easier to ignore, harder to help, and harder to socialize with.
I have generally chosen to deal with this problem by refusing to acknowledge the existence of bonuses or promotions. It is harder for a warped incentive structure to warp your behavior if you refuse to be motivated by its prizes.
That's noble, but many aren't inclined trade personal profit for the company's profit. I don't think they should be, either. It's the company's job to ensure that their success is aligned with their employees'. Raise the issue to management, and if no action is taken, then you've done your due diligence.
I can imagine situations where you should care about the company's performance, like a non-profit or certain startup situations, but I think the majority of enterprise developers fall under the above.
> I could be stuck for days because following the README in a repo just wasn't enough to get the project to compile and run. Every standup I was telling "I'm completely blocked for the last two days because I cannot run the repo, so I cannot run any tests and at the moment I'm doing absolutely nothing.
Certainly it might've been more efficient if one of your colleagues helped you debug the issue. But in the absence of help, it's not clear why you didn't dive in and start debugging on your own (rather than doing nothing). You'll learn more that way too.
That's not really undebuggable. Process Explorer shows registry calls, and IIRC even marks them in red when the key is missing. Again not arguing that this is efficient, just that doing nothing is less efficient.
I generally assume that when someone says "blocked" they're not literally staring out the window, they're just not operating with anywhere close to the efficiency they should be.
Also it's not clear to me that learning how to use Process Explorer + using it, when that's not your job, is actually meaningfully more efficient than staring out the window all day.
If learning to use Process Explorer can aid you in performing your job duties, then I feel it would definitely be more meaningful and efficient than just staring out the window. I can't judge OP's situation because I don't know all the details, but I think there is something to be said for being willing to dive in and "unblock yourself" like some people have said. There's obviously a gradient between being helpless and the situation being genuinely out of your hands, though.
I think the point was that it's silly for someone who's meant to be on a team to spend possibly days trying to debug something as silly as that, instead of someone on the team just spending the few minutes to help them and give them the registry key they need.
It's not time efficient for everyone to work separately down to reverse-engineering the machine code to work out what's missing every time.
Are you a junior dev or was the work in a domain unfamiliar to you?
If not, you should be able to at least start debugging. You should get some error message (maybe that indicates some env variables not set, something not installed, etc). It seems odd to have not spoken directly to your supervisor or product owner to say "hey, this isn't working because X. What do I need to do / who can I talk to for help?"
Most jobs I've been at if you're the first new hire in a while you are going to experience something like this, and it's important that you spend a lot of time adding to the documentation to make it easier for the next new hire.
I don't have an answer to your question, but I've certainly felt that pain.
Years ago I worked at a desktop software company that hired me to be their "Windows expert". They had a team of strong programmers who mostly had a Mac background, and they wanted to improve their Windows offerings.
I often spent a couple of hours a day doing what I called "house calls": finding out who was having trouble with a Windows issue and stopping by to lend them a hand.
This was before "pair programming" was a common term, and full time pair programming would drive me nuts, but these house calls were very productive. The other team members knew the product code much better than I did (being a newbie), and my Windows experience helped them work through platform issues. Plus sometimes just having another pair of eyes look at a problem.
Often a 20-30 minute house call would save a team member many hours of work.
At one point, a teammate thanked me for helping with a tough problem and asked me, "How can you afford to be so generous with your time?"
Some time after that, the VP of engineering (who I'd worked with previously and was on good terms with) asked me if everything was OK. He had noticed that the other developers on the team had lately been more productive than ever, but my own productivity wasn't quite keeping up.
I think I mentioned that some of my time had gone into helping the other developers work through their Windows and other issues, but I didn't have the presence of mind to say, "Maybe my help has been one reason their productivity has gone up. I thought that's what you hired me for. Can we make it part of my job description?"
I've never tried it, but I suspect you'll run into the same issue that sports does, where "assists" don't accurately reflect the people who are making things happen on the court.
We miss the perimeter shooter who positions themselves just so to pull two defenders out of position, but give credit to the point guard who makes the now-easy pass for the "assist". It's entirely possible for the person most responsible for a play succeeding to never touch the ball. You really want something like value-above-replacement, but that's pretty hard to measure without a clear definition of what "success" is and reams of data.
> We miss the perimeter shooter who positions themselves just so to pull two defenders out of position, but give credit to the point guard who makes the now-easy pass for the "assist".
The NBA has an advanced metric for that called gravity. As you could imagine Steph Curry's gravity is through the roof.
Another related metric is +/- for when a player is on the floor. That one is more nebulous and error prone but it is meant to describe a player's impact on the floor outside of things like assists, points, rebounds, etc.
> The NBA has an advanced metric for that called gravity. As you could imagine Steph Curry's gravity is through the roof.
Happen to have any additional reading on that? (only if it's a handy link for you, otherwise I'll google it). I've re-started watching NBA recently after about 20 years or so, and it's the first time when I hear about this concept, one which sounds very intriguing and interesting. Maybe it was also used 20 years ago, but back then I didn't have access to Internet and as I don't live in the States the local sports newspapers (which care more about football/soccer) weren't mentioning it.
I can't think of any one particular article that goes into the details of gravity and some of the related metrics (respect and distraction being the others). Maybe lookup NBA gravity scores and SportVU. I believe the company that sells the SportVU system are the primary ones tracking that metric (and delivering it to teams).
Gravity isn’t metricized (and it would be hard to do so), the best we can do at the moment is indirectly measure it via +/- or other stats like what SC30’s teammates’ true shooting percentages are with him on and off the floor.
Second Spectrum has supplanted SportsVU for tracking, and is also the generator for all the advanced metrics that are now provided for the NBA as well as ESPN.
You're forgetting that you could also track secondhand assist as well.
I can't speak for programming but in soccer this is well-known. It's just very hard to Value just like an employer would in terms of having that person on board for morale. Fortunately the better people get it and understand that that person should be value even if they weren't directly involved in making the change but they were definitely a part of it.
This is what plagues me with most corporate companies in the United States. It should be about teamwork not about what this person did or I did, rather what the team did.
Sure you can value people individually, but teams trump individuals all day long in the long run.
A person’s projected morale is very under-appreciated. I don’t know if I’m at the point in my career that I boost morale just by being on the team, but I do know there are people who, if they left my company, would have an impact on my morale even though I never ask them for help or work directly with them.
Just something about knowing a person is an amazing and knowledgeable resource and he’s choosing to work here of all places is a subtle morale boost. When they decide to leave, others might start asking “well if that guy left, am I still making the right decision to stay?”
I’m sure we’ve all experienced someone important leaving our company and having a waterfall of turnover that follows it. That is very disruptive and more should have been done to keep that one employee or at least keep the rest that followed them out the door.
I've seen people like that, and I've been that guy. Ironically, I've been that guy only because I left years after those other men and women that I'd looked up to had moved on, and I'd filled their role for the junior devs who were hired after.
Moving on is a fact of life, more so now that ever. My parents both worked for the same company over 35-40 years before they retired; I don't think I can imagine that'll be even close to the norm for many in the workforce now (especially software / other tech).
A family friend (manages a programming group) kept a basically incompetent but calming, friendly programmer on her team for 2-3 years because of his effect on the emotional state of the rest of the team. She his pay was worth it for that reason. Unfortunately the company's owner eventually realized that he was paying the salary of someone who couldn't program and fired him.
The problem is much more about underperforming teams than well or overperforming teams. In order to dissect the problems of an underperforming team, they need some objective metric (even if it doesn't tell the whole story) to look to, or else you have to deal with people flinging accusations around and poisoning the entire culture.
This is what +/- and its variants are for. Especially useful for sports like hockey where scoring is rare. Of course, in a work environment this is even harder to try and measure, if for no other reason than that people are almost never 'off the court' and if they are there are many confounding factors.
Yeah, there's a bunch of advanced stats that more accurately model this in sports. My point is twofold:
(a) despite the existence of better stats, a lot of people still care about goals/assists, because they're much easier to understand.
(b) the difficulty of measuring these stats in low-scoring sports (soccer, hockey) is greatly magnified in business; instead of scoring O(1) points per hour, a small software team may "score a point" every couple weeks or quarter or year. Additionally, the number of people contributing is often greater, and we see fewer "identical lineup, but with player A swapped out for player B" situations (and ~never for long enough to get meaningful stats about "points scored").
This is what I worry about, and what I hope to avoid by asking who's tried it. I am hoping that my analogy is more imperfect than that. ;) Requiring people to hand out an assist quota to anyone that helps them, I hope, might be closer to each player rewarding assists to anyone else who passes them the ball. Since it's all players giving assists for any reason to each other, and not a single ref only handing out assists for the shot, perhaps that will allow for a fuzzier definition of success and a system that is harder to game?
My experience coaching a college sports team for a few years is that even very good players are often pretty bad at assessing the contributions of their teammates. They tend to significantly over-weight the contributions of players who have a similar style of play to themselves, and underweight the contributions of players who play other positions (or play the same position differently).
They especially undervalue actions that happen "away from the play" (the software-engineering equivalent might be something like rewarding a teammate who steps in to help you fix an critical block-ship bug, missing the person who had the foresight to write a test case two years ago that prevents a bug from occurring in the first place).
The NHL at least takes into account not only the person who passed to the player who scored, but sometimes also the person to passed to that person as well.
In one of my previous teams, we did away with individual metrics because they "punished" team players. If an engineer spent most of a week helping another with their issue, it made them look bad in whatever tracker we used by almost every metric. I prefered sticking to trying to optimize whole team velocity and using 1 on 1s and peer reviews to track individual performance where it was needed (eg: compensation review and promotions).
It's still not perfect: people could be spending them helping someone from another team and that would go under the radar (except the peer reviews), but it still worked a lot better.
relatedly, a friend of mine worked at a place where everyone was on "20% time = bug time". You were expected to spend ~ 20% of your time taking bugs from the bug tracker and fixing them. Bugs closed were part of a metric towards bonuses.
People were taking one bug, breaking it in to 3 or 4, then closing all of those.
"Add button for X" became
* "add button on form"
* "add CSS for button"
* "update CSS for button"
* "add CSS files to build process"
Friend of mine took a bug - it was a moderately big one, touched multiple systems, but was a bear to work on. No one else touched it. He spent a few weeks making it work. Eventually closed it. Was reprimanded because his bug-closing stats were bringing down the department's numbers. He was counseled that he should have broken it down in to a few dozen issues, so that he could have closed more and "kept up" with his team mates.
No one else would touch that bug. Most people hadn't been there long enough to decipher how complex it was. But.... they got bigger bonuses because ... bug closing numbers.
We all probably have similar stories to share, and I know the "what gets measured gets gamed" mantra.
It's still frustrating to know that this mentality infects so many professional "thought workers".
> It's still frustrating to know that this mentality infects so many professional "thought workers".
The article explains the process by which good, honest workers become corrupted into this mentality. Whether you want to do good work or not, most people don't have the confidence, security, nest egg, etc. to quit their day job, so they assimilate out of the necessity to keep their bills paid, and this becomes a cycle. The naive worker comes in determined to do a great job, soon learns that "great job" is not a straightforward evaluation and that people who have no real involvement, knowledge, or investment have control over their career trajectory, and then optimize to improve those numbers.
These people then get promoted and a) consider failure to game the system evidence of naivety, and expect this same political awakening in candidates as a "rite of passage"; and/or b) rise through the ranks without realizing that they've gradually internalized the system, and that they're inadvertently making the same loose determinations based on impersonal, macro-level metrics that are easily gamed, partially because as you get promoted, you can't possibly come to a deep intuitive or direct understanding of the work of everyone in the ranks. Rinse and repeat. The ABCs of employment are "Always Be Campaigning".
Strong, personal leadership from invested, intelligent people with conviction is the only antidote, and even when you get these people, it's a constant battle to keep their disinclination for political games from destroying things.
Ah yes, bug counting. There is an even worse variant: when the manager only looks at the number of open bugs/issues, rather than what has been closed, or code that was checked in, etc. I had the misfortunate to work under such circumstances.
I was supposed to complete an existing project. Only problem was, new bugs kept getting added by the tester, and were then assigned to me by the project manager, so the number of issues kept growing. I would fix them (or existing ones) during the week, but at the next conference call it would still look like I was not making progress, because the total number of open issues did not shrink much.
This led to weekly inquiries of "when will it be done", and pointing out that it was essentially a moving target, due to circumstances beyond my control, did not help much. Eventually I did catch up, but the damage was done. Apparently this manager now thought of me as somebody who wasn't very productive. They moved me to another, completely unfamiliar project, where I got essentially no help from others, then decided that I was not productive enough and let me go.
In retrospect, being asked continuously when the project would be done, was an indication that I was the only one being blamed for its delay. I was afraid to rock the boat though, and quietly assumed that surely this manager was taking all the factors into account (especially since I did point them out a few times). As it turned out, they were not, and I will not make this mistake again, should a similar situation come up in a future job.
This is really at the heart of why most people don't speak up (or speak up more loudly) - fear. Fear of losing the job, income, etc. There are no easy answers to this, short of having - if not FU money - a comfortable cushion to see you through a year or two between income streams.
In my naivety I've never heard nor thought about "what gets measured gets gamed". It's been my intuition for a while, but it explains much of what used to frustrate me -- I wish someone had told me this 15 years ago.
It's worth explicitly considering the two concomitant problems:
What does not get measured gets ignored. It's not necessarily a bad thing to measure stuff. But it's easy to overlook a metric that later turns out to be important.
If you're measuring the wrong stuff, or measuring it in the wrong way, your metrics are again erroneous and your decisions will be based on ghosts.
Lao Tzu was truly onto something when he said "the more I learn the less I understand".
This was one time where I appreciated living in the bay area. I found out 3 months into a new job that such metrics mattered. I was out before 6, and could've been before 4. Now I know to ask questions to prospective employers and look for signs of such crap.
It sounds like you need a 'reasonable person' clause - when considering bonuses, each metric gets looked at by a randomly selected group of peers and they can jointly decide if someone's gaming the system.
To take a narrower view, it sounds like that place needed to assign weights to the different bugs, so that the big scary bug was worth 20x (or whatever) of the trivial 'tweak the CSS' bugs.
Yeah we don't do metrics, we gather short appraisals from a list of people that you provide, that you worked with throughout the year. You are encouraged to provide names of people that are outside of your group. So if you helped some other team, this is the way you cash in on that.
Helping other teams succeed is a cultural thing and your organization has to encourage it with its practices and procedures.
The issue then is the other side of the story: you end up with popularity contests. Someone with a silver tongue will always win in peer review-based processes. No solution's perfect.
That is gamed too. You just don't include the names of people you have wronged: the bugs you have left for others to fix, the undocumented code, the projects you were supposed to complete but got reassigned so that you could work on something with higher promotion possibilities, the delays you cause in other people's work, etc...
This is exactly what I've seen happen in my experience and have often wondered about alternatives. Bridgewater, for example, uses a system that allows for feedback from anybody at any time [0], but I suspect that presents new issues like negative reviews from rival employees. Additionally, a lack of context could result in negative feedback for decisions that actually help the business (e.g. passing on requests to help colleagues with relatively low-value items in favor of focusing on higher priorities).
Nope; the benefit of being a good team player is that it helps you build relationships. Those relationships often turn into opportunities down the line. But it’s hard to drive that behavior at an individual level without creating some conflicts of interest.
This is why I generally dislike measuring individuals against specific metrics. I try to set individualized metrics for all of my direct reports — for example, if I had a developer who keeps checking in code without valid tests, code coverage might be a metric I track with him just to help him understand how he’s doing against his personal goal. Even within the same job description there is often room for a significant variation in technical and interpersonal skills, and you need a team with a good mix of both to be successful. A one-size-fits-all approach to individual metrics just leads to a team of people with the exact same skill set.
I think the way you measure teamwork is at the team level by having a consisitent set of team-level metrics the entire team is judged against. This means individual personality and career path variations don’t matter as much. I’m talking things like say, number of broken builds, velocity improvements, production issue counts, etc. These should be business outcomes that the product folks care about. If you make it the team’s responsibility, you’ll find teamwork happens more naturally.
This approach to performance management is in direct opposition to stack ranking, and a big part of why I refuse to work for a company that forces its managers to stack rank within teams. The question of “what makes a good employee great?” is so difficult to answer that if you have multiple high performers, it’s effectively arbitrary. It also tends to punish people who are maybe less technically brilliant, but have strong interpersonal skills (after all, how else does a mediocre programmer make it through a CS degree?) Those interpersonal skills often lead to better technical discussions and avoidance of miscommunication between technical folks.
> I think the way you measure teamwork is at the team level by having a consisitent set of team-level metrics the entire team is judged against. This means individual personality and career path variations don’t matter as much.
Interesting management approach. My questions are:
1. How do you account for new team members in a team's metrics? A new junior dev will take some time to become productive so do you add the new member's progress to the team's metrics by making it responsibility of one or two experienced teammates? For instance, you might have internal progress indicators like - by month: 1 the junior should have read access to certain repos that are crucial to a good understanding of the application architecture; by month 2: made a certain number of commits; by month 3: written X number of tests etc.
2. How do you use these team metrics to recommend promotions and raises for individuals?
1. I normally just dedicate sprint points to training up new resources for a few sprints. I’m not so heavy-handed with the internal policies so it’s up to each team how they do it, but most have some form of pull request review / gating process.
2. I don’t! Well, not directly. Team metrics usually have some impact on bonus money and raises, but that’s all based on budget so it’s not consistent year-to-year. Metrics should be used to drive behavior, but behavior should be used to determine promotion. Is the person a team player? Have they developed the skills necessary to be successful at the next level? What does the next level look like for them — is it management or engineering? A lot plays into it, and you have to have career path discussions with folks. Can’t solve this one with metrics :)
Good points but the thing that will prevent adoption of your approach is the element of variability.
Businesses don't handle processes that produce variable outcomes very well since it opens them up to risk. Or stated another way, businesses love the notion of fungible employees and a one-size-fits all process to shepard those employees.
Fungible employees makes sense if you are in manufacturing where low skill workers are the norm, but not so much if you are in the knowledge industry, where there are variations in the skills each employee excels at.
IMHO, the dynamics of knowledge work is similar to a team sport like soccer/football but businesses would prefer that the dynamics be similar to manufacturing line workers.
>Being a decent human being is not about getting practical advantages - otherwise sociopaths would be the most helpful people around.
Sociopaths are exploiting the macro-level heuristics we've developed as a species to come to an immediate, reasonably-likely evaluation of safety, tending individuals toward trust, and danger, tending individuals toward defensive action. It takes knowledge and experience to differentiate between a good read based on sincere action and a bad read based on intentionally-deceptive action.
Sociopaths often will appear like the most helpful people around, and sometimes may actually be the most helpful people around, for the duration of the time that they need you to be on their good side. Then they'll dispose of you ruthlessly. If you don't pick up on the signals ahead of time, you don't find out until you've been backstabbed. If the sociopath is skilled, they ensure that you can't "come back from the grave" and cause problems for them.
It's not that corporations don't like cooperation in principle. From a high level, in fact, many corporations are actively and honestly trying to cultivate it. It's just that our systems keep the exploits too open to the relatively-rare maladjusted power player (normal people will sometimes flirt with the same tactics to avenge personal grudges, etc). This forces everyone into a defensive position, if it doesn't force them out of the game entirely.
Once an organ grows to exceed the direct influence of a very small handful of truly good and wise leaders (the more palatable tech word may be "visionaries"), it will inevitably mandate this weird perceptual bastardization.
Tbh that's one of the reasons I always focused on teamwork at my last job, even though I was evaluated only on how fast I closed out my specific tickets. Since everything at the company was dysfunctional, I didn't really care about being employed long-term or getting promotions. Thus the only things relevant to my career beyond doing enough to maintain my salary was to build relationships with my team mates (building out my network) and developing transferrable skills for the future.
So basically the long-term effect of these systems was me to slack on whatever my manager deemed important to get done to whatever extent I could while I did whatever would win me clout points with other developers or let me learn a cool skill.
I also got pretty jacked from sneaking off to the gym etc. Lol it was lit
> This means individual personality and career path variations don’t matter as much. I’m talking things like say, number of broken builds, velocity improvements, production issue counts, etc.
Even on the team level you're still reading tea leaves if you're going into the issue tracking system. The only measure that should matter is execution towards a business goal. A team that is killing it consistently is doing the right things, or is being given the right projects.
In a previous company, there was a feature in the intranet to give kudos to other people. Everyone in the company could see a kudo as a post in the global intranet "chat" sidebar (not really chat, more like a communal facebook wall of sorts that would bump posts to the top any time anyone commented on them)
You could give as many kudos as you wanted, and it had zero impact on performance appraisal/raises, but it sure gave people the warm and fuzzies reading about random mini-stories of someone going above and beyond here and there.
I think it was pretty effective at fostering a culture of collaboration, and it also helped surface issues-masked-as-heroics, such as people working chronically late on some projects. It also gave greater visibility to the existence and importance of the various departments, which one might not have gotten an appreciation of otherwise.
I think I may have seen a system like that, it had a sort of bluish tint to it :-).
Its interesting that throughout my career I've found it has been challenging to communicate group and individual contribution in meaningful ways to people without the whole picture. That is especially true when the activities don't translate directly to sales revenue.
As a result, you get people who leave who were much more important to the organization than the organization understood and the folks left behind are stuck trying to recover and make up the loss.
If there was a manager measure of 'OQ' (organizational quotient) it would measure the ability to recognize and optimize the net productive energy of an organization. Groups without that awareness seem to wither as they pursue one or two 10x employees without regard to how to support their activities.
At SFDC we bought and used internally a tool called Rypple (later work.com) that was precisely that.
As you say, zero impact to people in terms of their performance reviews (which were a topic of discussion in their own, each employee having to write a mini-manifesto for their year ahead) and kinda became a popularity contest where people were giving kudos to their office buddies.
And don't get me started on the people that left heart stickers with messages like "thank you for starting the best company in the world" on Benioff's profile.
Hah, similar thing happened for me. I think the main issue was that departments that relied on creative problem-solving (e.g. business development) ended up being way louder than others that didn't as much (e.g. QA)
Another issue is that tech orgs are extremely political. There was one person who had single-handedly created a viable migration path from the company's aging proprietary framework to .NET MVC5, but no kudos there. Just lots of pushback and skepticism.
Every lead developer position I ever got came from helping people, but that doesn’t always stop the judging on output.
On one project I shaved about 40 minutes to an hour per day of finger twiddling frustration off of everyone’s work. For the team size that paid for itself in under two months. Only ever got appreciation from subordinates and flack from my good for nothing boss.
But what I figured out not long after that was that you understand the work a lot better when you see it from other peoples perspectives, and you only learn about that if you are willing to help and to be candid about the state of the project. People open up. They share maybe-bugs before filing them (especially from the QA team) and then when the inevitable OH SHIT meeting happens you are the only one who has had time to think about a solution ahead of time.
At Google you can ask your coworkers to write feedback that would be included in your promo package. I wonder why the author didn't ask the developers that they helped to write a few sentences explaining how much time they'd saved thanks to the author's help. The hiring committee wouldn't ask for any metric if there were already a few people testifying that "I saved XX hours".
I am manager who has a few levels under me (Total team of about 100 people). I find it difficult to buy into your experience, and hear me out. I am arguing in your favor.
If you are assisting, and this includes providing support for someone else's code, or doing "invisible" tasks (setting up a redis cluster or writing end-to-end tests), this often finds its way into one email group or another. In fact I remember the names of the "helpers" as you put it by the virtue of their emails and to that end I value them, just as much if not more, for keeping us shipshape.
IMHO, you should find these emails, and I am sure they exist already, and use it as your metric.
Scott Fitzgerald wrote “Show me a hero and I’ll write you a tragedy.” I say show me email chains, and I will show a hero.
Of course I am not stating the obvious fact - which is that your manager, if he or she is worth their salt, should totally value the role "helping" plays in making good software.
Yea if a developer helps someone there is no guarantee that there will be an email chain generated as proof. If you truly believe this in all due respect you are out of touch with reality. People still walk up and ask for help. Maybe not in remote teams but in the office all the time.
Supposedly making these type of interactions possible is the whole point of open offices
Yes, people who have been around the block come to the same realization that the author does re: filing bugs. Any type of written communication or record that is not either undeniably necessary or unambiguously favorable must be avoided.
I've had the experience several times where extensive, detailed emails would always meet the same reply: "Thanks. Let's talk about it in the morning." These people are wise to the game, and they rely heavily on the imperfect nature of human memory; memorialization in the written record must be approached very carefully.
The email chain that shows Employee A to be a hero equally shows Employee B as a victim in constant need of assistance, obstructing other work, taking valuable time from others. As such, "Bs" will avoid asking in a recorded form, avoid giving credit to helpers when writing about their work, and so forth. Even if they mean well and earnestly want to credit the many people who've helped, it's too easy for a malevolent actor to turn it against them, so after a few raw experiences, they learn to avoid it -- just like what happens in the article.
The only real answer to these questions is to have an informed, rigorous, and fair-minded chain of command that has experience and cares deeply for the company's long-term well-being, which is, needless to say, quite rare. It is also worth noting that such noble persons are likely hurting themselves by acting this way and making enemies for themselves, as people who truly didn't deserve promotions, etc., whine and search for flimsy excuses.
In my experience people also ask for help through private messenger discussions (slack/skype etc), which of course don't leave any "official" trail and are not supposed to. Of course, a not-so-bright manager could force his/her direct reports to only use "official" channels for work-related communications, but then people would be afraid/ashamed to ask "stupid questions" in public/through an official channel and then disaster happens (it's also my experience that lots of serious bugs have been caused by people afraid/ashamed to "just ask").
That reads an awful lot like you're optimizing for people good at leaving email trails, i.e. at office politics.
There's only one way to actually measure someone's output, and it's working close to the team and witnessing their actual productivity. Getting involved. 1-on-1s.
Of course the farther removed from this process the more politics come into play.
^ this. Weekly or bi-weekly 1:1s + read the check-ins. Check-ins don't lie. If you are not reading check-ins as an engineering manager, then you are going to be heavily affected by politics.
I've spent a few weeks working on an extremely high priority business issue and the only artifacts are a few issue tracker stories, my team's slack channel, and a "huge thanks to @...." on a public chat. If you're at a high enough level that you can't value someone's contribution from those artifacts, then you need to be trusting those under you and asking about people... you'll be missing why high performers are leaving.
The irony is that Google actually has a separate eng job ladder (Software Engineer, Tools and Infrastructure) which is designed to reward that kind of work specifically. If he had ladder-transferred (a fairly easy process) and gone up before a SETI committee with that same data (peer feedback on stuff like saved development time carries weight that metrics may miss), it probably would have turned out very differently.
If you know ahead of time that you are doing "the dirty work", which usually you don't, do you know how long you are going to do it for, and if it makes sense to apply for a transfer?
Would your manager approve of your ladder transfer if he/she needs to back-fill your position when a good engineer is hard to find or keep to begin with? (Actually, there are scenarios where a manager wants to keep underperformers from transferring out, too)
Lastly, did you really want to be a Tools and Infrastructure engineer to begin with? It started with just being scrappy in order to get the project to a place where you can contribute shiny new features, which everyone is waiting for you to do, yourself included. If you transfer ladders, what are the chances you will be able transfer back in any reasonable amount of time to get credit for doing a good job on what you wanted to work on?
Michael's work situation seems prevalent through a Google-like org, and a ladder transfer is unlikely a realistic solution for that.
At my job, we use HeyTaco, a Slack bot. Every day you can give up to five tacos out by @mentioning somebody with a :taco: emoticon. If you answer a question or help somebody out, they will probably give you some tacos. The app keeps track of who gets the most tacos with leaderboards and the like. This has instilled a culture of helping others out on the team, by gamifying it and making it visible.
We use something like this in my org, and Google has "kudos", but I've found that the problem is their usage is highly variable even within a medium-sized or (we're roughly 1,000.) On some teams, the average employee will have hundreds of these kudos, whereas on others, the average might be 1 or 2.
This was highlighted to me recently when someone in Slack noted that one person had ~50X the kudos of everyone else and asked how that happens... and they said "just go hang around in team X's channel, they'll give you a 'taco' any time your answer a query."
Given Google's promo system (where a committee looks at candidates from very different teams) this then means they'd have to try to normalize the data for it to be useful.
At least at Google I don't think that was their original intent - the idea was to provide a way for employees to express their gratitude away from the perf context.
I don't think they should be used for perf as they are not intended to be objective feedback.
Some roles (ops, sales) are just naturally a lot more inclined to use these kinds of systems. I generalize like that after having watched similar systems at 3 companies; you hardly ever see an engineer give a "taco" for a code review, but someone in another team might give one simply for someone knowing which room a meeting is happening in. It's hard to normalize the culture. I think peer bonuses (which usually require manager approval on one or both sides) tend to be a bit more normalized across teams.
That's what's dysfunctional about a lot of companies/our society. You persuade others to do/[help with] your work you are showing leadership and get promoted.
Sorry, you got duped into having sympathy and showing teamwork.
The better you are at convincing others we're a team and hard work is it's own reward when it's really just dog eat dog the higher up the food chain you go.
In our team, we have our custom implementation of a web app like https://bonus.ly . The idea is, every member would be given a certain points(we call it jems) at the start of the month. He/she can give those points to other members who help them with the reason specified and it will be posted on a slack channel through slackbot.
Using this, everyone can know what other people is working and who is helping who like that. For every month at a team meeting, people who got more points will be appreciated in front of the whole team. The total points is calculated using a formula such that a member who just start giving/receiving more points per jem than other so that new comers will be appreciated more.
It works really well and at performance review, managers will consider this stats. Also the performance review contains two peers (chosen by us), and all kind of managers who we are reporting.
We estimate and log our time at my company Matters. Teams estimate a large budget if they don't know how to do it. When you assist another team you generally deal with the task in way less time than planned, therefore performing greatly.
It helped us discovering the importance of assists, so now we have support teams dedicated in assisting project teams.
Project-driven environment + no estimate or log = politics
At my company we use something called bonusly. You get 30 bonusly per month to give out to your teammates, and each bonusly is worth $1. You can buy basically anything with the bonusly you receive (amazon gift cards = anything). On top of being able to buy stuff, it also keeps metrics on which teammates help the most. The only downside is that you must give out all your bonusly at the end of the month (if not it's wasted money), so sometimes you start giving it out without really taking into consideration which teammate has helped you the most.
In my experience this often just becomes a popularity contest/give to whoever you eat lunch with.
I can have my head down all week neck deep in code but because of this I'm not interacting with people around me as much and basically get punished for focusing on work.
Yes. You can't change people. Your actual job description is worth 20% or less of your workplace success; as long as you're doing enough to stave off disaster, you are pretty much fine.
There is a simple metric for workplace success: if most people are happier while you're around (including and especially anyone above you in the chain of command), you're winning; if not, you're not. This can probably be summarized as "popularity". A few things can overcome this, so you may want to be careful not to go 100% focused on it, but you will definitely prosper if you make it the focus of 95% of your work.
Developers who like to debate and don't take it personally when a colleague disagrees are at a distinct disadvantage here, despite simultaneously frequently being some of the best qualified coders, because they don't understand that they're rare, and that many other people take any form of non-ambiguous criticism of their ideas or work as a major offense: taking bread out of their kids' mouths, etc.
I have a theory that the more a person cares for niceties, appearances, and superficialities, the more it shows that they're incapable of operating at a level any deeper than that (that is, capacity and the presence of superficial tokens generally understood to mark capacity are inversely correlated; e.g. an expensive watch does not necessarily indicate power, wealth, or importance; compare Dunning-Kruger Effect). People who are capable of doing useful work seem to find these superficialities asinine and have difficulty hiding it, whereas people who aren't capable of useful work try as hard as possible to keep the focus on the surface level, because that's their only hope.
Since the surface level is immediately apparent and the importance of the subsurface may only become apparent after some experience, imposters can, and frequently do, find great success.
I 100% agree that you have to put a lot of focus on how you interact with your coworkers. And further--most people can't take criticism well. At the end of the day business is all about people.
But I would say a lot of managers are trying to get their coders to be more productive with coding. When they introduce something like "assist points" which is really just a popularity game, it doesn't incentivize coding as they think it does. It incentivizes more socializing. Which frequently also means less productive with coding. Don't get me wrong I do need to work with my coworkers, but a lot of the time I need to be working alone, or just send a quick email. And I'll get punished for doing that even if that's the most productive thing to do.
On your theory--I think it depends. Sometimes it's "I'm insecure and need to compensate in some way, like expensive watches" but sometimes it's "I take pride in everything I do, including my looks." Also while in some ways I'd probably be happy to wear PJs every day, I understand that the easiest way to get more power in social situations and business is just to dress nicely.
> I have a theory that the more a person cares for niceties, appearances, and superficialities, the more it shows that they're incapable of operating at a level any deeper than that
The third category, of whom I’ve known several, is “capacity/competence, with an attention to the superficialities because they know how much those matter.”
That is, I haven’t found it to be an inverse relationship. I find it to be a 2x2, and if you happen to be someone with less capability and more ability to BS, then yes, you lean on your strength to obfuscate your weakness. But you can also be strong in both, or weak in both.
I also think there are a lot of jobs where deep mastery doesn’t generally exist (eg, mopping), and so those people just have no idea what deep competency discussions are like. It’s like flatland, only the third dimension is expertise and complexity.
You're right that there are definitely people who are competent and have accepted the necessity of the surface-level accoutrements. The problem is that their direct knowledge of the worthlessness of those tokens means they will never wear them as sincerely and earnestly as an incompetent person, who believes that the surface-level tokens are competence itself.
A little bit of pushback on the more technical or capable people and they will start dropping the facade, whereas the less capable will just hold it up higher, because it's the only thing they know. They have no conscience accusing them of misrepresentation, no sense of guilt or concern that they may be drifting afield from an acceptable core. They only learn that they need to wear the tokens more convincingly next time.
So yes, you can have people who wear the tokens for expedience and out of a recognition that its important to fulfill the heuristics, but they're no competition for those blissfully unaware that there is anything else involved.
I really don't understand your point. It's not as though wearing clothing "good enough" to get "social brownie points" is difficult in most places. An ironed, well tailored outfit and simple color coordination is usually good enough to signal to people that you're credible. Most folks will not need to get into the finer points of fashion like the very latest trends and brand names. So I don't see where sincerity plays in when the bar to just reach the level of "looks like they have their things together" is pretty low.
In my experience (I sort of enjoy fashion) most people cannot tell the minute differences between "tailored outfit & basic color coordination" and "total fashionista who keeps up with everything." I can walk around with a fake Coach bag, get compliments all day, most people just do not know.
There's exceptions of course, certain offices/fields of work. But it's not common.
At Google there is a peer bonus benefit. You can peer bonus someone for doing something awesome. Each peer bonus is $150 before tax. You can list your peer bonuses on your promo packet. You can give as many of these out as you want ... within reason. Sometimes at Google they do something called a peer bomb, where many people will peer bonus a single person.
I can't imagine working at a company with this kind of system. Sounds a little too much like MeowMeowBeenz (http://community-sitcom.wikia.com/wiki/MeowMeowBeenz) from the TV show Community. ("MeowMeowBeenz™ takes everything subjective and unspoken about human interaction and reduces it to explicit, objective numbers.")
How do you feel bonusly avoids this potentially awkward gamification/popularity contest? I feel like most people I know would just give 30 bonusly to a random person on the last day of the month, but maybe your colleagues are a little more disciplined :)
Yep - See some of the work my last team published here while we were rebuilding deployment tooling: https://blog.coinbase.com/scaling-developer-productivity-d23.... For hard to measure metrics, softer measurements like repeated team surveys work fine too.
I'd be cautious of directly measuring "assistance" outside of peer input in perf to avoid an unhealthy incentive. The most helpful people I've worked with in the past tend to grow through strong peer reviews and having the most opportunities to join new projects.
The OP here really misses the point of demonstrating impact. Doing the right thing ethically then for the business is a strategy that is, well, rarely the wrong thing. Optimizing just for getting promoted might is a greedy strategy that might get you promoted once, but good luck finding peers that want to work with you again.
It would be nice to have some sort of metric for this. I haven't experienced any.
As a former manager, the closest I came was making a point to observe who was helping who within the team. And following up by asking people during their 1 on 1s if they felt anyone was being especially helpful to them, explained things well or not, etc.
People who helped their teammates very much got a boost in their prestige in my view, and that was good for additional job security or additional pay. If they struggled with a couple tasks, but had helped some teammates when they DID understand what to do, that very much increased my willingness to help them or give them extra time to figure things out.
Where I work we are big on peer review. So in theory helping others should have value. It's still not perfect. For example, a junior may not be able to asses the value of your help, or anything else for that matter. People will still tend to rank their friends higher. While your boss doesn't rank you whether he promotes your work to others has a big influence on how they view you.
In a previous company we had a better mix of peer reviews and input from technical leads (that were not your bosses). This was by far the best system I've seen to date for a company of that size.
In small companies there is typically good visibility into what everyone is doing. If you have good management then things tend to work very well. If you have bad management it can be worse than large companies.
One thing I've come to learn is that this sort of stuff, call it culture, has to be a priority for the company's leadership, they have to keep inspecting it and fixing it. They need to talk to people and understand their concerns. Otherwise it always drifts the wrong way. The Google system from what I read here sounds terrible.
EDIT: Having people try to optimize to play the system, just like the article describes, is inevitable so you must ensure the incentives are aligned. Make sure you find ways to look at the stuff that really matters and make sure you're not antagonizing your employees with your review system.
> For several jobs in a row, I've felt that helping others on a team is undervalued and under-recorded. I've been planning to implement the "assist" metric, similar to basketball, on my own team for a while.
The metric might be a good idea, but I think the real problem goes deeper than that.
Take a step back and ask yourself a more fundamental question: should everyone who does good work be promoted?
By definition, promotions are limited and each level is more difficult to attain than the last one. Obviously, you can't promote everyone. Even worse, not everyone will be happier when promoted. So why does literally everyone want to be promoted? Why do they work so hard to get it?
It's because there are no other ways of rewarding good work. And by "rewarding", I don't only mean "giving people recognition" or "giving people fun work". Those are important and should not be overlooked. Nevertheless, if you don't also give people money, sooner or later they'll fall behind their needs or just the rising costs of everything around them. Then they'll have to choose whether they need money more than intrinsic rewards. And then we're back to the original dilemma: do I play the promotion game or do I just bugger off somewhere where I can get what I need?
TL;DR: Retention shouldn't be only about promotion. Give people real rewards and incentives for doing a good job at their level.
That's not really the case if the promotion ladder is a skills/experience based ladder and not a responsibility ladder. If being a "senior engineer" just means you have reached a certain level of proficiency, then there's no reason everyone can't be promoted to that level.
I academia, if you're a PhD student that writes the paper, you're the "first author", but anyone who helps you do research or process data or other contributions is a "second author".
I think that's a better system, because one person may have 2 first author publications and 3 second author publications. Another person may have just 1 first author publication but 12 second author publications. So you can weigh and evaluate each person based on their assistance and contributions to others.
This system is actually designed to disincentivize assistance because only first author publications will net you scientific credits when it comes to jobs or funding.
At my workplace (Mixpanel), we use youearnedit.com. Everyone gets 2000 points per quarter and you distribute them out to whoever helped make you life easier or reward others for their effort.
It has slack integration so sending /yei @someone is super easy. I probably do it multiple times a day.
I have to say, after it was introduced, there definitely was a slight shift in culture.
I strongly believe your 10x programmers are the ones that help the whole team become 10x by building great tools and showing best practices that makes everyone move faster.
So, would you mind helping me see if I understand quadratic voting correctly?
Let's say there are 4 voters: A, B, C, D, voting on policies 1-10. Everyone gets a certain number of votes (say one per policy, so 10 total).
Voter A only really cares about policy 5 & 7. So they want to buy votes for these 2. They can give up their votes in, say, policies 1-4 to get 2 more votes for policy 5, and can also give up votes 6, 8-10 to get 2 more votes for policy 7. So they put 8 votes into the "common pool" in order to ensure that, in addition to their vote for 5 & 7, each policy also gets 2 extra votes?
That's basically it. So say you had 10 people on your team, each person gets 9 votes. Say someone wanted to give only one person all their votes, the recipient would accrue 3 votes (sqrt(9)=3). Or they could give 2 people 4 votes each which would accrue as 2 votes (sqrt(4)=2) each for those two people and then you'd have one left for someone else. Then you tally up all the accrued votes (not votes spent) for all the people and then you'd be able to rank people according to how helpful they are (but this ranking wouldn't be transferable outside your team; you can't use it to compare them with the helpfulness of people on other teams).
FWIW, the paper describes it as a vote currency that can only be used for the vote, as opposed to cash you could spend on anything. The voice credits have to be distributed to voters.
'individuals pay for as many votes as they wish using a number of "voice credits" quadratic in the votes they buy.'
If you are referring to the same QV paper I read, it spends most of the paper arguing for money but then at the end suggest a virtual currency at the end for those who can't accept their argument for allowing buying votes with money.
We use bonusly. As a proof-of-concept, I imagine it'd tackle some of your requirements. Bonusly points are tied to real world rewards, so there's incentive to use it.
People who had a lot of "assists" at our company regularly brought in 1000+ points a month, which then translated in $100+ in amazon gift cards or steam points.
I don't have much experience with this, but I find it a very important topic. My opinion is that there are so many different aspects to teamwork and social cognitive biases are so huge, it's an intractable problem to do on an individual level.
The only thing we know matters is the team does well or not. So I suggest trying different methods of building/running a team, and let them go. See which teams succeed and try to copy that success or give them more power.
To give some flavor to my idea - maybe the level of helpfulness you should ideally give is very low. Perhaps it enables weak members to suck up lots of time. We just don't have good data in this area.
The company I work for has a kudos system. You can give any gift, but in practice everyone gives $25 amazon gift cards since they are almost cash. Employees are encouraged to give at least one a month to someone who deserves it. I haven't been through a performance review yet, but I wouldn't be surprised if they are taken into consideration.
From my experience it works amazingly well. It's silly, but if you are ever having self-doubt, a gift card from a coworker really helps.
The company that I work for employs a system of peer review - where at the end of the year your teammates write reviews for you and your appraisal is based on that. It seems to work pretty well our case and definitely aligns the incentives of the employees with having an impact on the people around them.
> For several jobs in a row, I've felt that helping others on a team is undervalued and under-recorded.
I completely agree with you on this. Also, I like the idea of formalizing with "assist" metric, so that people are sure that their work won't go unrecognized.
Google actually kind of has a partial solution for this. Your performance review is not just based on manager's assessment, but you also need to gather reviews from your peers.
However, you need to strike a balance between helping out peers and having impactful projects.
I work at Microsoft, and I know around promo time in addition to people on your team, managers ask for feedback from people not on your team that you’ve interacted with as well so that can be used in conjunction with whatever projects work you’ve completed.
For several jobs in a row, I've felt that helping others on a team is undervalued and under-recorded. I've been planning to implement the "assist" metric, similar to basketball, on my own team for a while.
The idea would be something along the lines of everyone gets a set of assist points they must distribute to people who help them the most.
While I don't love the idea of gamifying it, all the places I've ever worked have had too strong of a reverse incentive towards individual achievement, especially when it comes to promotion, and not nearly enough support for teamwork in specific and measurable ways.
Anybody have experience with teamwork metrics like this, or others that worked or didn't work?