When an almost identical interview questionnaire was posted a few months ago, I was hoping it was some sort of anomaly, or that Canonical would learn from the feedback and alter it. To see something so similar (for a different role) two months later makes it seem far more likely that they have one of the worst hiring pipeline processes I have ever seen.
I’m good at these sorts of written/verbal assessments, and even my eyes start to glaze over just reading the list of questions, let alone thinking about how to answer them. A quality response to this sort of questionnaire would take at least an hour and that’s before you even get into the actual interview process or hear a salary range.
And as dumb as questions about high school and college are for professional job interviews (unless you are hiring straight from undergrad, I just don’t see how this is relevant. And even for college recruits, I have a lot of problems with this approach.), it would be one thing if I thought the recruiter or hiring manager was actually going to read each response from each candidate. But I cannot imagine they will.
For a company that has openly expressed its struggle to hire, it’s stunning to see them double-down on such a terrible recruitment process.
The people who you want for this job probably have better things to do than waste an hour or more answering an overly-long questionnaire before they even find out the salary range.
I applied for positions there late last year. I appreciated that they were upfront about the steps of their process. It made it a relatively quick decision to not continue.
Honestly, it would take me a few hours too. I just wanted to undercount the projected time, in the event there were “I could do this in X minutes” responses. But yes, it would take me a few hours to do this too.
But then, how many job applications do you do? Maybe Canonical wants to filter out people that apply to hundreds of firms, and only consider those for whom Canonical is on the short list - and a list short enough that a few hours spent on those questions is not a big deal.
Yes, that's one significant benefit of the written interview.
We had 110,000 applications to Canonical last year, which is a lot to process for a company of 800. We offer a very high-quality remote-first employment experience, with international travel twice a year for up to two weeks so teams can plan, bond and play. That's attractive to a lot of people.
Candidates these days apply for between 5 and 75 roles in a batch, and I'm pretty focused on selecting for people who genuinely care about open source and the role we can play in making it a level playing field with high quality developer and production experiences.
About 30% of the people who are moved to written interview submit them, so it does serve to separate out those who understand our mission and are excited about it, from those who just clicked on our link along with 50 others.
The written interview also helps reduce culture and gender bias in our hiring.
Given that we hire globally we know that the people reading the candidate resume may not have context on the resume norms and expectations in the culture and place that the candidate comes from. We ask them to move any plausible candidate to written interview, and then we generally have the written interview reviewed twice by two separate people, in an anonymized fashion so the reviewer can only assess what's there, with no bias from resume.
Food for thought: I care about open source and I'd say I'm the kind of engineer canonical would benefit from hiring (not to flatter myself too much) but at this point in my career I no longer look for work. People try to poach me and usually offer to skip interview requirements. Good engineers are never "looking" for work since the work finds them. Increasing the bar of application just means filtering out top candidates.
I'm not even the best SWE in the world, just in a very special niche (which Ubuntu IoT serves poorly). Engineers who are better than me have it way easier than I do. It's exponentially easier to job hop at every skill percentile.
> We ask them to move any plausible candidate to written interview, and then we generally have the written interview reviewed twice by two separate people, in an anonymized fashion so the reviewer can only assess what's there, with no bias from resume.
The SATs essay portion is known for giving horribly biased signals. My assumption is this selects for essay writers, not technical design excellence. There's a lot of great stuff I've seen come out of canonical (MaaS, etc) and a lot of horribly designed legacy tech that no one enjoys using (juju). It seems like getting someone in the pipeline that can sort out a clear modern infrastructure vision would really help canonical and the broader ecosystem at large but I think it won't be happening largely due to hiring practices.
I once interviewed at Northern Trust for a programmer role. Interview was 2 hours long divided into 4 - 30 minute segments each with different interviewers.
None of the interviewers asked anything about tech. Absolutely nothing. All of the questions were in the style of “if you were a (blank), what type of (blank) would you be?”. And each interviewer asked the same questions in the same order working off the same interview sheet. By the third 30min interview I was concerned that I was stuck in some sort of time paradox. By the fourth 30min interview I just wanted to get out of there.
They ended up offering me the position but I declined. Don’t know what happened but they felt very disorganized to me.
I worked with Northern Trust tangentially for a few years (I left my role in early 2010s) in a quasi consulting role (it was actually a joint venture where we did the software, they ran it sort of thing). NT was hands down the most painfully bureaucratic company I've had the displeasure of working with. Something simple like a DNS change for a server was a 6-8 week turnaround. On top of that, they weren't exactly the best at following instructions, either, so it was often multiples of these multi month cycles to get even the smallest thing done.
Very frustrating. You likely dodged a huge bullet by declining the offer.
I noticed this last year when I applied for a software engineering job. For that position, the process is:
- Initial screening (done)
- Personal essay (this stage)
- Standardized aptitude and personality tests
- Meet and greet with an engineer
- General interview(s)
- Recruitment screen
- Technical assessment
- Role-specific interview(s)
- Hiring manager interview
- Senior leadership interview
I highly doubt the initial screening is done by a human. So they want the essays and personality tests done before you speak to a human at the company, and they are probably not even on the team you're applying for. If you're unlucky, every human involved in the process will be on a different team until you get to the hiring manager interview, which is at the very end.
Not only this but on Glassdoor I found dozens of people saying that they ghost you before the hiring manager interview so you could spend tens of hours interviewing and never speak to anyone on the team.
Wait, they have SEVEN distinct interviews? Meet/greet, general, tech assessment, recruiter, role, manager, senior leader? Holy shit. Wow. That’s insane. What a waste of everybody’s time.
At my current job, I did about the same number of interviews. Not counting an initial 15 minute screening call, I had a 30 minute phone call with two of the engineers on the team, a technical panel interview, and 3 exec interviews. So that's six interviews. I would be working with all of these people directly and I wanted to avoid "broken stair" coworkers I have had in the past. While I found this valuable, I didn't feel the same way about the Canonical process and refused to proceed. Each stage of an interview process needs to provide value to both sides. As a candidate, a personality test or an essay does not help me at all except to raise a red flag. And speaking to many people who I might never speak to again is useless to me.
If you interviewed 3 executives, I’m assuming your senior leadership yourself. Or, at an actual start-up where the execs are still doing IC tasks. I can see the need for a higher bar in either case.
But for an average developer at an average company? That just sounds insane to me.
So many tech companies are bloating the interview process with seven or more interviews. For my current role I did several interviews and the record during my most recent cycle was ten IIRC. The particularly annoying part is that most of the time the rounds are almost complete duplicates of each other just with a different person.
edit: I just checked my calendar and I had nine meetings/interviews with Cruise before they rejected me.
FWIW, our normal process is 4 interviews… phone screen with the recruiter, 1 interview with hiring manager, a live tech screen (could be another manager, could be a senior IC), and a quick chat with director or VP. We try to get a candidate through to offer/decline within a week. I really can’t see the additional value of additional rounds of interviews.
These are almost always extremely discriminatory when it comes to neurodiverse populations and I'm saddened that Canonical uses the equivalent of phrenology in its hiring process.
I mean, there are plenty of bad tests out there (Meyers Briggs etc.), but good aptitude and personality tests might well be the best (objective, reliable, valid...) tests of aptitude and personality there are, no?
So, what training do you suppose anyone has in interpreting these tests? No one in the hiring loop is a trained psychologist, why would I want you to try to interpret the results of the MMPI or whatever? First of all, and I know the law says differently, but this is medical information. I'm not sharing the MRIs of my back (which you also can't interpret), and I'm not sharing mental health information either.
But, suppose you did have a degree and specialized training in these tests. Do you think such a person can reasonably predict job performance and fit? It's reading tea leaves. The two biggest predictors of on the job performance is first, previous on the job performance and then general intelligence. After that it is all noise. You can go to an office and talk to a shrink for weeks and they still won't really grasp how you think or be able to make predictions about your future. If you dispute that, show me the research.
It's phrenology and biases all wrapped under the blanket of "objectivity".
Example of bias: Test X output Y shows correlation with OTJ performance (really, peer reviewed and all that). Cool! Correlation, not equivalence. Which means at least some portion of the population will do poorly on the test, but great at the job, and vice versa. What are the chances that neurodiverse people are in that category, or women, or blacks in a white culture, or.... And, regardless of that, suppose you are one of the people, in no particular protected or disadvantaged group, that does poorly. Should you never get hired, or get offered lower pay/responsibilities, because you do bad on this standardized test that gets run by so many companies? Is this what we want from society? Even pretending that the person seeing the results of the test in fact has the training and ability to interpret the results. There are cultures where they tend to show up as lying on these tests, but it is a cultural artifact. We know this. And a real medical professional consider the tests as indicative or suggestive - a point from which to start. Are you actually paranoid, or did you have a bad test day, or does your culture value something that pings poorly on the test? You don't know without a lot more investigation. HR is merely going to say "pass". Well, they'll ghost you, but anyway.
I'm sure someone can come up with some good examples of tests (well, maybe not sure, but I'll credit it is possible), but these tests all seem to be someone's hobby horse, promoted because it makes things seem objective and fair (so we don't get sued!), or because it is woo.
I've overstated my case a bit, but cmon. No one interviewing is a doctor, they shouldn't be using tests designed for medical relevance, and we should strive for inclusiveness. Can you code and not insult your interviewer while being questioned? Sounds like a hire to me. Oh, the MMPI says you are on the introverted side, or have issues with depression? Go fuck off, interviewee? I don't want to live in that world.
Our talent assessment team include several people with Masters and PhD level qualifications in psychometric and behavioural assessment, so that we can provide a rigorous and professional perspective in this regard. Every candidate who gets past early stage interviews meets a dedicated HR professional.
Thanks for replying directly, Mark. Do candidates receive transparent feedback and information regarding exactly how the standardised testing/behavioural assessment has been interpreted, how it has effected their candidacy, and relevant information regarding the background of those interpreting the tests? Is there any means to contest this interpretation and/or request a secondary opinion by a qualified professional chosen by an independent third party?
To be clear - every application resume is screened by one of my colleagues or me, and every written interview is read by at least one of us, usually two.
I personally screened 40,000 resumes last year, and hired 100+ people into the company. It has been very exciting to see how we are lifting our confidence and aspirations as a result.
Do you think it's normal for the CEO of a 800+ employees company to be reviewing resumes? It sounds like you don't trust the people you hired to do that job, and decided to do the job yourself...
Also, 40,000 resumes screened means an average of more than 100 resumes per day, every day of the year. Again, do you think it's normal for the CEO of a big company to be spending their time doing this?
And finally, don't you think the whole process is biased by the fact that you review the resumes? Let's say you personally approve a resume, don't you think the next people in the company down the line will be in a position of "Well, if the CEO is pushing for this candidate, I better not reject this candidate!". In which case, no matter the numbers of steps to filter out, it's a failure.
No, it's not normal. But your conclusion that I did this because I don't trust colleagues is mistaken. I did it to figure out how we should shape the hiring process, and so I could relate to the workload and problems that hiring leads face daily.
When you are going to ask your colleagues to tackle a hard problem, and do it in a different way than they did before, it helps if you have actually done it yourself. If that is going to be lots of work, and if it's going to make them vulnerable to trolling on social media, even more so ;)
We started to plan growth a few years back, but seemed to be struggling to get people in. Hiring managers would say "there were no good candidates", which didn't seem right to me. Back then, every hiring manager did things independently, as they saw best. We had very little standard process other than HR onboarding.
When an opportunity came up for me to hire, I adopted a new process, then expanded that to cover a number of roles. Once the process was relatively settled, we picked a small number of senior people, and they ran this process across all roles, with a weekly meeting for us to figure out how to evolve it further.
As you can imagine, this change caused all sorts of angst. Reddit is not the only group that has thoughts :) Many of the same assumptions and opinions expressed here were expressed internally, some resigned. Systemic change in an organisation is painful, this one was particularly hard.
Now, however, most see the benefits.
Our flow of applicants has increased 2-3x. We now have 10-20,000 applicants per month, much more widely representing the world's talent. We are making more appointments than ever in our history. Our weekly meeting is now mostly a celebration of new hires - the whole hiring lead team looks at every new hire's resume and talks over the process of hiring that candidate, then discusses improvements to the process. We are able to analyse the data in our hiring process and look for any indications of bias at each stage, then engage and address it if we have concerns.
You ask if the "CEO is pushing for a candidate" biases the process. You misunderstand my role as hiring lead. I am not pushing for a candidate, I am simply running a few roles as a hiring lead. My job as a hiring lead is to do the initial lightweight screen, then let the process do the work. I don't meet candidates in my roles until the very end of the process, and then only sometimes. I'm running the process to hire for other managers, not for myself. Those managers meet the candidates in late-stage interviews. My job as hiring lead is to make sure we have a good bench of candidates before we make an offer.
Some of the roles I run, which attract the bulk of the applicants, are open-ended roles, for "engineering manager" or "engineering lead". When candidates get to me in those roles, if I think they are great I then send them to a manager who is looking for someone like that. So it has been my great pleasure to give tens of amazingly great candidates to my colleagues for consideration for roles they hadn't even started to hire for. Think how nice that feels, for me and for that manager :)
To circle back to the beginning - beware the trap of "normal". You will never achieve something exceptional if you limit yourself to normal.
That of course will vary based on role, experience and location. We have hired people from 50+ countries which is remarkable for a company of 800. We tend not to hire in the most expensive locations because we get fantastic talent who live in less polluted, congested and occasionally self-important environments.
We do pay competitively; we have hired people out of FAANGs, and we constantly benchmark pay against our market data around the world. True remote work is worth an extra 20%, and quality of colleagues and focus is priceless. We also rigorously assess pay raises for gender equity.
* Who was the person coming up with the interview questions?
That would be me, mostly. I generally would have reviewed them, though occasionally some slip by.
Some of the role-specific questions would be from the hiring lead on the role, who is usually a senior person in that part of the business who has been through an onboarding process run by our global head of HR and me, and who handles applications for multiple jobs. These hiring leads are less susceptible to bias, and more experienced, than first-level managers, and it frees the first-level managers to focus on their teams and get good at being managers.
* What is that person's psychological profile?
If you are asking about my psychometric profile, I scored 94 on the test we use. If you really mean psychological profile, I don't know. I've been called all sorts of things over the years, but not by professionals ;)
* What is that person's biggest achievement since high school?
That would depend on what one values.
As a student I helped a small newspaper in Cape Town be one of the first in the world to be online. Started a company, Thawte, that helped lots of businesses outside the US get certificates for secure trading. Sold that well, and then trained to join a flight crew on Soyuz to the ISS. Ran several experiments in space, including the first stem cell experiments, which helped shape stem cell therapy research here on earth. Started a foundation which has funded tens of social change leaders working in health, digital, civil society and environmental areas. Started Ubuntu to democratise access to open source, made some mistakes but stuck with it to help Canonical survive and now thrive. Learned not to yell at people, learned to hire, hired hundreds of people. I also have helped start some botanical gardens because I like them, and am active in helping a small country chart a course for themselves that has made a measurable difference to the lives of that population. What, I wonder, do you value?
* Who are the people working in the team?
They are generally outstanding technically and socially, from all over the world, with a strong sense of shared mission to help open source be easier and cheaper to consume, personally and professionally, for companies and individuals.
* What are those people's psychological profile?
They vary greatly, it's useful to shape teams that complement one another.
* What're their education levels?
Generally but not exclusively they have undergraduate and graduate degrees.
* What're their work experience?
That varies widely, we hire both new graduates and people who could happily retire but like what we do and how we do it.
* What're their biggest achievement since high school?
If you can get a place in the company, you could ask them yourself.
> If you are asking about my psychometric profile, I scored 94 on the test we use.
So you're confirming that these "psychometric profiles" can be summarized as a number. Which means it's very easy to automatically apply a filter (reject any candidate with a score lower than 80), and even if there is no automated filter in place, you understand that people with access to this number will immediately be biased because of it? ("I have to review this candidate, but he fared 70 at the psychometric test, where every other candidate fared 90 or more...", and then obviously, during the interview, and after, the reviewer will be biased by that number.)
Yes, data shapes decisions. Should that not be the case?
A hiring lead is expected to find the best candidates, not just the ones with a higher number on a simplistic test. Your point is correct that a hiring lead with bad judgement or low engagement might get a bad result, but what hiring process cannot be undermined in that way?
When you don't like something, it's common to mischaracterize it with anecdotes of stupidity, then say it must be stupid. The argument is basically "a biased and obnoxious hiring manager could misuse that data to make biased and stupid hiring decisions". Yeeeeeees. Obviously.
The psychometric test we use is a series of numbers covering different aspects; it's simplistic, but not as simplistic as I reflected with a single number.
Hiring leads often progress a candidate that didn't ace that test because they see evidence of excellence in other data. And I often reject offers to candidates who aced that test because I think the result doesn't gel with reality, although I will usually interview a candidate if I have concerns about an offer.
To be honest, that seems fair given that they are pretty community focused. You might view it as an unpaid internship, but my own contributions to open source and the community have always gotten me jobs. I view it as extracurricular activity... similar to what they pushed us to do in high school for getting into college.
Sure. But I have a full-time job and a family. Combined with the essay's focus on the candidate's high school career, it's a pretty strong signal I'm too old to work there. I'm sure as hell not going to spend a year of my precious spare time on the gamble it'll get me a job where they expect me to continue working off-hours.
I'm "old" (~50) and I don't take it as age related. It just sounds like you are at a point in your life where you have different priorities and should look for jobs that match those.
I have two complaints there. One is the free internship thing. The second is the focus on one's high school career in the essay section. Two yellow flags makes red, IMO.
The tragic irony there though is that Canonical commits very little engineering effort to contributing to upstream projects that they don't control. They'll fix a few bugs that are affecting customers, but there's no generalised effort to grow the garden they live in.
Which is a huge difference from Red Hat, which contributes heavily and is very good at leveraging the resulting control of important projects to their advantage. It's why they keep out-maneuvering Canonical, as far as I can tell.
Na it's not about the size, it is that every product canonical build's is closed-source and has "mostly" shitty quality...quite the opposite to redhat.
Snap-Store? Ubuntu-One? You always try to create a wallet-garden with proprietary software as the main-repo/server with no additional gains against other opensource OR closedsource competitors.
Then you guy's see that it's not adopted as you wished, then you stop supporting it. It happened with Ubuntu-One and it will happen with Snaps.
Both times you made the server closed, but the clients open, just stop with that BS, and if you really have a vision at least stick with it (mir, ubuntu touch, unity...) you try to create something, be the leader of it, but at the same time expect that others work on it. If not instantly successful you drop it.
Do you really think someone wants to invest time/money in software that is owned and steered by such a "leader"?
You know when you presented Ubuntu-touch, i was thinking:
Wow that's the end for linux-laptops in like 5 years but no, you dropped it...i was completely disappointed, that was your chance to redefine client/terminal-computing.
I can see how you would draw those conclusions, because yes, we have made mistakes, and we do have some products that are proprietary, and we have made some shitty software.
At the scale of hundreds of engineers working on tens of codebases, who hasn't?
That's my dispute with your position - it's flatly generalising and characterising ALL our work as if it was our least successful work. By all means, call us out on our mistakes. But if you think everything we have ever done was hopeless, then you must have some theory about why so much of what we do has in fact been widely adopted. Surely you can see how fragile your own arguments must be if they are at odds with some clear data.
I like challenges and criticism; it helps us get better. I know we have made mistakes, that helps us get better too. I respect people who have differences of opinion on what needs doing, or how it needs to be done. Why not take the opportunity to become a counterpart in a rich discussion, rather than a one-dimensional anti-everything-Canonical mouthpiece?
We killed Unity for three reasons. First, I ran out of money to fund it. That was rough. Second, we built some very shoddy pieces in that codebase (though we also had some teams that did amazing work). That was sad and taught me a hard lesson. Third, we were the fourth horse in a two horse race, and the telco and ISV ecosystems were clearly not going to engage.
I agree with you - the vision of Unity was and still is beautiful. It's probably right - convergence between phones and tablets and laptops is the future for laptops. I think it's fantastic that a small but feisty community continues to work with that code. I was therefor, like you, completely disappointed to fail.
But why hate on me, and Canonical, when we literally put everything we had on the line for that vision, but screwed up? Why be an ass to the sorts of people who are willing to put every ounce of strength they have into something that you really wanted to see succeed? We failed because we made mistakes, yes, but for heavens sake recognise an ally when you see one, even if they make mistakes.
First and foremost, thank you very much for taking the time to answer my comment.
>At the scale of hundreds of engineers working on tens of codebases, who hasn't?
It's not about the mistakes, but about the way you create new stuff. It always feels that canonical try's to copy other technologies but not to make them better but to own them (snaps).
>counterpart in a rich discussion, rather than a one-dimensional anti-everything-Canonical mouthpiece?
I am totally not against canonical, it's extremely important to have a professional alternative to rpm-based (RHEL,SLES) distribution's, i am just not happy how ubuntu/canonical changed in the past ~10 years, maybe you see it as hate, but for me it's a bit more like a friend who betrayed me, i know sound's stupid, but ubuntu was that friendly distro who send free disk's around the world and is now the most proprietary one who had a Amazon search function.
>But why hate on me, and Canonical, when we literally put everything we had on the line for that vision, but screwed up?
Wait, no one hates you...quite the opposite.
>Why be an ass to the sorts of people who are willing to put every ounce of strength they have into something that you really wanted to see succeed?
Then stop brewing always your own soup, or at least make it better then the rest. Linux already has a massive NIH-Syndrome, and here i have to admit that Ubuntu/Canonical made the right choice with ZFS, when others (SLES) use that terrible BTRFS.
that is the key insight here. All of Canonical's marquee software projects over the years have been attempts to single-handedly control a given space, and they have essentially all failed.
Canonical is good at integrating other peoples' software though, which isn't surprising since that's been the core function of the company since its inception.
They did, but they still seem to be clearly leading among commercial linux distros, and the software they push keeps getting adopted while Canonical's does not. To me this reads as strategic maneuvering by Red Hat within the Linux ecosystem, and they're doing great at it, from what I can tell.
I write that as someone who finds them to have astonishingly poor taste and wishes they would knock it off, so I'm no Red Hat fan—but they do seem to be succeeding at some kind of broader own-the-core-software-of-Linux-outside-the-kernel-without-technically-owning-it strategy, by steering Gnome and systemd and having significant control over a complete (and, conveniently, architected such that it causes a ton of work for everyone who doesn't just adopt Gnome) replacement of heart of the GUI stack. Not that Canonical's much better (OMG, Snaps, WTF are they thinking)
Snaps are pretty fantastic for IoT, which is where we started. We have some sucky things to fix for snaps on the desktop, such as astartup performance and permission handling when you want to go outside the sandbox, but we will fix those.
If you really care about security and reliability across multiple Linux distros, then containerised software distribution is particularly challenging. But it's a fun technical challenge, and we have an amazing team that's dedicated to getting it right.
Our goal is very secure, very reliable, very seamless software distribution across any Linux, and as far as I am concerned it's great that there are multiple teams working to figure out that hard problem. How lucky for you that we're not all just stuck with one option!
Either that was a mistaken position by the hiring lead, or they were just shy to tell you that you were not the best candidate. Either way, my apologies.
Would not recommend working at Canonical. The CTO is incompetent, and bullies people out of the company if they disagree with him. Suggesting that the hiring process might be discriminatory may get you told that you "must live on fairy island" from where you should "send a postcard" on a company-wide email chain.
Please, please, please, find a new CTO. You've been led astray by the current one. Canonical has done amazing work building the foundation of large parts of the tech industry, but that is all despite, not because of, the current CTO. I'd love to continue recommending everything Canonical works on as best-in-class, but that sadly can't happen with the current technical leadership.
There are always lots of opinions about how to build a piece of software. Unlike a hundred yard dash, where the best performance is obvious to all, the risks and benefits of different engineering approaches remain a mystery until years after some work is done.
I've been around for those years :)
When we need something difficult and fundamental thought through very deeply, because it has to be precisely correct, I've learned to trust our CTO. Others usually want to go faster, or fix-it-later, or we-don't-need-a-spec-just-be-agile. That creates tension. But when I think it needs to be done very carefully and correctly, some tension is worthwhile.
I wish we had been more careful with some of the foundations for Unity, for example, because they would then have served others better even if the project itself failed.
People have different views of what matters, or how to do things. Whatever your views, there are going to be some people that agree with you, and some people that disagree.
Broadly speaking, when you lead something, the people who will step up to lead alongside you will be people who mostly agree with the position you take. Obviously. Why would they align and invest precious years in something they totally disagree with? It would be dysfunctional to have a leadership team in any organisation made up of people who have hugely different opinions about everything at a fundamental level.
Of course, that group has disagreements on stuff. We spend a lot of time talking about the things we disagree on, but we live with people we broadly agree with.
So from the outside it may look like the leadership team of any company are highly aligned. If you have been at Canonical, you know that in the leadership team we have lots of hard debates and discussions, all the time, in search of the best. You also know we are all working to the same goal, which is to help people consume open source cheaply and safely, at scale, across all kinds of compute.
So, here's how I interpret your comment. You saw a group of people that were broadly supportive of a position I hold, and which you disagree with. Sure, you disagree, and that's fine. Calling them sycophants is just finding a nasty word for people who happen to be aligned on something you disagree with.
The Canonical founder was recently bemoaning the lack of available talent limiting the growth of the company. The issue really seems to be a ridiculous hiring process limiting the available talent pool and therefore limiting company growth.
I’ve seen quite a few suggest that the “labor shortage” of this past year isn’t actually a shortage, but just people believing they’re worth more. Especially in the tech community; people are just more selective about how much BS they put up with in hiring.
"He noted that Canonical’s revenue last year was $175 million and that the company’s biggest challenge right now is that demand is bigger that the company’s ability to service it, in large part because there isn’t enough talent on the market for the company to hire."
I think that quote says the opposite of what you seem to imply it does.
It says that bullshit like Myers Briggs is misused by laymen. That implies that actual psychometric tests done by experts can be correctly used.
The real headscratcher though is that either Canonical thinks psychometrics work, but still bother with all the other nonsense; or they don't think they work but still leave it in in such a long hiring process...
I used to work at a company in which psychometrics were a major component. If you've ever taken a computer adaptive test, that test was based on a psychometric model.
For example in an adaptive test, question are trialed in order to determine how discriminating they are (i.e. which side of an ability estimate correctly answering places you), the likelihood of error in responding (maybe how confusing the item is or some other interference), likelihood of guessing, etc. It is essentially a statistical discipline with some grounding in psychology, but all of the pyschometricians I worked with considered themselves primarily statisticians.
I've never understood psychometry to be a theory about personality, though I suppose you could apply it to such a theory. Maybe its a case of the same term applied to two things?
In any case, "personality tests" is not the same as psychometrics. Just like astrology being fraudulent doesn't imply astronomy being unscientific, Myers Briggs being bullshit says nothing about the validity of psychometrics.
It's astonishing. If this line of reasoning was used against other fields of science, many people here would call it out immediately, but psychology is apparently always fair game.
(NB: I don't believe psychometrics are very good indicators in the hiring process.)
Psychometrics work, but they are only one dimension. Interviews and competence assessments give us other perspectives on a candidate. For a single role we will have 500-3,000 applications, each step of the process helps select the best.
It's not quite as much as Canonical, but it's still a _lot_ of writing to go through just to apply for a position. I considered applying there during my last job search but I just wasn't enthused about it enough to spend all that time writing before I even knew if they had any interest in me.
Yeah, it's not as ridiculous, for sure. But I think it'd be better to allow people to apply first for an initial screening and then ask for this stuff in pieces over the course of the process. It's really not right to front load so much work into an application, IMO.
There's merit to the idea of having a writing sample from a candidate. If you want to know that a candidate can communicate effectively, you'll want to both hear them talk and read their writing.
That said, I'm not sure whether or not these interview processes were set up with that in mind. If they were, they should also consider that writing can take a long time to produce and so the writing sample should be adjusted to not be too long.
> There's merit to the idea of having a writing sample from a candidate. If you want to know that a candidate can communicate effectively, you'll want to both hear them talk and read their writing.
Absolutely. I'm all for asking for a few paragraphs of writing later in the process, or for candidates to submit existing work early on.
> > there is absolutely zero chance anyone on the other end is going to take the time to actually read and evaluate each of them
> This is understandable to think, but is simply wrong -- I know because I'm a software engineer at Canonical who helps review these written submissions. They're sometimes read and reviewed by our CEO, though often he delegates that out to a pool of us reviewers. I always read mine thoroughly, whether they're 3 pages or 30. I don't have exact stats, but from where I sit maybe 80% of them are quality submissions.
Shuttleworth is directly involved in, and approves of, this "write ten pages about your childhood, then take a battery of personality tests, and then maybe we'll proceed" shit? Is this a South African thing? I know some cultures have very different expectations about how job applications should work.
First, it is a response to a massive flow of applications. We can more fully and fairly assess those who show they are willing to do some work to be part of our company.
Second, it helps address bias. The written interview is not a resume, which a hiring manager would read through bias like "I recognise that university" and "I like that country". The written interview is anonymous work, which can be assessed more objectively. We get it assessed, twice, anonymously, in the common case. For my roles, more than 100 people at the company help me with this, and I'm grateful to them for the work they do.
Third, it helps us focus our resources. We have very light resume screening requirements - be a plausible candidate from somewhere in the world - but we spend a LOT of time on the written interview, and interviews. So for those candidates who show they care, they have insights, and they are quick on the uptake, we can spend a lot more time on them.
Fourth, it helps avoid repetition. We can share a written interview with interviewers later in the process, and they get a lot of context on the candidate very quickly. They don't always read it before meeting the candidate, but they can and they should.
I'm sorry Mark, but you are also indirectly selecting for candidates that:
* Have ample time to write multi-page essays about themselves. Quality candidates are in demand, with multiple offers or solicitations for interview all the time.
* Feel comfortable sharing fairly intimate details about themselves and their lives - I personally would never, ever want to share some childhood trauma stories with my potential future colleagues, or complete strangers.
* Are willing to put up with ridiculous, time-wasting tasks - maybe you want this, because these employees will be more easily managed and manipulated, but just understand you ARE selecting for it, whether you intend to or not.
* Have a personality that you personally, or someone at your company, has decided is "an acceptable fit", thus missing out on a wide variety of potential candidates that might offer qualities you never even thought to test for.
It all seems very misguided, and that's the reason why there have been numerous posts both here on Hacker News, Reddit and other forums on the internet, lambasting your process. It's broken, and most people outside your company seem to realise this, while you do not.
> If 80% are good submissions it’s a useless filter and should be dropped.
How did you arrive at this conclusion? I can easily imagine that this process is so laborious that plenty of people don't even submit anything. Like the person who wrote about this on reddit. -- Now what reason would we have to think that the people who don't want to go through the process would also be 80% good? This may be a very valuable filter for Canonical!
I'm of the same mind. Interviews are not like school assignments, there's not some reason why the average person should "pass" most questions. Every question or step should provide clear information that lets you decide who to hire.
In practice it's not like this though, it's probably true that < 10% of what goes on in most interviews actually differentiates candidates.
I think part of it is it's just hard to figure out which 10% is the useful part. Though certainly if 80% are passing an early step in the process, it should just be scrapped altogether.
That wouldn't necessarily be true even if 100% of submissions were good, as you've completely discounted the cases where no submission is made. You can argue about whether the whole thing has any point or use, but a low % bad on submissions isn't on its own a sign that such a filter isn't useful.
Looking at the funnel holistically, I doubt anyone who didn't write the essay would make it through the "standardized aptitude and personality tests". And even so - if the filter is submitted vs. didn't submit - there's no need to actually waste time reading them...
(If that sounds absurd, it is and this funnel is broken as hell.)
That’s why I show them middle finger and went and worked for MSFT. I am not going to talk about why I installed Arch linux 12 years ago, when I am applying to work on kernel.
The trend in screening toward length and complexity, IMO, is not healthy. To me it suggests little confidence on the employer's part (either in HR and in working staff-members) -- and an attempt to remedy that squirminess by methods supposedly 'scientific'. A truly more 'scientific' approach might compare the 'results' of such a gauntlet with the predictions of a tea-leaf reading and an I Ching toss.
I'd rather work for them free for a week, then let them decide, and have something to show for this goofy Wechslering.
Right? There's no way they're hiring good engineers with this BS. I ghosted them after they asked me to do the essay round because I was already nearing the offer stage with multiple companies. Reading the comments on this thread and others it seems I'm not alone in that decision.
I’d love to take a questionnaire like this. It gives me then impression they are trying to get to know the people applying as people. I know bias sneaks in. I’m not sure this is necessarily a bad thing. Ultimately we are people, and will be working together as people. You can’t make everything unbiased and precise without treating people like replaceable robot cogs. Maybe some people like this or feel it is fair. I don’t.
It's a lot of work, to still be very early in the hiring funnel and not to know what the comp is likely to be.
Plus, I personally hate writing about myself, and writing something a bit boastful about myself is probably my single least-favorite thing to write. It feels really uncomfortable and gross. It's the kind of thing I hate enough that it might prompt me to go do something else I hate, like re-painting a room, to put off doing it. Résumés and cover letters are bad enough without "please write 500 words about your accomplishments in high school".
But that's just me. The large time commitment for only a sliver of a chance at a job granting unspecified compensation, is why it's bullshit for everyone.
Yeah I was through 4 or 5 “steps” before salary was discussed. And I ultimately think that’s why they didn’t move forward.
Lesson learned. I wasted quite a bit of applying, doing an initial interview, a second interview with a teammate, doing weird aptitude tests that had me comparing images, a personality test, a coding test while a manager watched, and finally talking to an hr person about salary requirements.
We generally discuss salary after we've got through early-stage interviews. By then we have a sense of whether someone might be great at Canonical, and we're trying to figure out where they might fit best. If a candidate has higher salary expectations, then that narrows the list of roles we could consider them for. If you had interviews after the HR interview, then I guarantee you that comp was not the reason not to proceed.
Lots of people won't do this work, it's true. But the ones who will, are much more likely to make phenomenal colleagues. They care, they are focused, they are diligent. Things work better when you are in a team with more of that. Try it someday.
How would you know, if you reject everyone who doesn't fit your predetermined box? Have you ever actually tried a different strategy for hiring, or building a team?
Also, for people standing outside the company looking in, things do not look happy and healthy at Canonical, which makes it even more likely you are being fooled by some of your own staff, or the figures they present to your, into thinking things are better than they are. I've failed to find a single positive review of working for Canonical, yet we have a smorgasbord of bad reviews all over the internet. You may want to evaluate that as "evidence" that something isn't right.
Try what someday? Making people jump through silly hoops and run a business that gets ripped on for odd hiring practices? No thanks. I'll stick with my current hiring practices.
Thanks for Ubuntu by the way. I've got well over 15k servers running it right now and wouldn't even consider anything else.
Personally I'd prefer to fill this out than have someone try to interview me and scribble some notes down that I later get evaluated on. I'm a good writer but I'm not the best extemporaneous orator.
Of course if it's this and then 14 hours of interviews (looking at you, Plaid), then it's a different story.
Whenever I see an unusually obnoxious hiring process, I assume they're hoping that only desperate H1-B's apply, so they can be as abusive as they want to their workforce.
Most of the questions are very funny. It's like when someone comes to you and says, define yourself as a human being. I would like to meet the very smart engineering managers who prepared these questions.
Though I think they should offer salary ranges (it's a legal requirement several places and is also just good practice), asking people to fill out essay questions seems...totally fine?
If you want to build an engineering culture which has writing as a core skill, this seems like a really good filter.
Essay questions are fine. Their psychometric assessment is a personality test and a fast-twitch "intelligence" test. Unless the job involves quickly rotating shapes in your head, I fail to see the point.
High school is also a very similar experience for people all over the world, which is not the case for university or work experience. If you are interested in understanding underlying interests, social dynamics and attitudes in a global pool of applications, then you are interested in things which are similar globally even if the connection might not be obvious.
I think you are mistaken about the relevance of high school. For a lot of people, high school was a traumatic and/or violent experience that they would rather forget. Does making high school "suddenly relevant" put up an artificial barrier-to-entry for candidates that were able to overcome bad experiences in high school and move on to something better? What about candidates who say something like "high school was awful and I got out as quickly as I could"? Maybe you have a bias in hiring toward candidates who are good at "making stuff up that sounds good"?
Life after high school is also full of trauma and difficulty. I think one of our problems in society today is that advertising and social media pump out a false narrative of beauty and success which makes people feel like failures just because they are suffering and struggling, when both of those things are the act of living as far as I can see.
How do you deal with adversity? How do you deal with peer pressure? How do you motivate yourself to work on things that feel irrelevant? How do you choose your hobbies and the people you associate with? Those are important things to assess in a potential colleague. And this is just one potential angle on how one might assess them.
As for "making stuff up that sounds good", you are correct, some people are better at that than others. Fortunately we have clear and objective measures of that to work with so forewarned is forearmed.
> What was your first Linux distribution, and why did you start there?
"Red Hat Linux, and I started there because I found a CD to install it in the cover of Red Hat Linux For Dummies at my local library."
What does that say about my qualifications for the job? I can scarcely imagine. Clearly the questions are prompting for answers that will be evaluated in some vague bullshit way, so why not evaluate the questions themselves in the same manner? I see a lot of questions about what I was doing in highschool. I therefore conclude these questions were written up by an airhead do-nothing who's life peaked in highschool and they've been dwelling on the past ever since. Am I doing this right?
In fairness (not that I think it's a good hiring process), your story about linux obviously shows that you were interested enough to get a linux book with CD from the library. That's more interesting (and in a real interview might lead to a more interesting follow on discussion) than someone who first encountered it in university because it was part of a course.
To be fair, in an in-person interview talking about something like this can be a nice way to settle into the interview and steady any nerves they might have - it's a question that doesn't have a wrong answer and doesn't (or shouldn't) have a "gotcha" attached that the candidate can be afraid of. It can spark a little conversation that could lead to other, more specific topics - but if it doesn't then no worries, it's served its purpose.
It's a bit odd to be asked to write this out ahead of time though, because if when it comes up in the interview it's like you're being questioned by the police on a statement you had previously given them and being tested what you said. Weird.
I don't think being confronted with an email I'd sent (or is it a form?) and being asked questions about it is natural at all. It is much weirder than actually having the conversation face-to-face.
It’s odd that you approach interviews with a confrontational frame of mind. I would totally expect, and be prepared for, an interviewer asking me to expand or comment on anything I send in as part of the process, be it the CV, application questions, code samples, coding exercises, and whatnot. It’s actually worse when an interviewer asks you questions that show they clearly didn’t read all those materials.
I fully expect to be asked about my CV or experience in an interview, I do not find that or interviews in general to be confrontational. Or even if there was a programming assignment I had to submit, reviewing that stuff is normal. What I find weird is to fill out what is effectively an GTKY interview over text, then have someone pull out it out during an in-person interview to discuss.
Also even though I usually don't feel that way, it is completely ok for a candidate to feel confronted or cornered in an interview and not at all odd.
Mandrake was my first too, but it was because I was at some tech "flea market" show thing in San Diego in the 90s, and they were selling Mandrake CDs for $5.
At the time, I was frustrated at the cost of WinGate, a socks proxy, and I wanted to be able to play games with others on my local network with remote friends.
I had a clever idea, that I could dynamically rewrite IP packets to an internal IP address based on the port assignment (I had essentially re/invented NAT). So I wrote it in pretty low level c, got it mostly working, then a friend told me about this thing called Linux that had a feature called ip masquerading that did everything I was trying to do.
Mandrake had an easy way to compile the kernel with IP masquerading support, and I immediately fell in love with all things Linux.
I remember showing friends the customizable desktop, with silly things like snow falling. They thought I was a leet hacker :)
Slackware downloaded in the format of ... 13 or 14 1.44MB floppy images from a BBS using my crappy 2400bps modem over many nights. Got it installed, played with it a bit but realized I couldn't run any of my games, shrugged and re-installed Windows... stayed like that until Red Hat 5.2 and then I've been running Linux as a daily driver ever since.
Considering another step is psychometric testing, and given the nature of these questions, I assume there's some woo-heavy tea-leaf-reading process to evaluate the submissions. "This person used 'I' too few times while writing about themselves, so they're clearly not confident enough" or whatever.
I love these, they always fool employers that I'm a lot better than I actually am. For some strange reason, these tests always determine that the kind of people who write tests are ideal employees. "System builders."
And if they think this is a good idea for hiring, they probably don't stop there. Imagine all your work emails being run through such evaluations every year during performance review.
But I get it. They're trying to get people that "love linux". It's part of the cultural fit, sure
Now think for a bit about hiring for people that "love something" and what does that tell about the professional environment of that company. Wouldn't it be better if they actually hired people that are good at the job?
For all the talk about diversity and inclusion, this is a very good example of how the opposite looks like.
Yes, I do want to work with people who are passionate about what we do.
It makes for more interesting meetings, more interesting ideas, more interesting software. I love what I do, and I choose to set things up to maximise the chance that the people around me are equally excited about what impact we could have in the world.
Life is short, the way I want to use it is to help people who contribute to open source - all open source - have the biggest possible impact from their generosity. By making open source easy and cheap to consume and rely on in production, I think we amplify the contributions of others, and I feel very good about that. We also try to innovate on pieces of the landscape, and those are fun too.
But you may be confusing the first step of our hiring process - the written interview - with the whole of the process.
If someone does well on the written interview, which is usually twice-assessed and done so on an anonymous basis, they get additional screening tests and interviews to assess professionalism and experience and competence, which helps us place them in the best part of the company for them. It also helps them learn about us and how we work to make sure that they are making the right choice for the next segment of their career.
In the past I think we over-prioritised passion-for-Linux relative to competence, and we paid a price for that. So now I'm pretty focused on looking for both, and ensuring discipline in the offers we make across the whole company.
Thanks for answering this, and it definitely makes a lot of sense. Last paragraph is an interesting tidbit.
My original point was in the sense that passion can also cause a bit of tunnel vision. On the other hand I certainly can't say Canonical isn't innovative.
> Wouldn't it be better if they actually hired people that are good at the job?
The context of this whole conversation is an attempt at automating a process that doesn't take well to automation. This thread isn't about hiring people, it's about building a system that hires people.
Funny enough, I have a great answer to this question - my first distro was exactly Ubuntu, in 2005. They shipped me a couple of dozens of CDs that I distributed in my city, mainly to my collagues and members of the local LUG. I think I still have some CDs from this batch.
Sounds like their application process works. If someone is unwilling to fill out a basic questionnaire they are probably aren't a good candidate. The questions seem reasonable and they aren't asking for you to write a book for each one.
25 questions, some of which would require a paragraph or more to adequately answer, is not "a basic questionnaire," is completely inappropriate to ask someone to do before even an initial phone screen, and is artisanal small batch bullshit of the highest (lowest?) quality. Whoever put this together should be fired, and the people defending it should be ashamed of themselves.
Yeah, I'm sure their application process works great, for keeping their pipeline nice and empty of potentially worthwhile candidates. These questions are anything but "reasonable", either in their quantity or in the depth of demanded response.
I'm sure that the team at Canonical feels this process works for them, and maybe it does, but I'd love to see what metrics they have to show that it does. (Or maybe they for some reason have to practice "hire slow, fire never"...)
To be clear, we are hiring lots of people every month now. We have 10,000-20,000 applications per month, it's just that we have a very high bar, and we are rigorous about it which takes time and effort.
Also, our process doesn't work as well for some roles and some locations, so we continue to evolve it.
But on balance, I am very happy with the growth of our teams and the calibre of our new colleagues.
The questionnaire looks pretty reasonable, but I'd be concerned about the sketchy looking "Psychometric assessments" that's next in the list.
Also, at what point will the candidate be able to vet whether Canonical is a suitable match? Is that only after this questionnaire, psych test, and take-home assignment?
Pretty entitled freshly-minted BS degree, no doubt. Goes from wanting to work for Canonical to wanting to "punish" them by telling them to stuff it and then posting a screed to reddit -- all from just seeing the application questionnaire.
Doesn't seem a good indication for conscientiousness, integrity, and tact, frankly.
> Doesn't seem a good indication for conscientiousness, integrity, and tact, frankly.
Those things are for people. Companies can get fucked. They're a tool, and that's it. I don't practice tact toward a screwdriver. This application process is a joke, and warning other humans that this particular tool likes to waste their time is conscientiousness and integrity.
Being kind to the people employed by them is commendable, of course.
Yeah, those things are for people, one of whom is the applicant in question.
If you want something, you will have to jump through hoops. I'm not saying the hoops are legitimate, but you will have to jump through them anyway. My comment was about the person, not the hoops.
If he really didn't want the job that much, then don't bother. If you do, jump through the fucking hoop. It honestly just seems like the person didn't want the job enough to invest in actually applying, but enough to bother publicly attacking a potential employer. What does this really say about the person's character?
> It honestly just seems like the person didn't want the job enough to invest in actually applying, but enough to bother publicly attacking a potential employer. What does this really say about the person's character?
They're thoughtful enough to warn others. Even at the risk of people like you thinking ill of them over it. Rather kind of them, I think.
[EDIT] Do pointed reviews of bad products bother you, too? Seems like the same thing, to me.
I am sure that is the exact attitude they are looking to filter out. And to be honest I can't think of any employer that is going to want hire an applicant that thinks they can get fucked, or think that what they do is nothing more that work for a "tool".
I’m good at these sorts of written/verbal assessments, and even my eyes start to glaze over just reading the list of questions, let alone thinking about how to answer them. A quality response to this sort of questionnaire would take at least an hour and that’s before you even get into the actual interview process or hear a salary range.
And as dumb as questions about high school and college are for professional job interviews (unless you are hiring straight from undergrad, I just don’t see how this is relevant. And even for college recruits, I have a lot of problems with this approach.), it would be one thing if I thought the recruiter or hiring manager was actually going to read each response from each candidate. But I cannot imagine they will.
For a company that has openly expressed its struggle to hire, it’s stunning to see them double-down on such a terrible recruitment process.
The people who you want for this job probably have better things to do than waste an hour or more answering an overly-long questionnaire before they even find out the salary range.