This is perhaps even worse than the state of interviews today.
The exams mean very little. They are easy to study for and tell almost nothing about a candidates performance, just knowledge. If we wanted to go this route, we'd be using GPAs to differentiate candidates, but that rarely seems to be a factor today.
You'd be surprised at how many people I've interviewed over the years have been unable to hide the fact that they would be terrible to work with. In programming circles at least, I think the ability to socially manipulate and mask true character in an interview setting is way less common than the ability to practice questions and memorize answers.
Yeah, that's one part of it. The other part of gaming the interview is networking. It gives you a massive advantage if you're posting into a role and the hiring manager or someone on the team knows you. It has nothing to do with your performance vs the performance of the other candidates. The other candidates might be just as nice to work with. This creates a ton of potential bias in your favor (aka gaming).
When I'm doing an interview, I really don't care much about the code being written; I don't even care if it runs, is correct on some percentage/point scale or whatever else like that. I care about how they're thinking about it, how they react to my hints, how they act when they are stuck, if they ask for help or not, if they try to cover up what they don't know or that they made a mistake or are open and honest about it and treat it as an opportunity to learn, etc.
A standardized exam result won't give me any of this information/experience. I'm going to be especially suspicious with people who have a perfect score, experience has shown me that the best engineers often (not always) don't have great academic results, while those who had perfect results often can't function well independently in a real-world unstructured environment.
> I really don't care much about the code being written; I don't even care if it runs
Great! Then you're not one of those interviewers with "the one true answer" in mind.
> I care about how they're thinking about it, how they react to my hints, how they act when they are stuck, if they ask for help or not, if they try to cover up what they don't know or that they made a mistake or are open and honest about it and treat it as an opportunity to learn
Then again, maybe you're one of those interviewers with "the one true thought process" in mind and the luck to have double-majored in Computer Science and Psychology, so you have the ability to understand how people think.
There isn't one true thought process. I'm usually surprised by the various different ways people can approach a problem. What I want to know is whether they actually do think about the things that matter in software engineering such as complexity, readability, maintainability, scalability etc, the specific approach isn't important.
Not sure why would I need a psychology degree? I'm not doing a psychological evaluation, I'm looking for a good engineer that will fit in the team and company.
> I'll care about how they're thinking about it, how they react to my hints, how they act when they are stuck, if they ask for help or not, if they try to cover up what they don't know or that they made a mistake or are open and honest about it and treat it as an opportunity to learn, etc.
That's what you say, but in practice it's a guaranteed way to engage in discrimination at multiple levels using personal biases that are not present in a standardized test.
As I said in another comment, most of the best engineers I ever hired had bad grades or no grades at all - they didn't care about that and spent their time building real things instead. And on the other hand, some of the worst people I ever hired had perfect grades. Of course, this is not universal, and there is a large space in between - but I just can't agree with any usefulness of such tests. Real world engineering is totally different from any standardized tests..
How will a result from a standardized test tell me that their approach to engineering matches ours, and that they will fit in with their future team and company?
True, but using standardized tests for hiring is legally perilous in the USA. Employers can incur serious liability if they use a test which has a disparate impact on a protected class, and they can't prove that the test is directly relevant to the job.
If anything a candidate caring too much about that is a warning flag. Competitive programming is a very different thing from writing software which is actually intended to be useful, and the approaches used for winning competitions are very bad for code that isn't going to be thrown away immediately afterwards.
Interesting curiosity, but it says nothing about their ability to function in a real world software engineering organization - so we're going to be coding on the interview anyways. The interview is really not just about the coding skill itself, that's actually the much less important part of it.
optimizing for people who do competitive programming means you optimize for education culture where repetitive practicing of a routine is important in contrast to education cultures where careful thought and planning and critical thinking matters more.
in other words it benefits cultures heavily into all kinds of olympiads from age 5 on… i will not elaborate further as you know what i mean.
I honestly can’t tell if this is sarcasm or not, so forgive me if it is. But no, I don’t really see how it would translate to being a high performing employee. Probably most of all, I don’t have the relevant context to know what sort of an achievement or skill any ranking could represent. I’ve never heard of this before this comment, so if you were like… 6th in the world or something, it’s hard to consider it impressive when 99.999% of the population has never even considered competing in it.
This is exactly how you filter out careful thinkers. Put them under some arbitrary time pressure. They'll fail time and time again. Then you'll be stuck with clever yes men who can't see the big picture.
Yep, that's me. I'm the guy who asks about the requirements then asks additional questions about the edge cases. Basically, I attack the business system before I build the technical system. Any flaws in the business system would manifest in the technical system, so might as well discover them first. Then I fail the interview because I'm slow.
I see two kinds of technical job interviews: the FAANG style and the rest.
Let's start with the latter: the problem is that not every company wants exactly the same developer profile. Acing a standardized exam means that you are really good at what the exam is testing. But because you excel at leetcode interviews really does not mean that you are the best fit for most companies in the world. Maybe this company will want somebody with experience in a specific framework ("we have a Qt mobile app", "we work with Laravel", "we write firmwares", ...), that company will want something with domain knowledge ("we need someone who understands the medical domain", "we need someone who understands maritime navigation", ...). You can't standardize that. For the small parts you could standardize, it exists: a CS degree, "5 years experience as a dev", etc.
For the former, it seems to me that it is already pretty much standardized. When you interview for FAANG companies, you need to read a lot about the interview process and prepare it like you would prepare an exam. If you are super ready for the interview at Meta, you are also super ready for Google. It is some kind of standard already. Maybe the FAANGs could team together to share the exams, but the economy would be marginal (otherwise they would have done it).
Would this really be any different from certificates? And I don't think those are efficient in anything other than increasing billing rate or marketing for consulting.
Such tests would be gamed in multiple ways. Or cheated. Or corrupted. Or info dumped...
Management and those hiring don't want this because they like to silently discriminate in hiring using various personal biases. They will never admit it, but they do. With a correct standardized test, they lose this opportunity. Right now, the closest thing to such a test may be your StackOverflow or similar score.
As someone with a nearly 40 year career in tech with 30+ in a hiring and management level…I’ll gladly admit that a candidate’s personality and demeanor in an interview factors higher to me than technical ability. Is that a personal bias? Yep, but it’s a bias based on experience as to what works better on technical teams and worth the skill trad off.
So I would gladly higher a friendly and open average skilled SWE than some aloof technical genius who is going to be likely regarded as an asshole to the rest of the team.
If I was forced to higher based on test scores alone, I’d opt to quit that role…because that to me would be a recipe for failure.
> some aloof technical genius who is going to be likely regarded as an asshole to the rest of the team.
We can agree that no one wants a true ass, but I don't understand why some aloof technical genius needs to automatically be regarded in that way. It looks like the connection is in your mind.
Moreover, it obvious to me that your goal is to use up VC funding, not to generate revenue for the company. An average SWE isn't going to do the special work required to generate revenue.
>I don't understand why some aloof technical genius needs to automatically be regarded in that way. It looks like the connection is in your mind.
…Or it could just be long experience. I have seen these kind folks sink teams and products. If my goal is to make what I am managing successful, why would I hire personality types that historically have proven to impede success in my experience? Believe me, the type I am referring to usually comes out in interviews.
>Moreover, it obvious to me that your goal is to use up VC funding, not to generate revenue for the company. An average SWE isn't going to do the special work required to generate revenue.
No, my job is to build successful teams and products. To do that i need to find compatible people to get the job done. An asshole rockstar might get me 75% there then prevent me from achieving the final 25% because they can’t work well with others. A cohesive team can get me to 100%. And…not all rockstars are assholes, so I choose to find the wheat and discard the chaff.
Would it be offered any time I want to take it or is it only available to take once a year? If the latter why would I take it if I’m happy in my job? What happens if I don’t take it and get laid off later in the year?
How someone talks during their technical job interview should be seen as equally important to the content of what they say during it. Will there be an oral component of your exam?
Are there different exams for different languages/technologies and industries? How many exams do I have to take?
Do people cheat on the yearly standardized exams in place (that are used, for university entrance, for example) now?
Do hiring companies tend to believe that their needs are very similar to other hiring companies? How much faith would they put in such a standardized exam?
The yearly standardized exams could be company-specific. So, if you want to apply for a software engineering position at Google, you would have to take the yearly Google standardized exam for software engineers.
PRIVATE universities maybe. I graduated at a public university.
companies need to educate you on the job. that will never change and it is unreasonable to demand you already have all skills ready to go. but that is where we are in the US….
it is also crazy that you need to copy paste job requirements into your CV role per role. a resume or CV is something you make ONCE. but i guess HR drones are too dumb to read so that is where we are.
We pay one group to teach us and expect a different group to pay us and teach us?
Reality for the many might be somewhere in the middle, off to one side.. there’s academic organizations with billions not pushing it off into organizations with millions to deliver what students paid tuition for.. relatively speaking?
Companies don’t always have to hire graduates to educate them on the job. Profession and career development so important but a part of the work.
It’s a skewed tech reality distortion field where tech companies compete by playing defence to tie up labour so others can’t grow… that results in those pesky mass layoffs. Repeatedly.
Organizations very well may take over educating the workforce and leverage it like salarymen in Japan, low pay for lots of hours and stability. Not sure if I like that either.
Or maybe they’ll just take the demands for remote work locally and let it be a global market.
In the scenarios where a company is paying to educate you .. for free .. in addition to paying you a salary.. to learn? That you’ll become more valuable with, not necessarily give back or create value greater than what you’ve been paid, in addition to you paying an academic institution money to learn.. often under debt is.. sound?
The gap between academic degrees and the workforce is generally increasing without cherry picking small groups as exceptions.
Historically, Universities that taught you how to think didn’t have STEM. They consciously moved from liberal arts into into stem for the funding opportunities.
Most of all I believe everyone has a right to learn and access opportunities to improve their life. The above doesn’t sound like it’s meant to speed that up.
Most companies wouldn’t want to spend the time making company specific exams so they would buy some off the shelf exams and we are back to Leetcode problem.
In my long experience, the type of questions that will be on this test will quickly morph into a "Marketing Type Test".
In IT, changes happen so quickly, the tests will be out of date as soon as they are written. So as time goes on, you end up being quizzed about marketing instead of real IT terms.
Plus you will need many tests, one for admins, support, development. As a final note, I can also see these tests end up being about methodologies like Agile instead of real Tech.
1. You've just passed the cost to the unemployed developer.
2. Some of us do badly at standardised exams but are still highly competent and can prove it in other ways if given the chance.
3. If I fail an interview, I can reflect on where I did badly, study hard, and hopefully do better at the next interview days, weeks, or months later. A yearly standardised exam means I'm potentially unemployed for a whole year.
Because realistically, software engineering requires different skills from leetcode.
The FAANG-style leetcode nonsense is deluding people into believing that selects for the right engineers. It doesn't. It is flawed and only works for factory-style companies to do a bare-minimum job while hiring.
Well if I know how to do my job and I do it well, so must those recruiters, right? I'm sure the recruiters would agree.
Conversely, if other people who think and are told they're doing their jobs well but we have evidence that they aren't, what does that say about me doing my programming thing?
Just think how much businesses would pay to keep their employees' credentials current!
You could also charge exorbitant fees for training, books, and videos people would use to prepare for the exams.
Maybe you could even setup a school business built on teaching people stuff for all the exams and then give them an "everything" certificate after they took all the exams the school had?
The IEEE has their own software engineering certification which covers all the basics and isn't tied to any vendor or technology. The content is generally pretty good, but hardly any hiring managers take it seriously.
Most interview processes are for making decision-makers feel better, not for finding the best candidate. Very few companies use interviews with a theoretical basis, let alone an evidential basis.
And most decisions of any kind are made emotionally, then explained with post hoc rationalizations. To magically seem logical and objective, you can withhold your decision until you've explained the rationalizations.
Consider also the typical engineering "which thing should we buy" matrix, which is often thinly masked biases of the author and arguing points against other views from the team. Consider also the rationalizations in design docs ("why we should make it like this") that barely seem to cover up the designer's favorite methods and tools that are actually driving things. We spend a lot of time/energy trying to seem logical when we're fundamentally not.
I feel like the answer is that people like their gate-keeping.
There are a lot of programmers who like being part of a special club and want to manage who gets in. It’s couched behind meritocracy, but I think it can be boiled down to gate-keeping.
you mean like go to university and get a real CS degree and have companies use that to give you a job? oh wait…
this was supposed to be how it works but then some indian dude came along and made a platform and then some woman wrote a book and got in a HR decision making position in a FAANG and destroyed the value of everyone’s degrees by introducing coding interviews you can only pass if you grind them
The exams mean very little. They are easy to study for and tell almost nothing about a candidates performance, just knowledge. If we wanted to go this route, we'd be using GPAs to differentiate candidates, but that rarely seems to be a factor today.