Maybe there should be a standardized test for software engineers like the LSAT or MCAT. there are just too many software students every year now to filter the top 200k or so out of 2 million using interviews. Who has the time or interest for that ? we need one to track progress in LLM’s anyway. humaneval is weak.
MCAT filters the med school application pool down from 80k to 23k. LSAT filters from 120k down to 40k. Without them, there would be millions of applicants every year.
How long can we keep relying on high school level tests that prefilter into computer majors for college. We need a post college test.
Yeah I thought many times about a standardised interview process. There are pros and cons in such system though.
Would the exam be broad or go deep, would it test all kind of coding problems or would there be categories?
In a way Big Tech's interview process is this standardised test, but once you go a tier lower, the interview process is a wild west. Much easier questions, and much more conversational interview which introduces a lot more bias and subjectiveness.
In every company I have been, this has been proposed in some form, and the argument is always that the current coding questions approach is a reliable way to filter out a lot of the unwanted candidates, at the cost of also rejecting some very capable engineers. It's an accepted trade-off.
According to https://survey.stackoverflow.co/2023/#section-developer-role... , about 58% of developers are web front-end, back-end, or full stack. That's probably an undercount since their survey includes non-practitioner roles such as management, students, and QA. All of those will be using more or less the same frameworks, libraries, and other software to target the same browsers, database engines, operating systems, etc. Doesn't seem like that much of a stretch to be able to write a test that evaluates proficiency in the common underlying concepts used across web software.
> What we do as software engineers isn't standardized. If you are doing things that have been done before, you are wasting your time.
At the same time, this type of thinking is what results in engineers wasting time and money re-inventing the wheel, on things that usually have no business value. Just look at how much time and money was wasted at Uber reinventing simple technology solutions.
you can absolutely create a standard test with predicting power on success as a software engineer. Real Law isn’t standardized either, neither is real medicine. They are partly art as well.
I think it's a valid suggestion, but where are you getting this from:
> MCAT filters the med school application pool down from 80k to 23k. LSAT filters from 120k down to 40k. Without them, there would be millions of applicants every year.
How could one reliably estimate the number of people deterred from pursuing medical or law school due to the respective standardized tests?
There are a 140k premed first year college students. That number itself would be larger without the MCAT hanging at the end of the road, if med schools relied on haphazard interviews alone without the MCAT.
There are 5 million college students per year, a lot more of them would take a shot at law school and med school, if it wasn’t for MCAT and LSAT.
You can confirm these numbers, they are public, and gathered annually by various administrative bodies involved in the standardized tests and credentialling of the professional schools.
I don't doubt that there are 140k premed undergrads or x million college students.
I doubt that there is any reliable estimate of how many people would apply for med school or law school BUT FOR their having to take a standardized test.
All I am saying is there should be some scalable if crude method of quickly ranking new software engineers by aptitude as a pre filter for interviews as there is for doctors and lawyers. it’s just an opinion.
MCAT filters the med school application pool down from 80k to 23k. LSAT filters from 120k down to 40k. Without them, there would be millions of applicants every year.
How long can we keep relying on high school level tests that prefilter into computer majors for college. We need a post college test.