1. Docusign, for not using Coffeescript, Angular, and Stylus (3 technologies I had never previously used) to solve their very long take-home test. I apparently provided a great solution, but they were concerned that I didn't follow those rules specifically. No, I'm not going to spend hours getting up to speed on your framework just for an interview, nor am I going to submit sub-standard solutions to to conform to your arbitrary restriction of using the framework du-jour.
2. Google, for not being able to, "prove a random number is truly random," during my onsite interview panel. The guy offered no hints when I wasn't sure where to start, he simply continued to repeat the question, a bit slower each time.
3. A local Seattle startup, who required 3 onsites: A technical, a culture, and then a full-day of "integration" (working on real problems with the team). I apparently passed the first 2, but was not invited to the 3rd. Turns out the HR director that was working with me quit before scheduling the 3rd. I never found out until after I had already accepted another position.
I’m in Germany, and I keep forgetting to ask them to slow down when they German at me too quickly. Of course, when they know I’m English, then they English at me and I don’t learn German anyway…
Reminds me of a question one interviewer asked me to do at the very beginning of the session when I did my first onsite at Google - write a regex for emails. I told him no, and explained why. He then asked me to write a naive email regex to catch a simple situation, which I subsequently did.
I actually would argue that a single number alone is always random. This is because a pattern is a recognizable arrangement of elements. A single number is a single element and cannot be arranged, and so it has no pattern. Since the definition of randomness is the lack of a pattern, and a single number can never have a pattern then a single number is always random.
Yeah, my first reaction to that prompt is "it can't be done. Saying whether or a not /a/ number is random is meaningless, only sequences of numbers can be treated with statistical methods"
If the last 'n' generated numbers were also given to you, this seems like a fair task - to check the randomness of a sequence of numbers. Wikipedia has an article about "randomness tests". [1]
An example algorithm could determine if a run of integers represents a random sequence with some probability 'p'.
Those tests can "disprove" that a sequence is random (quotes because of course a true random generator will occasionally generate suspicious sequences of arbitrary length)
But you can't prove that a sequence is indeed random: the sequence [sha256(1), sha256(2), sha256(3), ...] passes the usual tests and isn't even clever. For any set of tests you can write a pseudo random generator that passes them
To prove a random number is truly random you would have to prove the random number generator that gave it to you is truly random.
If the output was plotted (from the minimum possible output to the maximum) and run a sufficient number of times, I would expect the chart of the output to have a very good distribution, and not have any serious hotspots.
To prove this, my solution would probably go something like:
Probability of number N being output = 1 / MaxPossibleNumber
Then I would run a test a sufficient number of times (depends on what MaxPossibleNumber is) such that as the amount of numbers generated approaches infinity, the percentage of times any particular number is output approaches 1/MaxNumber.
The harder part would be making sure sequences of numbers don't repeat. In addition to looking at the total distribution, I think I would also want to log the order in which the numbers were generated and then run an algorithm that looks for sequence repeats.
I am NOT familiar with how random number generators work, so this is a naive solution off the top of my head and could be completely wrong, but w/e.
Re 2.: It's not possible to prove that a single number is random. You can run some tests on a series of numbers. If this were me getting that question, I would first say that I'd google for some standard tests, then if being asked if I know some by heart, I would start with the average and standard deviation, then I would draw a histogram and expect the distribution to be flat and finally I would try to visualise the numbers in a two-dimensional grid and look for patterns. A truly random number generator doesn't produce patterns.
No. Numbers have to be uniformly distributed in order to be random, since Randomness is the lack of pattern or predictability in events[1].
You can randomly sample from any distribution, but as long as you do it from a non-uniform distribution, you can start predicting the pattern (e.g. if you are sampling from a gaussian, you can start predicting median value)
If you follow that assertion, a great number of mathematical concepts become meaningless, e.g. probability distribution.
That assertion is even at odds with the second and third sentences of the article you linked.
> A random sequence of events, symbols or steps has no order and does not follow an intelligible pattern or combination. Individual random events are by definition unpredictable, but in many cases the frequency of different outcomes over a large number of events (or "trials") is predictable. For example, when throwing two dice, the outcome of any particular roll is unpredictable, but a sum of 7 will occur twice as often as 4. In this view, randomness is a measure of uncertainty of an outcome, rather than haphazardness, and applies to concepts of chance, probability, and information entropy.
It's funny how this thread shows the exact problem identified in the article : a LOT of people have some basic idea of how to answer the question, and come up with strong opinions they "know" are right, and arguing with others.
Fact of the matter is that the question is too general, and there are no real answers to it that aren't extremely general themselves.
(One mistake in your response would be that obviously random numbers are taken from a particular distribution, and of course there is some information there about what to expect the next random number to be. Taken to an extreme you can take random numbers from the collection {1}. Can you predict what the next random number will be ? I think I can ... Too general questions means you have to take trivial cases into account)
I wonder how many of the posters into this thread would reject candidates based on the candidates strongly disagreeing with their statements. Especially because I've got a degree in statistics and I would agree with that this question can't be answered.
um no. Even on an infinite sequence, it's infinitely unlikely to get a flat distribution. That's what's annoying about real randomness, compared to 'a flat distribution'.
More generally, one should probably say that random numbers taken from a distribution match that distribution.
Taken to an extreme one might ask, can you predict random numbers taken from a collection with only one element. Obviously you can predict the next number. It'd be like a roulette wheel with only red numbers. The outcome is random, but perfectly predictable to be red.
I'm currently interviewing with a company where the HR person in charge of my case left the company. My emails to him were getting Postmaster responses, so I emailed someone else asking what happened, and apparently no one knew that I was just hanging out in interview limbo.
Being proactive in your interviews never hurts; so long as you aren't bombarding them with emails it's unlikely to hurt your application.
What we can do is slice up the number into a given length and estimate the H_{\infty} entropic measure among our sliced-up numbers, treating the numbers as samples from some random source.
edit: if for example you're a crypto researcher, then this question seems warranted.
At Level 12, we pay for the time spent taking our skills tests. About 16 hours worth if a candidate makes it through all phases. But I've not paid for the on-site interview, other than travel expenses for remote candidates.
He's saying that's in total/spread out, but still seems on the high side. Between a take-home coding challenge (2h?), online coding challenge (1h), talking to recruiter/HR/hiring manager (1h), multiple interviews (coding 1h, culture/fit 1h, system design/architecture 1h, whatever 1h), you're already at 8 hours for a lot of places.
I appreciate them even knowing how long a person would spend on their interview process though.
Since I am an external recruiter, I ask for payment on behalf of my candidates but I can imagine if you formulate it nicely that it might work out, too without having a recruiter representing you.
You can argue that you are interviewing with five companies and all have 4-8 hours coding task, you can't possibly do all of them and you will just do three maximum.
I would not take excuses like this for rejecting a candidate, especially a highly qualified/skilled candidate, at face value. Despite the pervasive rhetoric about wanting the "best" or hiring only the "best," both individual interviewers and companies as a whole are often afraid of hiring candidates who know more or are more skilled than themselves. The unstated fears include that the candidate will, if hired, outshine colleagues, his or her boss, or even, in some cases, the founder or CEO of the company -- especially in the case of startups where venture capitalists or other investors could be impressed by the new hire.
Since it is usually not acceptable to reject an applicant for this reason, some pretext will be generated. One way is to focus in on some area where the candidate is weak or different such as using a different software tool. Another way is to use trick questions or problems.
This is an article on how this was allegedly done in academia in the Soviet Union:
Actually, I had drafted "Horror story 4: Candidate was rejected because he was better than the interviewer" but I thought it is too complex to be included. I might write a whole blog post about it. Thanks so much for linking to the paper.
He didn’t even understand the question “what would you have to see to convince you that you’re wrong?” meaning he wouldn’t even be able to recover from the error!
For older engineers this is one of the reason why they are not hired. There are always various excuses, but basically no one wants to hire his next boss.
>>both individual interviewers and companies as a whole are often afraid of hiring candidates who know more or are more skilled than themselves.
Reminds of me of an experience a few years back, where I aced one round, in the next there is this guy who comes along, asks questions. He got very angry everytime I gave a correct answer to his questions.
Its almost like he was looking for an excuse to fail me, and wasn't getting one.
afraid of hiring candidates who know more or are more skilled than themselves. The unstated fears include that the candidate will, if hired, outshine colleagues, his or her boss
At one company the joke was you should never hire someone taller than yourself, because you would end up reporting to them. It was funny because it was true...
Isn't that your job as a founder to find and surround yourself with people that are much smarter then you in their given field of expertise?
You as the founder should then coordinate the efforts of your different experts and handle all that other startup stuff (talking to investors, aquiring customers, etc)...
Venture capitalists for example are notorious for investing in startups and replacing the founder and/or CEO with "more experienced management" as the company grows.
A well known prominent example is the ouster of Steve Jobs at Apple in the 1980s by John Sculley whom Jobs had supposedly hired -- with the support of the lead investors such as "Mike" Markkula and Arthur Rock.
Steve Jobs is not an unusual case. The founders of Cisco Sandy Lerner and Len Bosack were ousted by Don Valentine/Sequoia:
I remember one company rejected me a few months ago for not knowing enough algorithm/data structure knowledge - I subsequently passed interviews with two teams at my current employer, one of the famous big tech companies.
I had a startup pass on me as well that prior job search for reasons unknown even after acing their interview. I found out only 2-3 weeks after interviewing, and I think they’re still trying to hire for the position (it uses Angular 1 - I also happen to probably be still one of the most knowledgable people about Angular 1 outside of Google).
I got insight into one company’s interview process after doing a take home project, and one person mistakenly replying to all with me in the email chain - the person didn’t think highly of my work on it, despite that I only spent 4 hours (the time I could spare on it) and kept clean abstractions, setup, and accomplished the main point of the take home project. That experience cemented to me that take home projects are a race to the bottom with it being necessary to spend an extraordinary amount of time and little reward back.
Amazon ghosted me while we were in the process of setting up onsites in San Francisco and we were taking about compensation in parallel (they started the comp conversation) - I’m pretty sure they did so after a few back and forths due to not being willing to bend on comp numbers. I still get emails from recruiters about 10 times a year from them.
I’m sure I’ve had others, pretty sure most people who’ve been in the industry for a while gets them if they have interviewed around.
That's fine, but ghosting while also in the process of setting up an onsite is pretty bad, especially for a company that should be at the stage where they get that right. There's no good reason for that behavior.
1. Honestly, if I were giving a simple coding exercise and the candidate submitted work using their own custom framework (which is implied here), I might reject them also, or at least put them at the bottom of the interview list. It's just a complete misread of requirements, and points to an inability to follow directions.
2. On the one hand, Joel Spolsky created a lot of problems with his idea that a "no" from any interviewing dev results in a "no". Some people just don't evaluate interviews very well.
OTOH, everything looks like it went well here. One interviewer voiced concerned about a specific technical field, but the team as a whole gave that item its appropriate weight. No problem.
3. Yeah, HR often sucks. But I don't see how a story about the recruiter completely dropping the ball for two months is somehow a defense of how valuable recruiters are.
1) The candidate implicitly thought he should use it, because I liked his stuff during our technical interview. But then the interviewing engineer at the firm did not take time to look into my notes. So it's more an example of me pushing what I like instead of assessing what the firm likes / how much time they really have looking at profiles.
I'm with you on 1) because I've worked with these kind of people before; smart people who disregard existing solutions
AND disregard team choices for technology stacks. Their code is impossible to integrate, difficult to test, difficult to review, and impossible to support. But by god they think it's the better way.
You cannot deduce from that that that engineer is one of "these people." There is not enough information in the situation to draw that conclusion. It's like claiming, solely based on your remark, that you are one of those people who jumps to conclusions prematurely. It doesn't work like that.
Regarding your point 1, wouldn't the sensible thing be to question them about this sort of thing? Find out their reasoning for implementing the solution the way they did.
A lot of this depends on context, and obviously we're not getting any of that in a quick anecdote (not OP's fault, just the nature of things).
In a larger company, where you're always hiring and there's specific time allotted for it, that approach makes some sense. But in a smaller company where I might have one or two positions to fill and the interview process is a serious hit on productivity, presumably the net step is to set up interviews and the first interviews are going to go to the people who didn't completely misread the exercise and submit Enterprise FizzBuzz.
Now try being not an ex-googler, or not so great on the spot. Have been a great Python developer for years but personally have given up on jobs that require “tech” interviews.
I have to encourage you not to give up on all tech interviews. (By that I think you mean "hands-on"/"practical" type tests).
I do hiring for our agency, which consists entirely of non-rock-stars, at non-rock-star pay, solving non-rock-star problems, in non-rock-star time. We still do practical tests for all interviews. Some people, actually, do just get up and walk out. I wish they'd at least look.
We do problems on the level of "fizz-buzz" or similar. Quick 10-20 minute problems. We don't make full completion a binary success/failure metric. We just want to hear you discuss your thought process and see that you're aware of variables, if statements, and for loops. Because, yeah, turns out some candidates who claim years of programming experience ... aren't.
But no tricks. And we also make sure the request is suitable to the resume (think about it as resume validation). So we wouldn't say "Python person, write a method for us in C#" or vice-versa. Smart people can easily cross-train themselves once the job starts. It's more about finding someone who has the fundamentals and can apply them.
In conclusion, I would defend the hands-on "tech" interview as necessary and not even evil.
You might not make full completion a binary success/failure metric but so many other companies do, that people assume any random tech interview will be like that. Its anecdotal but both my friends and I all have the same experience of making a single mistake and seeing the interviewer visibly check out of the rest of the interview, and only getting offers from places where you passed every aspect of their interview correctly.
I do very well on technical homework-type tests, but they still want to bullshit in the phone about culture, and be very exclusive (28 year-olds with beards) etc.
I know right? Interviewers say "the solution isn't important, we just want to understand your thought process" but if you don't get the solution dollars-to-donuts you're not getting the offer.
When I was a hiring manager, people came in for interviews, some of them with many years of experience on paper, who couldn't walk a linked list.
IMHO basic screens are needed.
I also want to see how a person handles discussing problems, I like writing out an API and discussing how to use it to solve real world problems, what are the holes in it, what are the implications of the choices the API author made?
I understand the problem they are facing, but that doesn’t change the fact that getting hired is a circus that I am not interested in. Just wanna write code, lots of it on bitbucket and github.
Well at least they all got replies. I have had at least 3 interviews where I did a take home exercise and didn't even get a reply. Unless I really want the job I refuse them now. I have a decent score (top 5%) on Stack Overflow for my tech of choice, so do I really need to waste an afternoon or more writing some mini application for you?
I'm with you on take homes. A large Seattle tech company had me sit in a hotel and do a take home after a full day of interviews and then after no reply for a week claimed they never got it. I re-forwarded them the original email.
Now that I'm older I refuse. Why? Because I work full-time, I have a family that I either care for, cook for, or put to bed. And when I'm not doing either of the above, I try to work-out for personal health and sanity reasons.
The take-home test only works in the "app does everything for you and you have no family" that defines living in SF.
Well, I'm in the top 1% for the tech I love on Stack Overflow. Does it change anything? No. Did I get any job offer because of that? No. Does it matter for the recruiters? Well, half of the recruiters I meet only want to know my salary requirement, and after getting that information they stop contacting me. Does the score help me in any salary negotiations? No.
And of course I get lot's of offers to take part in a multi level recruitment process, some of them last for about 2 months, with multiple meetings, and some take home tasks (often with requirements to spend a couple days on each).
And I simply don't have time for such recruitment style.
About a year ago, I interviewed with a couple of companies. One particular interview I thought went very well. They never got back to me so I kept doing interviews and eventually landed a job that I love.
About a month later that company finally got back to me and wanted to set up a final/offer discussion.
I told them that because they took so long, I wasn't interested.
I'm not going to wait around and hope you get back to me, my time is more valuable than that.
If no one follows up they don’t care enough. My time is valuable too.
When I was searching for a job earlier this year that bit one company. They apparently really liked me but didn’t call back until weeks later by which time I had already found something.
I was recently contacted by a company whos CEO had read some stuff that I'd written online and asked me to interview at their place. Their interview consisted of:
1. A logic test.
They sent me a bunch of logic questions unrelated to programming or my abilities as a programmer. I had a certain amount of time to answer and submit the questions. Sounds like something dreamed up by a HR droid with no knowledge of programming, but I went with it.
2. A programming test.
They sent me a list of instructions for a thing to build. I can't remember how long I was given, but it was something like a full days work, and I needed it.
I submitted the work. Heared back a few weeks later that they decided to hire somebody else. No conversation about the work whatsoever. I logged into skype today and noticed that the developer who sent me the instructions for the thing to build, tried to contact me 3 days later to ask where the work was. Even though earlier in the conversation I provided a link to the gitlab repo. Ultimately, I don't even know if they looked at my work. I don't know why they called it an "interview" seeing as I never actually held a conversation with anybody.
Next time a company contacts me because they think they might want to hire me, I'm going to demand an interview. If your interview is a series of tests, you can get lost.
I've been on both sides of the table and the reality is that high stakes full day interviews kinda suck - they're noisy and tuned for high false positives (if they're even tuned at all).
Sometimes people get lost solving a problem, or don't ask clarifying questions, or are just too nervous, drank too much coffee, or maybe just didn't get a full night's rest. For most processes - it usually takes more than one bad round but one bad round can rattle a candidate who can cascade.
I like to think i'm a decent interviewer (and interviewee - not particularly gifted at CS but I've run enough rounds to recognize most problem types) I've gotten offers from 'big 5' companies but have been rejected by some as well. One reasonably well respected tech company wouldn't even screen me. Just a vague rejection, 'not a good fit at this time'. Who knows why - resume went to a busy team lead who looked at it for 5 seconds and took a dislike to some word choice. Or maybe they were on a hiring freeze and the 'open positions' were just for show. Maybe the non-technical recruiter didn't recognize the technologies listed actually matched what they were looking for. Who knows. The process sucks. Everyone knows it sucks. Maybe 'of all the processes it sucks the least' Churchill-style. I think it's slowly getting better, people are getting more objective - less stupid brain teasers. As engineers we want things fixed fast (like a merged PR) but people driven processes move and change slowly.
My belief is most people would secretly like to hire (or see hired) people who are just a little less smart and/or a little less able than themselves. So this dynamic is often at play under the covers.
The exceptions to this are truly great managers and co-workers and companies that have a bright future but unfortunately it's kind of rare IMOP.
I really had this case where a 22yr old open-source contributing child-prodigy programmer would be rejected at the code-screening phase by a guy, let's call him "Jon". We had a phone call with three people: HR, Jon and me.
The reasons why Jon rejected the candidate were all a bit funny and I could not tell if Jon was serious or not. Also, Jon's Github contributions, pull requests and other things were rather shitty anyhow, but he was responsible for the first screenings so I had to listen to his feedback.
Jon pinpointed to some problems in the candidate's code which were more style and not problems. Other things had good reason (verbose try catch blocks). When I started to get defensive and mentioned that the quality of the submission is better than Jon's stuff on Github, HR stopped me and told me "We're not assessing John here"
2 and 3 are relevant and generally applicable, and basically boil down to:
2. HR had arbitrary unreasonable box-ticking requirements rather than assessing the candidate intelligently
3. HR was hopelessly disorganised
The first one however seems subjective. Maybe the candidate wasn't suitable for the role - the recruiter thinks he was the best frontender he'd found that year, the tech lead disagreed with his assessment. It does seem like the company HR handled things badly afterwards (incompetent HR is a bit of a theme here) but the original reason for rejection might have been valid.
In the first case, the tech lead discarded the candidate without looking at him/her seriously or interviewing, just by looking for a few minutes at the submitted piece of decent code and finding it somewhat unusual.
The article said that the interviewer looked for 10 minutes at the code. If it was really just a simple assignment and the candidate came up with a complex homemade framework then that is enough time to cast some very serious doubts.
1. As an engineer I would definitely not work with a team that comes up with the first reason for rejection (Provided that I have not mentioned that stack on my resume to begin with).
2. The second reason is more of a choice of the given company. There are companies on the other hand which give out an offer all based on cultural fit. More than often good interviewers never make a decision on the result of the puzzles but on how the candidate approaches it. Downside is that not all people are up for it in the interview hour even if they are really smart otherwise.
3. Third is bad luck (cause of disorganized recruiting). The delay in response has something to do with the candidate too, how much vulnerable they are for the job. Personally I discuss the expectations for the interview process beforehand which is super helpful since I almost always get the feedback the next day irrespective of whether its an offer or rejection.
On the flip side there should be another discussion on how engineers get the jobs. I am sure those stories would be interesting too. It is not about the technical competency for majority of the times. Interviewing is an art and the candidates who are able to make the interviewers feel good about themselves have much more chances of offers.
I've been rejected from Facebook at some moment. I assume that happened, because we had some sort of disagreement with the interviewer. That seemed like a system design question, and at some moment I suggested to use SHA256 hashes to identify unique arbitrary long human input sequences (no malicious intent assumed).
And he asked me what would I do if there's a collision. Estimating probability of such collision to under about 10^-40 over span of 30 years of everyone on Earth typing input for that system at max speed did not convince him there's no need to handle collisions, and he kept insisting on that question.
Well, I got hired by another big company here, on a more interesting project.
It seems to me that using those websites(stackoverflow, linkedin, whatever else) for landing in jobs is a big waste of time for prospective employees because of how messy and quirky the hiring process is. Most of the people I know that have been successful at being hired by big companies had some kind of (living, breathing) coach to walk through the process.
The signal-to-noise ration and linkedin is surely very low nowadays. I am actually working on a article describing what I learned hiring and firing freelance recruiters who work mainly with Linkedin.
Recently experienced No. 3 in form, was scheduled with the wrong person and the remote conference room was empty. Turns out the interviewer was another person in their own other room. Emailing the HR rep ten minutes in resolved the issue within five mins.
They might have, it's not entirely clear from that story.
The problem is - I've been here - there is only so much you can follow up.
If you follow up once, and they don't reply, then what? Try a 2nd time? A 3rd? If your unlucky and your first follow up was swallowed by whatever reason caused the original problem, do you keep trying and look pushy?
Or, you might have another prospect and althought you'd prefer the first one, you can't keep the second one waiting for ever so at some point you give up on the first and take the second. How many times do you give someone a chance to follow up before giving up?
I once had a boss interview, then a good tech interview, then for several weeks tried to get any response at all out of the boss for a follow up. I gave up and moved on. Several months later by chance I saw the boss at a tech event across the room, we nodded hi, and the next day he emailed me asking about the job. Dude, that was 2 months ago. I already had another job by then.
It's a good thing to do when you are in sales. Your entire job consists of contacting people. My job, as a programmer, is writing code. I will write and re-write code until it works so I understand you. However, contacting people is not my job. If somebody contacts me and then goes silent - it's just unprofessional in my view. Why would I want to deal with a company, which hires HR who are not able to respond to emails? For all I know they also hire accountants who are unable to cut paychecks.
1. Docusign, for not using Coffeescript, Angular, and Stylus (3 technologies I had never previously used) to solve their very long take-home test. I apparently provided a great solution, but they were concerned that I didn't follow those rules specifically. No, I'm not going to spend hours getting up to speed on your framework just for an interview, nor am I going to submit sub-standard solutions to to conform to your arbitrary restriction of using the framework du-jour.
2. Google, for not being able to, "prove a random number is truly random," during my onsite interview panel. The guy offered no hints when I wasn't sure where to start, he simply continued to repeat the question, a bit slower each time.
3. A local Seattle startup, who required 3 onsites: A technical, a culture, and then a full-day of "integration" (working on real problems with the team). I apparently passed the first 2, but was not invited to the 3rd. Turns out the HR director that was working with me quit before scheduling the 3rd. I never found out until after I had already accepted another position.