And the funny thing is, I work as an embedded developer and actually have to look up a topic or two from CLR at least once a year for work stuff (which is not very common). But DP? I've never needed it or used it. Why the emphasis in it then?
It feels a lot like software interviewing is just a bunch of rain dances and cargo culting and those that know the dance get through the hoop. I guess I'll have to order Elements of Rain Dancing from Amazon and get to practicing.
It gives both the employer and the candidate a chance to evaluate each other in context.
As an employer, you get to see how the candidate performs on two different real-world tasks, as well as how they interact with existing employees.
As an employee, you get to see what the company is really like to work with. No amount of whiteboarding can deliver that.
And at the end of the day, if that relationship feels good, both parties can choose to continue onwards towards full employment.
On the other hand, computer science trivia, by which I mean "implement Dijkstra or quicksort on a whiteboard", is a very poor means indeed by which to test for the ability to do real-world work in a real-world context.
Sure, it's valuable to have a broad understanding of algorithms and data structures. But whiteboard interviews on those types of problems only tests how well you would do competing in an ACM (Association for Computing Machinery) challenge, not how well you can deal with "here is a totally new-to-you problem that lacks a precise solution, now work productively with a team to solve it under the constraints of an operating business"
For the day-in-the-life, I got to work on real problems that I would be solving, in the manner of working that Pivotal espouses, with the colleagues I would be working with, in the environment that I would be working in.
I think that these interviews are particularly suited to Pivotal's model of working (in particular the pair-programming) and are probably less suitable elsewhere.
"We believe coding is a social activity. In fact, we’re testing you to see how social you can be around coding."
That makes the process less suitable for other types of working environments.
Probably activity is valued higher than progress.
They are certainly not looking for Knuth, who would likely fail the interview.
The other statements don't ring true for me, but I understand it's just your viewpoint.
But the generalisation is just stupid.
But I expect to go into interviews and have people demand a different rain dance, now that everyone is talking about DP. They like mugging people with 'so big it has to be out of core' stuff, which is a good way of finding out who went to the right grad school algorithms course. :-)
A cautionary tale about being overly clever-clever I suppose.
You are right about that, at least according to that article here:https://news.ycombinator.com/item?id=18942572
That guy trained on a well known website to solving minor coding issues and then (that's the important part) practised solving them in mock interviews to real persons.
Maybe that could help you.
You can practice all you want to get better, but no one in the corporate world is saying "I need you to code bubble sort, and I need in the next 45 minutes" and then proceeds to stand over your shoulder the entire time.
As for wasting a lot of time, what more reliable method is there that takes less time than a leetcode-style problem? Surely there are more reliable methods, they probably don't take 30 minutes or less though.
DP is the epitome of how absurd these interviews have become and now people are getting asked 2 of these questions in pre-assestment stage.
Jokes on them when they stop getting good devs and start getting good shortterm greedy puzzle solvers.
Hyuck hyuck hyuck
DP problems represent the next rung of the ladder, are somewhat less familiar. And most importantly, satisfy an interviewer’s possibly sadistic impulses by presenting a problem that has an easy 2^n solution. ;)
Sarcasm aside, I think this is just the inevitable result of the systemisation and gaming of the whiteboard interview.
Among the problems it solves are:
* Shortest path in a graph (aka Bellman-Ford's method) , A* path finding 
* Approximate string matching 
* Viterbi algorithm for hidden Markov models 
* Earley parser  for context free grammar parsing
* Dynamic time warping 
* Finding the base of a strong generating set for permutation groups 
I'm consistently surprised that DP crops up in so many places. While most people won't be required to implement the above list in their day-to-day programming life, it's quietly in the background as a fundamental algorithm.
Dynamic programming is one of the few algorithms that, in a "natural" way, reduces a problem that is naively exponential to one that is polynomial. For example, try counting where 2^T random walkers lands in an array starting from the middle after T time steps. Naively, this would require 'simulation' of 2^T but with DP you can do this in O(T^2) (assuming addition of atomic integer types is "free" and there's no overflow on count) using DP (by summing a 'count' of random walkers at each time step, at each bucket in the array).
I think they ask about it on interviews because it’s basucally a proxy for “did you get good grades in graduate level courses”.
At a top school, undergrad courses should be close enough. Of course, they could just look at your transcript, but maybe you cheated.
There are people who did only the multiple choice questions and moved on to the video conf interview. At least they can skip DP question for some people, so probably not an emphasis.
It's an IQ test.
It measures cognitive intelligence. Does it predict overall success? No.
For the dp problem, you need to have experience with dp problems, experience with general algorithm problems (from the acm-like programming challenge side specifically), and generally be good at applying those in new situations. Unless that's your actual previous work, it doesn't measure intelligence in any way.
Best know doesn't mean good. Any occupation also fails:
> for minimally-skilled activities, athletic strength (manual strength, speed, stamina, and coordination) are more likely to influence performance.
Also, that's a predictor for "any" job. You can do better if you know exactly what job you're interested in. Do a sample of work that's very close to the real one and you'll have a better prediction.
I’m an embedded developer too, and I end up reaching for DP every year or two. I think I probably use it _more_ because I’m in embedded: if I was willing to accept harder to analyze memory requirements and runtime performance, which I almost always am outside of embedded, most things I’ve solved with DP I could have more swiftly solved by composing algorithms and data structures from the standard library.
If a candidate can solve large, complex problems but bombs your interview, the problem is your interview. TripleByte should focus on this problem and educate these companies on effective interviewing techniques. For example, if your interview overemphasizes coding challenges, puzzles, quizzes, or otherwise feels more like being on Jeopardy than real-world applicable, you're doing it wrong.
Candidates should be focused on their skill set and filling in gaps (i.e. I've toyed with Python but I see it being in demand so let me get better at syntax, packages, thinking in pythonic way).
Maybe effective programmer hiring is just an unsolved problem.
I'm not defending whiteboarding the knapsack problem so much as I'm defending the difficulty of hiring skilled workers from different walks of life. Tech isn't special, hiring is just hard, and a lot of other industries are actually worse.
* You hire people who end up not being able to do the work (I hear my wife complain about this problem in accounting just about every three days)
* You end up needing more stringent certifications than exist in comp sci land (you mentioned lawyers, so that's one with bar exams, but also accounts with CPA exams, professional engineers, doctors, etc)
* You get proof of prior work
* You rely heavily on recommendations, with all the pros and cons there
Funnily enough, I've heard the same thing from some friends who worked at companies with algorithmic interviews.
> You end up needing more stringent certifications than exist in comp sci land
That sounds great. You just have to study up and take a very hard exam at the beginning of your career instead of having to study up and take a hard exam every few years.
> You get proof of prior work
And what about Github? Most employers want to see proof of prior work unless you've got a very good excuse.
> You rely heavily on recommendations, with all the pros and cons there
This may be the only decent argument against the accounting/engineering/medicine interview. Even with good performance, recommendations are a crap shoot (did you get along well w/your boss, are they pissed you're leaving, do they refuse to give out references to anyone like one contract employer I worked for for a year a long time ago).
But then again, puzzle interviews are also a crap shoot.
Have others had a different experience?
Related, in blog form: https://www.joelonsoftware.com/2006/09/06/finding-great-deve...
It's pretty terrible. There's no other profession that's quite as snobby as the legal profession. If you want to go work for a top firm you need to have gone to one of a handful of law schools. While there you need to have gotten good grades in your first years' classes so that when on campus recruiting happens at the start of the second year you get an interview. These OCI interviews are almost entirely what the tech industry would call culture fit, with all the implications of bias that implies. One a summer position is obtained, the job is yours to lose. Do reasonable work, don't get drunk and vomit on a partner's spouse or blow off your third year classes and you will probably get a job offer with the firm you summered for.
I don't think this is a process any other field should try to emulate. Whiteboard interviews may have their flaw but it's a lot better than:
Did you go to Harvard Law &&
Did you get top 1/3 grades as a 1L &&
Do you remind the partner you are interviewing with of himself and/or his kid
That said, I think all interviews have an element of silliness to them. I know a financial consultant who was quizzed over articles published in the days financial times newspaper. Also know one of our analysts makes candidates do long division on paper! Don't forget the infamous how many golf balls can you fit in a jumbo jet kind of thing.
Friends in more general office jobs like HR, managers, etc, it doesn’t seem much better. It’s different than how to get a software dev job, but to say it’s been better would be a bit much.
This approach is straightforward, real-world, and you can find out about a person this way in as little as 15 minutes.
For code, the key is to find a recent problem your team's have recently had to actually solve. Distill it down. As a interview question designer, give yourself 1/4 to 1/3 the coding interview time and see how far you get implementing the solution. That is the bar: don't expect a candidate to be able to get further than that in the whole time block. You also want to budget time to talk about productionizing the code (metrics, logging, monitoring, etc).
I had a question that I wanted to give and required a couple passing unit tests. After applying the 1/3 time rule, I realized that my question was unfair and I had to alter what I started with.
I've come to the conclusion that one (or both) of the following might be best:
- a small take home project with a reasonable due date (say 5-8 days) that is representative of the kinds of work that is expected on the job. Then, a 1-2 hour code walk through and presentation. Candidate should expect to build, run, debug and describe the code and architecture, tools choices, show source control use etc.
- a 3 month probationary period with a fairly narrow starting role that all new hires expected to be able to code are required to go through; ex: fixing bugs on project X; ramp-up and demonstrate _______ and _______ on project Y.
Doing the take-home project is certainly difficult for some candidates who may have a hard time carving out time to do it - but the hiring manager can offer some flexibility here - and this seems like a much better approach than taking online code quizzes.
The 3 month probationary period seems like a way to detect other on the job performance or interpersonal skills issues that may be a cause for disqualification.
The problem with this is that talented developers generally do not want to work for free on someone else's proprietary software and there is a very real risk of companies trying to abuse this practice to extract free labour from candidates the closer you get to "work that is representative of the kinds of work that is expected."
I do not want to spend hours of my life on unpaid labour for a company that may or may not hire me. Give me the 45-minute algorithmic challenge that puts me on the spot every time.
I also strongly suspect your 3 month probationary period would be an effective anti-signal as well, useful for attracting desperate candidates rather than the ones you really want. You can't reasonably expect a senior engineer to jump at leaving a secure position for a 50/50 shot they might be unemployed in 3 months because they're "not a good cultural fit."
Financial security is important, and I would put up with a lot of shit from my company before I'd consider taking a chance on another one like that.
Although I know some companies (my current one included) have a 3 month "probationary period" where they've only ever let 1 person go at the end out of approximately 150 since I've been there. It's just to make sure you're not a complete goon.
Problem is, it could be really hard to tell apart a company that has a "probationary period" like mine, or a real time that they judge you at the end.
This may happen in some places, but in practice I've never seen a task which actually could be applied anywhere. Most of the time these were toy projects, or simple exercises.
I think you'd easily notice if giving you the test was worth more to the company than all the recruitment time spent on taking to you. It's not free labour if you end up spending hours with each candidate, splitting up real work into "exercise chunks", integrating wildly different styles, validating results, etc.
Especially in the US, asking an employee to give up even more rights is a hard bargain. You run the risk of naturally selecting for employees that can't get hired anywhere else (which tend to either worse, or at the very least require more investment; something you're not willing to do or why have you made them use their time to do a take home and start this contract to hire business)
In my appraisal, more firms just need to recognize that they don't need half the technical skill they think they do. Building a CRUD back end or a mobile app doesn't take deep algorithmic horse power and the engineers you've managed to retain to administer the interview probably aren't qualified to do so anyway.
Also, I find that take home projects in particular are biased against people without free time, eg, parents.
might work at one place, but the next will ask completely different questions. You can’t know everything.
Google said I should try contributing to an open source project to improve my resume... when I had literally a million lines of code in a major open source project, and that was written on my resume.
Was it a mistake? Was there a mixup and had they interviewed me thinking I was someone else? Did they have a corrupt copy of my resume? More likely was it a throwaway comment from someone just saying something to be polite? No idea, but it's more frustrating than just 'no thanks' because it didn't even make any sense.
Imagine if you tried to improve yourself based on that feedback, applied again, and then they overlooked it again and gave you the same feedback!
This interview system is frustrating. I had 4 calls where I talked to the same recruiter thrice for this company I'm applying to. It doesn't sound like much but when you're in my situation wasting a few minutes is precious time. I'm driving 3 hours down and back to a full-day interview at said company. I had to take PTO. What do you tell your boss? I told em I had a drs appointment, luckily they bought it. I don't want to get harassed for looking for another job, it happens here.
Does it really take a full day to find out whether someone is a good programmer? I'd argue it takes longer. We just hired someone who washed out after 2 weeks without finishing a (significant) ticket.
Maybe instead of these nutty run-the-gauntlet interviews we should focus more on work-to-hire month long programs, with some guaranteed compensation if you fail(so that you are compensated for the risk of switching). I don't know. The current system is stupid.
I have gotten offers from a few major companies. But cramming this stupid Interview shit (What's the difference between an abstract class and an interface in Java?) when I've been running Golang, Python and Ruby for the past 3 years isn't fun. I should stop writing this comment and get back to studying.
If you think you got shitty candidates with your old system, this would only get you desperate ones.
Feels like a very similar occurrence to what happened to the homebrew dev who got reject by google (albeit different circumstance to your false feedback)
There's a non-zero chance that the Google interviewers were able to pick up on this. Those of us not directly involved only ever got one side of this particular story.
Community service projects? I maintain a bibliography I have on there, and I write a local history book on my GitHub. I made a website for a conference on there recently.
Slides, notes and code from conference talks you've given?
Hobbies? I used to have some repositories with things related to the sport I play.
Open source projects you contribute to in the course of your work?
Notes and experiments you make while learning about technology like new languages and libraries?
You must have done at least something like those ideas at some point in your life?
I mean you obviously don't have to if you don't want to and I don't know anything about your personal situation, but if you do want to have a GitHub profile I'm sure there's tons of things you're involved in in your life that you could upload even if you work at the NSA or something.
It is, I wouldn't entertain companies that do this unless you really need a position.
This is the trifecta of bad interviewing tests:
- No compensation
- Stupidly restrictive time limit
- High chance of not getting feedback
Personally I would turn them down and then explain these three things. There are plenty of companies with less annoying hoops.
This one kills the chance of my applying by itself. Turns what could be a interesting coding challenge into an exam, ruining the fun of it. I could see junior devs going through this to get some experience but why would anyone past that point in their career bother when there are so many options?
Curious if anybody has ever billed the interviewing company for their time if they didn't get the job.
But in a market like the current one, many developers can do some filtering of their own. When I'm asked to complete a test or exercise, over time I've made it a habit to ask questions like these:
- Do you have reference solutions that someone at the company completed on company time?
- Exactly how long did it take them?
- Do you have a rubric for evaluating submitted solutions?
- Will you share the reference solutions and/or rubric once I've submitted mine?
I don't need 100% positive/detailed answers to each one of these -- I know from the other side why they're not always there -- but how people respond seems to improve my early feel for (a) how the organization approaches process and problem-solving and (b) whether my solutions are likely to get a serious examination and useful feedback, which I can then use to inform my choice about whether putting in the time is worth it.
What frustrated me is that my solution passed all of the disclosed scenarios, but failed about half of the non disclosed. Neither the input nor output was disclosed. How do you expect someone to solve a problem when you dont disclose either the inputs or the expected output?
“Do you have what it takes?”
“Do think you’re tough enough!?”
“You call yourself a....”
“Then... SORT... MY... LIST... WHILE... TALKING... OUT... LOUD”
But really, I feel like the whole process is uniquely American, kind of like crossfit. Very intense very trying. Like Nike; “JUST DO IT!”
The problem about this very intense “game” of interviewing is it’s for a job, for you to eat, to afford healthcare, to survive. To afford basic necessities and save for retirement. To buy a house. To start a family. To afford a car. Or even simply to not be just another body stepped over on Market street.
You've probably enjoyed 7 some years as a web developer, as a productive team member or consulting for happy clients. Now some stranger is dangling a carrot in front of your face telling you to implement an LRU cache under bright fluorescent lights while badgering you with “what are you thinking, I need to know what you’re thinking” ... after you’ve just spent the last 5 hours talking about your work history etc
I work for a manufacturing company and my application space has nothing to do with manufacturing. Every space has their challenges and pitfalls. As long as you have good tech, good people, and aren't doing anything unethical, I'll want to work for you.
Also, I've thought about the interviewing problem a bit. I don't do the interview for my team...but if I have the opportunity to interview and was given free range, I think I would create a small application using our tech stack. Put some bugs in there, put some bad practices, and let the interviewee fix it. Talk about the tech stack, how to scale it, design trade offs, etc. IMHO reading code and maintaining code is more important than knowing bfs and dfs by heart.
Plus, at the end, at least you have a project you've done.
Not a situation for /most/ of us on a regular basis.
Find a good project and make it work well. By this process you will hopefully uncover and learn practical details, skills, programming language, how to write high quality code...
And learn Computer Science (algorithms, structures) to show that you are smart and can learn stuff and when given a problem you can solve it efficiently.
- we definitely check if you have something interesting on github
- we ask both computer Science questions and then some more practical questions (e.g. if you are Web FE, then rendering some interesting situation, maybe optimizing page speed...)
And I think it makes sense too. When I conduct interviews, I look more for how is this person thinking about the problem, than can they solve the question in isolation in the allotted time. Because unlike leet, it's not in isolation. There's always some back and forth, and that's where I think leet's shallowness shows.
It probably isn't brain stretching.
The under duress part is the dumbest part of coding interviews.
I understand the approximation of small coding challenges, why explaining your thoughts is beneficial, as is understanding how a candidate approachs problem solving.
Causing duress to a candidate purposefully has no value what-so-ever in the workplace.
That said, there probably is a reason for testing candidates under duress. Some companies may want to push people hard, and knowing they can deal with stress is valuable.
Anecdotally, programmers thrive in quiet environments where they can have focused uninterrupted time. As open-offices have become the norm, most programmers have headphones they use to isolate background sounds.
Constantly peppering candidates with questions while they are trying to do a task that is mentally intensive is not close to the type of stress candidates face on the job.
Even the most work-life balanced places do these same types of interviews.
I have no faith in TripleByte or their process at this point, especially since I feel I was somewhat taken advantage of (I did the online quiz expecting to get some feedback -- isn't that the point?). I'm certain they will use the collected data from my quiz to build their product, though.
I did the 2hr phone interview later and received my interviewer's notes on my performance, which were more detailed and helpful.
So of course let's talk about a dozen different types of them!
Welcome to Ford! To be considered for this position, please list every GM Model from 1983 to 1987.
What I do is start the Linux program "recordmydesktop", and then go about solving the problem as if I am interviewing, i.e. on the clock.
One thing I noticed is these programs use maps, characters, use 64 bit long numbers, do string manipulation, do certain type conversions and so on more than I usually do programming. Some of these things I would usually look up, such as how to type convert a character '7' to an integer, or create and populate a multi-array and things like that. So I made sure how to memorize how to do these test-tending things.
Then I watched over my recordmydesktop sessions to see where I ran into trouble. I noticed array one-offs were a problem - getting an index out of bound error, using < instead of <= etc. Or I would declare a variable before a loop and use it initially, but forget to assign or reset it at the end of the loop when it needed that. Sometimes I would declare a variable like k and then re-use it in the same method. Also, if I ever have to switch to a web browser to look up what call does a string reversal, I note that as well, and if it is common enough, I memorize how it is done so I don't have to look it up again.
Memorizing the string manipulation, map etc. functions is already working for me. I am getting better at some of the other problems, like not assigning/resetting variables at the end of loops (I put a comment next to them that they will need to be assigned/reset, and then do it later, and then erase the comment). Some I still have to work on, like reasoning about exactly where the array boundary is. These aren't things I mess up necessarily, but things that slow me down, and cause bad compiles or runtime errors.
I can already see the improvement in the problems I am doing. I had been letting myself naturally memorize some of these things, I guess it's not so bad that I am memorizing more of them that I may need.
basic exercise was in php (mix of systems, but there's existing php to touch). financial service industry, and they'd mocked up a rough idea of a typical problem. The example code clearly wasn't production, and was generic, but also illustrated the problem domain well enough for them to get an idea of how well I could adapt.
I got through the 'nut' of the problem in about 10-15 minutes, talking through some issues. One thing I'd ignored (and said I'd be ignoring) was date handling - they wanted some date sorting thing, but since this was just 'raw' code without any good helper libraries, I suggested we punt on that (and I named the libraries I'd typically use for date handling).
There was one spot where the interviewer suggested the ?? 'newer' null coalescing, where I'd been looking at doing some 'array_key_exists' checking. he'd suggested that the ?? syntax was easier and more compact, and I'd countered with I'd prefer to be more explicit about the specific keys I wanted to check for, and that ideally I'd be using a class or some other more defined structure to ensure some degree of valid data shape as a first level of sanity checking. I'd felt I may have pushed back a bit hard on that point, but, we moved on.
After the first 15 min or so, we'd spent another 15-20 talking about different ways to do other types of error handling, some potential performance issues, etc. High level stuff, mainly, which... there's only so much you can do with coderpad (my jetbrains and vim muscle memory did me no good in the browser!)
All in all, though, I'd felt it was a decent exchange. They were able to show a 'real' problem without relying on any proprietary stuff, it was difficult enough that someone junior probably would have struggled a bit more, the guy on the other end was pleasant, and engaged appropriately (responded to questions and casual banter, and interjected now and then to ask questions and give feedback in real time). It was probably one of the better 'tech interviews' I've had (although I haven't had many in recent years, so, hard to compare). It did seem better than many I read about, and certainly better than some from 10-15 years ago.
I suggest this type of interview:
Both the interviewer and interviewee get a problem to solve and they work together to solve it. It should be new to both of them, so the interviewer can better assess its level of difficulty without the bias of knowing the solution beforehand. This will enable natural empathy for the candidate.
It also shows how interaction would look like in every day kind of work. You can learn a lot about another person's skill by doing pair programming.
Logistically speaking, you can whiteboard the design/code, or sit around a big screen for the coding part, with whatever tools/IDE the candidate likes to use (s/he should be writing most of the code, even if the solution/design was a common process).
Sometimes when I read people saying the end goal is not to solve the problem weird. Does that mean those who solved the problem alone by losing a few hairs actually scored worse than an outspoken candidate who solved it with help from the interviewer?
I think actually solving those problems or failing in the end is random (unless you know the answer already). But if you get a good chat out of it, it may be in your favour.
I thought the interview seemed quite nice. Is this the type of coding interviews, HN is always complaining about, or is it some rosy best-case exhibition?
Example interview: https://youtu.be/tj_sBZw9nS0
Is this really true? Also, how do you just mention offers? And will the programmer interviewing me really care? Should this only be done at the HR stage?
Like, what's a good way to let the other person know that they're not the only one.
I'd never thought to interview on some of the topics presented until I took the quiz. Worthwhile Saturday evening accompanied by a beer or two.
Not a company that has things together to say the least.
1. Review the basics. Even experienced programmers and engineers can get tripped up when questions about algorithms and data structure come up. If you haven't yet mastered the basics, focus on this for a few weeks or months to get yourself ready to tackle the bigger problems. It's crucial, not only for a coding interview, but for the entirety of your professional career, that you understand the basics. They are your foundational skills upon which you will build and master your expertise.
2. Solve coding problems and work on projects. Online coding practice problems are a great place to start. Apply your knowledge and work on projects you enjoy. Write code. Build things. Put your skills to use. This should be the fun part - if you aren't enjoying the journey and the process of building your coding skills, you'll have a difficult career as a programmer. Sure, there will be moments of frustration where you'll want to smash your computer screen and throw your mouse at the wall. But forge on. Keep trying. Find those bugs and destroy them. Build great things, and immerse yourself in online forums and other communities where you can turn to for help when you really get stuck.
3. Use online coding interview platforms for practice. This is a great way to get exposure to how the real coding interview will feel. Smash your nerves and anxiety by conducting a full coding interview with a partner. On platforms like Pramp and Leetcode, you'll get paired with another programmer of a similar skill level, so it will scale to your ability. Together you'll experience the coding interview from both perspectives. Performing the role of the interviewer and the interviewee will teach you a lot. After the mock run, you'll both leave each other feedback, and this is a great thing to have to tailor your practice schedule and for identifying what you need to work on the most.