Hacker News new | past | comments | ask | show | jobs | submit login

I am sorry that you had a bad experience with us. Evaluation is a really complicated thing. The bar that we use for evaluating the take-home project is to treat it as real work, e.g. would a teammate feel good if you were working with them on this task, and you came back after half a day with this. Because we can't see process, all we can do on the take-home track is judge of the finished result is professional-level programming. We do indeed pass people who do not complete the project on the 1st call, but they need to be on a good path (where we think they can finish by the 2nd call).

I do not know who you are (and would of course not post details here), but from what you say, it sounds like we did not think that you were on track to finish, and has some concerns about the design that you selected (we track whether a number of milestones in the game have been reached). It is totally possible that we were wrong. We much prefer to see a working front-end / back-end combination that has missing features than we do just a front-end or just a back-end.

Again, I apologize for your bad experience. I hope we can make the take-home interview better in the future with some tweaks.

Just curious: are you making sure your interviewer are actually first trying to code the project in 3 hours before judging candidates? I mean, it seems to me that most of my colleagues (and myself) are always very optimistic with respect to "how long it will take". So if you are judging someone based on your expectation without having went through it yourself, it can lead to a perception gap.

This is a good question because I find that my colleagues from previous companies spent a lot of time thinking of 'good ' interview questions and leave it at that.

None of them actually tried solving it within the constrains that a candidate is put through.

I think that you almost have the right idea, but 3 hours is too long. I believe a programmer can demonstrate his ability to perform the basics in 1 hour or less. This respects the candidate's time and it also encourages the test's designers to select the most trivial project possible that shows the basic skills they're looking for. That's really what's important.

Larger projects take up more time and introduce a larger possibility that some matters of opinion or taste will impact the candidate's performance. I don't believe differences in taste are important as long as the candidate demonstrates that he is able to comply with a defined style guide.

The actual test is going to depend on the position at hand, but one of my favorites is a very simple program that asks the candidate to use GitHub's API to display the list of public repositories under a user-inputted username. That's it. I tell them they can use any language they want.

This simple test gives all the information we need about the basics:

a) the candidate is able to go online, provision himself an API key, find docs, and reference those docs to see an external vendor's API format

b) the candidate is able to use that information to craft a program that successfully interacts with the vendor's endpoint

c) the candidate is able to present the information in a concise, desirable manner.

d) the candidate is able to do all of this with a simple 1 paragraph description of the project.

The choices the candidate makes in the process of completing this simple task tell you a lot about his process, style, habits, and preferences, even though the project is very minimal in its actual requirements.

I've had people give me web apps, command-line apps, and GUI apps that accomplish this same goal. Many candidates would go above the requested specifications and many candidates would reply the same day they were given the test, which to me was an excellent signal that they felt their time was respected and that we were doing a good job of engaging them and making them interested in working for us.

As I stated, this test is not appropriate for all positions, but I think most tests should be modeled after those principles. Give the candidate room to express himself and demonstrate relevant practical knowledge.

I hope you'll consider a minimalist project like this over something like "design a multiplayer game ... in 3 hours".

As a wannabe junior rails developer, I really like the idea of a technical assessment like this. It seems very practical, yet it's not overly complex.

Have you considered requiring a thought process journal as well?

Evaluation is only a very small part of my complaint. The choice of projects that were offered, and the scope of the projects are what really need to be revisited (finishing the project wouldn't have been a problem for me if there existed a project that was relevant to my work that could be completed in 3 hours).

In terms of project choices, to me, it seems like you guys erred on the side of choosing projects developers would find interesting and challenging technically over projects that are practical and accurately represent the kinds of work most developers will actually be hired to do. I'm not going to post the details here for obvious reasons, but out of the four projects offered, only the multiplayer game was even remotely relevant to front-end/back-end or even application development in general, which are areas that probably account for the vast majority of development work available from startups. I realize there needs to be a balance to be stuck here, but in my humble opinion, as a recruiting firm, you should be erring on other, more the practical side in terms of project choice. I applied to Triplebyte to find a job, not to fulfill my intellectual curiosity (I can do that better on my own time without needing someone to assign projects to me).

In terms of project scope, I'm not really qualified to comment on the other choices, because they were way outside my area of expertise, but the multiplayer game definitely didn't feel like a 3 hour project. As suggested in another reply, I really hope you guys can actually give the project a try yourself and see what level of completion can reasonably be expected from three hours of work on something like that. Take whatever times reported by candidates who have successfully completed the project with a grain of salt, because people will have a tendency to understate the level of effort they spent to make themselves look more efficient (no matter how much you tell them you don't care). Here's another possible idea for making projects that take reasonable amounts of time to complete: just take your traditional interview questions and slightly extend them a bit with extra features, and simply expect better polish, architecture, test-coverage, and overall code quality, etc during the code review.

Anyways, my experience with Triplebyte's take-home interviews definitely didn't leave a great impression, but I still recommend you guys to my friends and colleagues because I do want to support what you guys are trying to do. Hopefully you can take some time to revisit some of the issues people have mentioned and make the necessary improvements. I'm happily employed now, but I'd love to give Triplebyte another try the next time I'm looking for work. =)

This guy has a great beard.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact