11 ms·
Guide to Take-home Coding Challenges
- booleandilemma 9y agoPlease don’t normalize “take-home coding challenges”.
- closeparen 9y agoAt least make them instead of, rather than in addition to, whiteboard coding sessions.
- Clubber 9y agoHow about neither. Should be: talk to the guy, talk about prior work. If you think they fit, hire them. If they end up being bad, fire them.
- toast0 9y agoI've worked with and interviewed enough people who can talk a good talk and can't code their way out of a paper bag. I would rather figure that out before investing in onboarding someone.
- Clubber 9y agoThis is going to sound worse than I intend it to, but If you can't tell when a programmer is blowing smoke up your ass during an interview, you are either asking the wrong questions or you need to have someone else in there with you. Interviews, technical or not is a psychological process. Ask them about their previous projects, ask them what their favorite projects were and why. Ask them what their least favorite projects were and why. Ask them what the hardest thing they've worked on to date was and why.
- pkaye 9y agoI would have to agree with parent post though. There are always people who can talk they way through well but can't code. Its actually amazing to watch the real pros at this deflect all the tough questions. It is a great talent for something likes PR, sales or marketing but it might not be programming.
- Clubber 9y agoI've only hired one person like that and it was my fault. I was bedazzled by his color printouts of his UI. It was probably fine, but he was super lazy.
- pkaye 9y agoHow many programmers have you hired using this method? How many have you fired?
- Consultant32452 9y agoI worked at a place once where they got so tired of trying to figure out which interview practices were best that they just started telling the recruiting firm to send people in for their first day of work without even bothering to interview them. This was a big local contact, so the recruiting firm wasn't going to send in known duds. The results worked out about the same as when we had actually done our traditional interview process.
- Clubber 9y agoI don't understand why people put up with all the bullshit that goes into tech hiring these days. I mean are companies that offer "market rates," putting people through this? No wonder they can't find anyone good.
- eee6e6e666ee 9y agoIf there is a guide or a book for taking an interview, the interview has already failed and is measuring the wrong thing.
- Arainach 9y agoI disagree. Even if you had a hypothetical perfect interview that tested exactly the right combination of raw logical ability/concept familiarity/effective communication, there would always be room for a guide that reminded people exactly how to best prepare. Interviews need to test for certain characteristics in a limited amount of time. Even if there was some way to say "here's the code I've wrote, e-mails I've sent, and videos of me leading team meetings in the last year", no company would have the time to look at those materials. As such, we're stuck with the common question formats trying to determine if you communicate with your interviewer well (understanding what's being asked, explaining your thought process and what you're trying to accomplish, talking about anecdotes and building rapport) while also showing off some technical understanding of implementation and design. No one's yet found a better system that's more respectful of both the company's time and the candidate's time. These "take home projects" are an interesting variation that I'm not going to dismiss out of hand, but it's hard enough to take 8 hours out of my day for an interview loop - adding more hours in the evenings isn't something that scales well to interviewing with a bunch of places.
- PeptoKelpto 9y agoI think it's awesome that you can become a software engineer without any engineering schooling. Where I live you have to take software engineering in University to be a software engineer. Bootcamps and certs usually only get you dev jobs.
- caymanjim 9y agoMany companies use the terms "software engineer" and "developer" and "programmer" and "analyst" interchangeably. There's no universal agreement on the difference. I'm not saying it should be this way, but it is.
- indemnity 9y agoIn my market, a title of software engineer generally means higher compensation. Even though most of us are not real engineers who studied BEs, the title has more prestige than developer (or god forbid, programmer).
- booleandilemma 9y agoIt’s a title arms race and that’s how it’s going to be until it’s illegal to call yourself an engineer w/o an engineering degree, such as in Texas and Canada.
- Clubber 9y agoThey're all meaningless.
- songzme 9y agoGood article, but I personally hate Take-Home coding challenge: Some people spend an entire week on the coding challenge, which really set the bar unrealistically high. Once I turned in a project 2 hours after receiving the assignment and got rejected because they think I didn't even try. Every functionality worked flawlessly, but they want tests and for me to treat it like a real production work. But how do you motivate yourself to treat such a trivial take-home (which will never be maintained because its a take-home assignment) with the perfection you would in production code? In production you have much better insights like how your component impact others, how users will be using your app, etc.
- PeptoKelpto 9y agoThis is my gripe with take homes. I have spent a week on a take home before and got glowing reviews on how awesome the tests were etc. only to be rejected down the line without any feedback whatsoever. If I apply for a job now and they give me a takehome I kindly ask for an in person interview or decline to proceed with the process.
- janephilipps 9y agoThis is tough - I've had similar experiences. I think the main issue here is that the company didn't give you any feedback, so it's hard to know what happened. I've submitted a take-home challenge, but had hiring needs change in the time it took me to complete it, so the company passed. Though I think there are a ton of things you can do to make your self stand out as a candidate with a take-home challenge vs. a whiteboarding interview, they are not for everyone.
- cyberpunk0 9y agoMore and more it is becoming "standard policy" to give zero feedback. It's quite pathetic to be honest
- kasey_junk 9y agoI have fought the “zero feedback” policies at companies going back ~12 years now. I think it’s terrible but I always lose. But here is why. For every personal story you have of working with a dev that can’t program, your recruiting staff has 3 of bad/lawyer invoking instances of dealing with crazy candidates. They are being honest with you when they tell you any other policy is too expensive.
- rboyd 9y agodon't do unpaid take-home coding challenges. particularly if you have no idea what the company is offering for compensation should you get the offer.
- olskool 9y agoAgree
- BurningFrog 9y agoDisagree
- janephilipps 9y agoFor a well established developer, this may be true, but for newer developers, they often have to jump through a lot of hoops to prove their skills first, especially if they come from a non-traditional background and do not have a CS degree. Do you think this issue is less pronounced with the move toward companies providing compensation guidelines up front?
- PeptoKelpto 9y agoAgree
- qudat 9y agoI only agree so far as the take-home challenge has no time restrictions. I would choose a take-home coding challenge over any white boarding challenges.
- thesmallestcat 9y agoThere's a good chance somebody on the other end's going to run your program without really vetting it. Especially if you include a convenient "test runner." Not a bad attack vector, just get your alias past the first screen and you might get arbitrary execution inside their firewall on a host with access keys and the like.
- dawnerd 9y agoI dunno, I’ve just walked from interviews that want take home tests. Plenty of other offers out there that don’t require it. If my Github and past experience are not enough then we’re not a match. Companies are going to put you on a probationary period anyways so if they don’t like your work they’d fire you then.
- janephilipps 9y agoSure - as someone who transitioned into development from another industry, I could only rely on my past experience for transferrable skills, and not my actual technical skills. While Github is great, my Github is not always an accurate reflection of where I am technically as I continue learning.
- Clubber 9y agoJane, It's not a reflection on your or the article, it's more of a reflection of how silly tech interviews are these days.
- qudat 9y agoThat might work for you, but there are a ton of applicants who are competent but do not have a great github profile. If you were hiring someone would you ignore all candidates who have an empty github? If so, what happens if you never find someone with a good profile? These are the problems we deal with when hiring. Furthermore, for entry level positions -- candidates right out of college -- how do you measure competency?
- dawnerd 9y agoI don’t think there’s any excuse why someone wouldn’t have something to show that they’ve built on their own. If they have nothing and no past experience I don’t know why’d they’d even make it to an interview.
- gtirloni 9y agoTime suggestions are usualy completely off the mark. Four hours turn into 20 for anything resembling production code (which is what companies are interested in). I like the idea of take home projects but it shouldn't come right after a phone screen or even the first real interview. More like as close to the last step as possible. And then there's the question of what other professions ask candidates to work X hours for free. Let's assume our field is too special and not get too deep into the rabbit role ;)
- deepGem 9y agoI can vouch for this. A simple 2 hour script turned into a 2 day affair since production quality code was expected. Thankfully the assignment was spread over two weeks.
- janephilipps 9y agoI agree that time suggestions are usually way off, which can be a positive and a negative. On one hand, you can spend a ton of time to go above and beyond, while on the other hand, you may have other priorities. I think if you're really excited about a job or company, you can show that by putting a lot of effort in.
- davidbanham 9y agoThat inherently biases the interview process away from people that have additional commitments in life. You're going to end up preferentially treating those that are currently unemployed versus people that have jobs, for example. Also, you're likely to be homogenizing your applicant base. Expect to see many more young people and recent grads versus people with kids.
- janephilipps 9y agoI agree with your point. I'll add that for new developers and those without the traditional CS background (both of whom are my target audience for this guide) spending the extra time is a learning experience for them that can make them better developers. Ultimately, it's not the applicant's choice what the parameters of the interview process are (I commend you for working to make it better) - given that, why shouldn't an applicant try to give themselves an edge in the process?
- q3k 9y agoIf you're thinking of interviewing your candidates with such methods, it's worth noting that their current employment agreement might prohibit them from participating due to IP clauses (more exactly, that they require permission to release any code that they've written, even in their spare time). This can restrict your pool of candidates quite dramatically, as most US companies seem to have their engineers sign such agreements.
- fizx 9y agoA take-home coding challenge will bias your pool of applicants towards relatively junior overachievers who are barely skilled enough to complete your challenge. More senior and skilled people will decline to participate, because they know they can get a better return using the time otherwise spent on the coding challenge to network for more and easier opportunities. So now you've done a great job of raising the floor of your interviews for little direct cost, but you've also lowered the ceiling.
- Clubber 9y ago>More senior and skilled people will decline to participate, because they know they can get a better return using the time otherwise spent on the coding challenge to network for more and easier opportunities. Absolutely this. No top would put up with this unless the company is paying an ungodly sum of money.
- Consultant32452 9y agoI agree with you, maybe it's because I spent so much time consulting. My small project rate is $125/hr. And I've usually got someone lined up who is willing to pay me for whatever free time I can offer. The potential return on your free coding puzzle game would have to be pretty insane to get me to give up real guaranteed money. A better option might be to just hire me as a consultant for some real ticket of yours and let me work it in off hours. Then we both win.
- Clubber 9y agoI haven't gone through the traditional interview process since 1999, and that's just because I was only a few years in and wanted more money. Every job I've ever gotten since then has been from someone I've worked with before. They tell their boss, they take me out to lunch and it's a done deal.
- on_and_off 9y agoI got my last 2 jobs after take home challenges (and I am a senior engineer). The last take home was a 6 hours exercise. I did not really see it at an huge opportunity cost. I would have trained 10 times that for a whiteboard entirely unrelated to my day to day job.
- PacifyFish 9y agoThere's a lot of contempt for take-home coding tests in this thread. They aren't a perfect solution by any means (1), but much closer to an engineer's real responsibilities than a whiteboard session. For this reason and this reason alone, I will continue to advocate for take-home coding challenges. (1) IMHO, the best practical solution is a pair-programming interview using a real bug or feature the company has already solved.
- Clubber 9y agoNo. Talk to them, get a feel, if they are a fit, hire them for a 3 month contract. If they aren't any good, don't hire them. If they are really bad, cut the contract short. You will never know what someone is capable of in a few hours. People who think that works are just fooling themselves and turning away good talent.
- praneshp 9y ago> You will never know what someone is capable of in a few hours. I guess the goal is to fine tune your interviewing process based on how past candidates turned out.
- Clubber 9y agoI'd bet dollars to doughnuts the people doing this are collecting zero data to analyze to see if this new process improves or hurts the number of good candidates they actually hire. The irony of it is putting processes like this is actually turning away good talent, but they wouldn't know, because they aren't doing any analysis; they just read an article somewhere.
- toast0 9y agoYou can certainly do analysis on the hires you make, but it's pretty hard to analyze the hires you didn't make. Very few will come back and reinterview.
- kirbypineapple 9y agoI failed a take home challenge from Netflix and was a sad with how it went down. I was given 24 hours to complete the assignment and told that I 1.) shouldn't try to be clever, and 2.) that it shouldn't take more me more than 2 to 3 hours to complete. Early on I realized that the only way to get optimal performance was to use an interval tree to store my data, but thought that an interval tree was being "too clever". I implemented a more naive solution and submitted it with comments indicating that I believed that an interval tree was the optimal data structure to use in this case. The response was that an interval tree was the correct data structure to use in order to pass the performance component of the assignment and that they were not interested in continuing with me any further. In hindsight I should have reached out to validate whether an interval tree was "too clever", and I think that was a more important factor in my failure than anything.
- DelTaco 9y agoMan that makes me so upset for you
- janephilipps 9y agoWow, so sorry to hear this! This seems to test nothing other than what your interpretation of "clever" is. I've definitely learned that it never hurts to ask questions in these situations.
- 220V_USKettle 9y agoOnce you are hired, will your coworkers at Netflix communicate better with you?
- davidbanham 9y agoI think the only humane way to administer a take-home coding challenge is with an enforced time limit. That way everyone is on a level playing field and you're not advantaging those candidates that happen to have oodles of free time. The author raises that point: https://www.fullstackinterviewing.com/2018/02/02/the-ultimate-guide-to-kicking-ass-on-take-home-coding-challenges.html#2-take-home-challenges-can-be-very-time-consuming https://www.fullstackinterviewing.com/2018/02/02/the-ultimat... But takes the wrong view on it in my opinion. The answer simply can't be "Just spend as long as it takes". That point is the thing that led me to build a tool that enforces time limits on challenges via a custom git server. I used it to interview around 100 candidates at a company I was running at the time and it was an excellent approach. I've since productised it as https://takehome.io/ https://takehome.io/ If you want the benefits that take-home coding challenges can bring to your hiring process, but are concerned about treating your candidates humanely, I urge you to check it out.
- janephilipps 9y agoThis is really cool, thanks for sharing! I don't necessarily disagree with your point, though I do think an unrealistic time limit is even worse than no time limit. I also emphasize the importance of not taking on a challenge that is outside the scope of your ability here: https://www.fullstackinterviewing.com/2018/02/02/the-ultimate-guide-to-kicking-ass-on-take-home-coding-challenges.html#1-taking-on-an-assignment-when-you-arent-quite-prepared-or-dont-quite-have-the-skillset-for-it https://www.fullstackinterviewing.com/2018/02/02/the-ultimat... What are your thoughts on timed coding problems like Codility or Hackerrank?
- davidbanham 9y agoThank-you! My contention is that it doesn't matter if the time limit is unrealistic (within reason) so long as it's consistent. ie: the challenge is not "Show me the best solution to this problem" but rather "Show me the best solution to this problem that you can achieve within an hour". As long as everyone is under the same conditions, the playing field is level. It's part of a holistic approach, too. The coding challenge is not the only part of the interview process. I suggest that there be an initial interview to assess experience, fit, explain the job, etc. Then if everyone is still interested, the coding challenge. The candidate's response to the challenge then forms the basis for a discussion in another, technically focused interview. It's not about saying "You gave the wrong answer" but "Talk me through why you solved it this way" and "If this criteria in the challenge changed, tell me about how that might change your response" The Codility and Hackerrank style problems are okay, but I don't like how constrained they are. I also think it's pointless having someone outside your organisation grade the results. What matters is not having a candidate who is "the best hacker" but having someone who fits what you need. If your team values robustness and debuggability above all else, you _don't_ want someone who can punch out 3 kinda-right prototypes before lunch. If you're an ideation agency, the inverse is true. That's why takehome encourages companies to create and administer their own challenges rather than the samples. They should fit the kinds of problems your organisation faces, not contrived ones.
- busterarm 9y agoIn general I find them to be completely unrealistic if you're trying to find anyone good. Someone really good will already have a full time job, a limited number of "sick days" to use for interviewing and is probably somewhere in the interview process with at least 3 other companies -- probably closer to 5 or 7. Then they might have family responsibilities on top of that. I budget 4, maybe 6 hours if I really like you, on any company in the application process -- tops. Almost 100% of this time is taken up by interviews. You want me to give your company a whole weekend of my time on some chance that you might hire me? In addition to hours of interviews? That already tells me that I don't want to work for you -- especially if I haven't at least interviewed with 3 technical people already. It's one of the clearest negative signals I know of. If I work 13 days in a row without a day off, I'm going to be a crabby asshole at my job and probably blow all of my other interviews until I recuperate. Your company isn't worth that.
- busterarm 9y agoJust for some color: The only company that's given me a weekend challenge that ever gave me an offer was 8th Light. I've done live coding and whiteboarding with other companies and received offers, but the take home work has only been productive that one time. I've had code I submitted to these looked over by better programmers than myself and heard very few negative comments, all minor. After some years of experience, I can't see myself doing one of these again pretty much ever.
- djtriptych 9y agoThis is a little silly. If you’re “really good” you may be talking about signing on for a 4-5 expected commitment at $1M+ Total compensation over the course of your tenure. Million dollar contracts require due diligence. 4-5 man days between both companies is totally reasonable. If you’re interviewing with 7 (!) companies at a time, maybe be a little more discerning about where you’re trying to work. Everybody wins if you’re specific on that point.
- busterarm 9y ago> If you’re interviewing with 7 (!) companies at a time, maybe be a little more discerning about where you’re trying to work. Everybody wins if you’re specific on that point. Oh please, honestly. I'm not a terribly good programmer, but after 2+ years at one company, my inbox was flooded with emails from recruiters representing companies with very interesting opportunities (and like 10x as much crap). Then there's the friends and former coworkers who have been regularly asking me to come work with them. Once I made the decision to leave my last company, I talked to about 9 companies over a 1.5 month period, and it only lasted that long because of Christmas & New Years. At least 5 of those went through multiple interview rounds and I was juggling 4 different, attractive offers before picking one. This is just from the 9 that I decided to talk to... I live in a major market and there's a lot of demand here and in my focus areas. A lot of companies that you don't think have interesting problems actually have fascinating problems to work on once you talk to them. Probably much more so than GOOG, FB, AMZN, etc. You talk about being discerning, but how do you know if you can deliver value until you have a conversation with them? You end up talking to that many companies at once because once you start the process with them you have to be respectful and see their process through to the end, but you don't stop interviewing while you wait for a decision.
- zerr 9y agoFlagged. Since no payment is mentioned, so it is assumed that a candidate will do it for free... Lets not encourage such deeds.
- megous 9y agoSo if take-homes are to become the best practice for interviewing, perhaps there should be some service for re-using old ones that companies can trust that the candidate did for some other company and under which constraints. Otherwise it's just too much of a time waste for candidates, repeating take-homes for every single candidate company. It would be like personal projects but with a few guarantees on top, like that the candidate got the work assigned by someone else, and s/he did it under some time/other constraints. And there would perhaps be a public free-form feedback from the other comapny (no rating). So the next comapny has to judge the work itself without ability to pre-filter based on whatever rating the previous one would have given (to avoid bias, because the scoring would be non-objective/non-standardized). But I would still not like the take-homes. :)
- mighty_bander 9y agoI was hired by my current employer following a take-home coding challenge. I consider the experience very positive, and it has allowed us to hire skilled developers. However, I notice a few differences between my experience and those of people who complain about these challenges: 1. The challenge was timed, and limited to 3 hours. This is critical, because it respects the applicant's time. 2. Similarly, other interviewing was limited to a very brief "are you actually a developer at all" phone screen and a 30 minute personality interview. 3. I was allowed to choose the framework and language I was most comfortable with, demonstrating that the employer was interested in how well I worked, not in how well I knew the particular framework chosen for their product. I think it probably helped that I used the same stack advertised in the job description, but I appreciate the flexibility being there. Interviewing is a separate skill from software development. A take-home challenge isn't necessarily going to tell you everything about a person, but it seems to sort candidates into high/medium/low skill levels with a higher degree of accuracy than a paper or whiteboard tech interview.
- vanderreeah 9y agoAs someone who's just entered the employment market after being a freelancer for a while, I have no idea how people who already have jobs can physically participate in the lengthy interview processes most jobs require these days. Ok they can take sick days, but if you're a bad liar or have an overactive conscience that's not ideal. I totally understand the value of coding challenges, but I've been endlessly surprised at companies that expect me to expend more than 4 hours of my life interviewing for a chance at a job that will be paying maximum 50k. These are small companies, hardly the big 4. It's really an imposition.
- deleted 9y ago[deleted]
- grim-jokes 9y agoInstead of take home challenges, I think I would rather give a potential employee some code snippet and ask them to explain to me in their own way as to what is going on and if they see/find any particular issues. This will theoretically let me gauge their attention to detail. Ideally I think it would show me how familiar they are with the programming language the snippet was written in as well.