10 ms·

How to Interview Entry Level Software Engineers

150 points by dsil 9y ago

gopalv 9y ago
> Also, we chose to completely scrap whiteboard coding and assigned homework instead. For an entry level engineer, this removes a bunch of variance and also because it gets you another scenario which happens way often in actual work. You show up with the code and it gets reviewed & discussed on how/why of the implementation. The most talented entry level people I've worked with have struggled with criticism instead of grading (i.e "grade the code" through a bunch of test-cases is different from "why did you do this and explain").
- hcoura 9y ago
  This also opens space for people from different backgrounds (degrees). In most cases you are not looking for the people with the most level of CS knowledge, but rather someone that can think logically, that learns/ramps fast and have at least some explanation, even if wrong, for his choices and is able to take the observations/review as a learning opportunity.
- taurath 9y ago
  I’m a senior engineer with an anxiety disorder that has trouble with white boarding. It’s honestly probably the most cruel environment possible for me because when I work I’m on the same team.
  - socanxdev-thrwy 9y ago
    My social anxiety makes interviewing very difficult. I freeze up to a single-track-mind. Every spare bit of processing available is used up on guessing social situation around me. I recently had an old boss try to bring me on to a new job with him, but wasn't able to get me through the interview phase, despite knowing I'm capable, because my interviews with the rest of the team went so bad. I've been a software engineer over ten years. All of the positions that I have been offered have come from especially kind/empathetic teams, and the interviews usually focused more on conversation than quizzing. It requires an unfortunate amount of rejection before coming across my more selective criteria, but I guess that is what it is.
    - zip1234 9y ago
      Our coding test is over Skype and a collaborative editor. Is that a scenario when you would freeze up? Genuinely curious as there were a few cases where it seemed that people were way nervous.
      - socanxdev-thrwy 9y ago
        I personally would. Any situation where I'm being judged by strangers in real-time sets off social anxiety. Some things that help me in that situation are when there's some light/friendly conversation preceding the test, or when there's a limited the us-vs-them size, like only one or two interviewers at a time.
      - taurath 9y ago
        Yes very much so. The problem is that it’s such a high pressure environment, something that is the exact opposite of ideal thinking position. The focus is rarely on past accomplishments and a “bad day” of white boarding basically makes you look like an imposter. I feel like someone could get everything they needed to know about how well I code by drilling into specifics about what I’ve done.
      - zerkten 9y ago
        I think there is a greater chance that I would freeze up in this situation than I would on the job with a new problem, or an executive breathing down my neck. I don't think I'd react that way consistently, but I know it's further down the discomfort continuum. In the test: * I'm under pressure to impress, but also be my true self. I've had little contact with you so I'm not sure how to gauge all the feedback when I have increased sensitivity. * Sometimes don't have a real calibration for the level of the test. I've done the recommended prep, but it doesn't feel indicative of the test, or the interviewer inserts and additional level in before we've achieved the basics of the test. * What if I want to rough something out, but feel I'm exposing what I know would be a weakness from my pre-test assessment? I've just gone and made a hash of something I knew I needed to know. I'd say "how you collaborate" would be the biggest factor. If you start out with some code and fix up something, have me fix up something, and progressively raise the difficulty, then I'm much less likely to freeze up. I don't believe I'm particularly unique, but I don't find many tests delivered in this manner. It's more a case of "here you go for your practical test, show me what you can do kiddo" which just demonstrates how little programmers know about assessment, interviewing, and psychology. I'm going to state the following because I think the question was directed to one individual who identified as having a particular disorder. I don't believe I have any specific disorder that would cause me to freeze up. I'm an introvert, tested under Myers-Briggs as an INTP (if you consider that any value), and am OK in networking events. In jobs I've always tended to end up in regular conversations with skip-level managers etc.
  - andrewflnr 9y ago
    So here's a crazy idea. What if the "interviewer" doesn't know the answer, and the problem is designed to be big enough to require both of you to solve it? That puts you and them on an equal basis in a more realistic scenario. Plus it helps calibrate the results by generating data on how existing employees perform. Problems: the existing employee ends with even more latitude to wreck the interview. You need to ensure that the interviewee is taking the lead. You probably also need to record the whole thing for objective review by someone else, which you'll have to disclose, which could make it awkward again. Ed: I've also heard of processes where you put multiple interviewees on a task together. That has some similar potential benefits.
    - taurath 9y ago
      Sounds wonderful to me, but I think the “point” of the solo whiteboard experience is to reduce outside factors as much as possible. Of course that is really just a bit of a pipe dream since you’re hugely disadvantaging good coders with bad solo whiteboarding skills - pretending that interviews can be purely objective really does the process harm imo.
  - 13of40 9y ago
    Sloppy handwriting and a not-100%-progressive coding style is my bane when it comes to whiteboard coding. For the last two senior level dev jobs I've gotten, I just brought a laptop with Visual Studio set up and told the interviewer I sucked at whiteboard coding and could I please do it in an IDE.
    - taurath 9y ago
      That’s definitely something I feel like I should try. I have failed interviews despite having optimal solutions because the way I went about it seemed haphazard (I was thinking in terms of 2 or 3 things at once and bouncing around).
- pjc50 9y ago
  How do you check whether they've done it themselves? Rely on the discussion?
  - smoe 9y ago
    Not OP, but we do home work assignment instead of white board coding as well. While I can't be absolutely sure that they've done it themselves, a big part of the in person interview is based on the homework, asking them how they would extend it, add specific features, potential performance problems they see, etc. Those answers are hard to fake if you didn't do it yourself We had one case where a candidate used some open source code snippets in his solution, not declaring and removing the comments and license headers. Still it was very obvious in the difference of coding style.
cabaalis 9y ago
We hire mostly junior developers. I give a simple whiteboard interview with 3 questions, and encourage them to solve it with mostly pseudo-code but have a few "extra points" things that I'm looking for. Absolutely avoided any kind of "trick" questions. I like the homework idea. I don't know about "coming back" but maybe send it to them a couple days prior to the tech interview.
- jamestimmins 9y ago
  The one downside I've found with homework problems is that oftentimes the set of requirements (or "suggested features") doesn't line up with the amount of time you're expected to spend on it. I've personally been in the situation of receiving a list of 10-12 required features, with a stack that the recruiters know I am unfamiliar with, and then been told to only spend four hours on it. While this was an extreme case, I find that the unreasonable time expectation is not uncommon. That is quite frustrating as an applicant.
  - NetOpWibby 9y ago
    I love the "this shouldn't take more than an hour or two" assurances.
  - walshemj 9y ago
    They might be looking for how good you are at time boxing
deleted 9y ago
[deleted]
pfarnsworth 9y ago
It depends on how much effort we want to put into training entry level programmers. Different companies have different requirements. If we intend to train the programmer from the ground up, then we can probably look for programmers who are smart, resourceful and self-sufficient. We can train the rest, like good coding, etc. If our expectations are higher, then what I look for is code sensibility. Do they cut and paste the code without thinking "hey, this is kind of dumb to do"? Do they see repeated code and think "this can be collapsed into something more succinct"? This is the hardest thing to train for, and the most time consuming from a mentor point of view, so if we have a high bar for entry level programmers, then code sensibility is one of the factors on top of the 3 above that I look for. I've rejected interns that worked for me from a permanent position because after 4 months, they still had terrible code sensibility. I had to pour through their code with a fine tooth comb because it was riddled with tiny bugs, and it needed constant rewrites because they just didn't improve over the course of the internship. I've had other interns that just "got it" and wrote great code from the start, or only needed a few reviews and then "got it". Those are the ones that will be productive quickly and won't hamper the rest of the team with lengthy reviews.
bg4 9y ago
We give homework and it has been a successful approach. Even if it's a trivial assignment one can gain a lot of insight on the candidate based on the overall professionalism of their submission.
kinkrtyavimoodh 9y ago
Every few days an article of this type is written and posted on HN. And all the comments are people either proposing another (same old) approach to hiring or people replying to those comments explaining why they'd hate that approach. This is then done for every possible combination of approaches (homework questions, whiteboard interview, pair programming, fizzbuzz questions, algorithmic questions, work-history questions, career-progression questions, mini-project, magical incantations, and so on...) It may sound cynical but at this point it feels like no one has any new ideas about it and the same content keeps getting recycled in the form of yet another article and yet another comments section. Half of these articles are from company blogs which exist largely as just a marketing tool. Since there is no actual new thought, everyone just cargo-cults the Google approach.
- Kluny 9y ago
  I've never done hiring, but I've chosen roommates, and I feel like it's the same. You make a checklist of stuff you're looking for and red flags, do a reference check or whatever, but ultimately, the decision always comes down to some gut impulse. My gut has always been right so far, but I guess the day it burns me is the day I start trying to find some optimal solution to protect myself from letting an asshole move in.
  - johnrob 9y ago
    Anecdotally, most if not all of the times I’ve been burned has been when I ignored my gut feeling.
    - dudul 9y ago
      I have the same anecdote :) the interview all checked out, everybody gave a thumb up, but my guts were telling me "it's not gonna work". I forced myself to ignore it, put it on the account of unfair bias or something and hired the candidates. After 6 to 8 months they were gone after a disastrous tenure. It actually shook me a little after the 2nd time, because I thought I was doing the right thing not dismissing people just because of gut feeling.
  - 9y ago
Kequc 9y ago
I cannot count how many interviews I've done where I've shown up with 10+ years of industry experience and been given an interview catered toward juniors. If anything there is too much emphasis on interviewing entry level software engineers. There are others out there with a proven track record. I'm just underselling myself, shooting too low. At this point do I need to be applying for management?
- zip1234 9y ago
  What do you mean catered towards juniors? I interview people a lot and part of the interview, junior or not, involves a short coding test. The reason it is done is because some people can skate along and make an impressive resume but there is nothing to validate that resume. How do I know someone did the work that they are claiming? Not everyone has a github, and even then it can take time to validate someone's work.
  - walshemj 9y ago
    can't you call there previous employers for a reference?
    - u801e 9y ago
      Some companies have a policy of only confirming that they worked there and how long.
- mikeokner 9y ago
  It's tricky for interviewers because a lot of candidates claim to be "senior" yet can't even solve fizzbuzz on a whiteboard. Look for jobs with "Lead" or "Architect" in the title, but beyond that, expect that you'll have to go through some sort of stupid process as a bullshit screen.
  - monocasa 9y ago
    Can confirm, have interviewed people for a senior position with a wonderful resume with 20+ years of experience, who seem to have never seen a computer before. Flip the bit endianess of a word was one of the problems, and I saw a guy who literally (supposedly) did embedded software for the space shuttle just grind for 45 minutes and somehow not make any progress. Like it was impressive. I would have expected a random walk to make more progress.
B-Con 9y ago
> Notably, “potential” is nowhere on the scale. Many posts talk about hiring entry level engineers on their “potential,” but the term is seldom defined. Most people use “potential” to describe a sense that, even though a candidate currently cannot be an autonomous engineer, someday in the future they could be. We did not attempt to predict the future for any of our candidates but rather evaluated them on what they presented through the interviews. Well... > Some of the qualities we looked for in our associates in particular were: > Resilience: Learning on the job is hard and we assumed that the associates would make mistakes and struggle through difficult concepts. We needed people who could endure these struggles and bounce back ready for the next challenge. > Willingness to learn and the initiative to do so: Clover would assist the Associates in their growth, and provide teachers and mentors to help along the way, but any incoming Associate would need to be responsible for their own growth. > Humility: This is an important trait for all Clover engineers, but we paid special attention to it in our Associates. They would have to learn from those around them, be respectful of others, and be able to take difficult feedback with grace. > ... > Deciding the technical skills to evaluate was a long process. We expected to teach our Associates most of the technical skills they would need to do their job, but we couldn’t accept candidates that were completely blank slates. It really sounds like this is measuring the candidate's "potential". I would argue they defined "potential" as "resilience, willingness to learn, and humility", which they feel is an atypical definition. (Personally, I think that's a decent set of criteria to look for in entry-level engineers.)
- zip1234 9y ago
  Resilience, willingness to learn, and humility can be the potential, but I would also say that they are the candidates attitude. Yes, I would say that attitude makes a big difference.
drakonka 9y ago
I was pretty nervous at my interview; no test was mentioned until the day, at which time (after the rest of the interview) I was presented with two sheets of xml-style configuration code for the company's proprietary build system. The task was super simple - basically just duplicating some of the configuration and renaming some values to create another job, but at the time I just froze and couldn't see it. I managed to get some sort of answer out, but wasn't happy with my response. They claimed that they were meant to give this to me in advance as a homework question and somewhere the ball got dropped, but I had a feeling they wanted it to be a more on the spot thing considering the problem was so simple. Thankfully I had the wherewithal to ask if I could take the problem home with me after the interview. When I got home and looked at the problem with fresh eyes I facepalmed so hard. I wrote up the correct answer and emailed it over to the company recruiter, hoping she would pass it along to my interviewers, but didn't get my hopes up. I guess she did, since I ended up getting the job. I still cringe a bit about how I just blanked on that stupid question. Thankfully I got ramped up really quickly once I actually started the job and had no problem figuring things out after that.
AlexCoventry 9y ago
It'll be interesting to see how it works out. Without some kind of real-time test of facility with abstract symbolic reasoning, I would expect them to get burned. It's way too easy for people to fool themselves and others about that kind of capability. If they asked the subjects to modify the homework slightly during the debriefing, that might be enough.
makecheck 9y ago
This is generally fine but “homework” is a terrible idea (trend?) because it isn’t looking from the perspective of the new hire: 1. Unemployment can be extremely stressful and there is a very good chance the person needs a job NOW. Every little thing you do to dilly-dally instead of making an offer is another hurdle that may send the applicant elsewhere. Act as soon as possible when you see a résumé you like (get on the phone, find out what you want to know, make a decision; if you bring them to interview and they’re struggling, give feedback immediately; don’t have a system where you wait 3 weeks to tell them yes/no). 2. Any effort you expect an applicant to spend will be greatly multiplied. That person is not talking only to you, they’re looking at LOTS of job postings!! If every potential employer starts assigning “homework”, it becomes a major hurdle and you can bet they’ll look a lot closer at an employer that might provide a paycheck in a couple weeks instead of “homework”. 3. You don’t need complex problems to understand if somebody can code. Let them use whatever language their résumé claims they know, give them something to do for a few minutes and see what happens. If they get stuck, give them a solid hint and again, see what happens. You’re trying to assess things like (a) if they appear to know what their résumé claims, (b) if they are the kind of person you can interact with to get somewhere on a problem.
- tptacek 9y ago
  I don't think you've thought this through carefully. What do we think the median interval from "established intent to interview" to "offer" is at medium-to-large software companies in SFBA? If I had to guess, it's on the order of a month. Nobody's hiring homework process puts a dent in that timeline. On the other hand, when done well, "homework" makes it easier for engineering teams to make confident decisions, which expedites offers. How much effort do you think it takes a typical candidate to get through an interview gauntlet at a tech company with more than (say) 50 employees? It's at least multiple full days worth of aggregate effort, almost always including one actual, on-site full day. To the extent that homework offsets on-site interviews, it reduces the amount of effort candidates need to put in.
  - walshemj 9y ago
    Is it your ignoring the fact that some good candidates will just look at the job add see the home work coding tests and think "fuck you pay me" and ignore the opportunity. Who needs a full 1 day of interviews plus time for phone interviews for an entry level job.
convolvatron 9y ago
entry level is bar far the easiest to interview for. with experienced people they always think they are at or near the top of the scale, but dont understand that you have a different scale entirely. with a junior person, ask them to describe a project they worked on. if they 'i dunno, school stuff', then you're done. if you can engage with them meaningfully about their school work, or if they've done personal projects, have developed any kind of insight, show a spark of excitement or creativity then you're done - hired. just keep an eye on them for a while to make sure they're engaged and absorbing culture. invest a good amount of time in their success (mentorship, pair debugging, whiteboard talks). if they dont start contributing more than they are costing in time after a few months, find them another role or fire them.
- mynameishere 9y ago
  I love your attitude, but when I got out of school, nobody gave a flying eff about personal projects. This was before Google Analytics, so I had to track IPs myself, but I think something like .1 percent of companies looked at my website--and that might have been a false positive. Even the company that eventually hired me didn't look at my website.
  - convolvatron 9y ago
    if you've spent years programming and dont have anything to say about it, there isn't anything for me to go on. you obviously dont have any real interest in it, and you're below the bar where i'm willing to invest the time to see if you're teachable. what would you suggest instead? gpa? top school? test? those dont seem particularly relevant. edit: oh, maybe i wasn't clear. school projects are fine grist, you just have to be able to say something interesting about them
    - mynameishere 9y ago
      I'm not arguing with you. Just saying the attitude is (or was) rare in my experience.
grad_ml 9y ago
I would say, try to test the fundamentals. Do they understands how memory is accessed, have sense what kind of algorithm is used (though it would be too much to code in 45 minutes), ambitious, have general idea to solve a problem, can they see issues if things are not performant... Having a conversation, comfortable conversation is the key, imo.
- walshemj 9y ago
  For a general developer role that seems excessive not sure what you mean by algorithem here? I would assume you'd be asking about how the address and data busses work which varies by cpu and architecture (von Neumann vs Harvard) presumably you'd be happy if some one explained how PIC memory access works ?
  - grad_ml 9y ago
    With algorithms, I mean if the problem demands some optimization, be it network flow, string compression or some kind of resource allocation (something like register allocation in compilers), one should have the vague idea which direction to pursue or approach. This requires somehow rigorous course-work in school. Assuming everyone takes standard CS machine architecture courses, having the breadth in multiple domain is, essential, imo for a well rounded engineer. Now knowing(or having no idea) how memory access, hierarchy etc works, they won't know when things just get stuck.
kelnos 9y ago
One thing I'd be interested in, but feel like I never see, is a post-post-mortem somewhere down the line, say a year or two later. This gets a little tricky because you don't want to air your company's dirty laundry or say negative things that could be easily traced to point to a specific person, but... In theory, I like the approach they've taken to hiring associates, but at the end of the day, what really matters is: are the people they end up hiring successful at their jobs? It might take a year or more to really answer that question. I don't feel like we're very good at this. As 'kinkrtyavimoodh points out in another comment, we see a lot of "how to interview engineer" type posts. But we never find out if, in the end, they were effective. This article claims that everything went really well because they had (or believed they had) such high-quality candidates that it was difficult to choose at the end, but they don't actually know that. They won't find this out after evaluating actual on-the-job performance. It might have been hard to pick from their candidate pool because their process failed to tease out flaws that are only going to become apparent later. We just don't know, and likely will never know.
rdiddly 9y ago
You with your sensible guidelines, your calibrated metrics, your well-defined endpoints. Where's all the VOODOO!??! Where's the WITCHCRAFT!??!?!
kelvin0 9y ago
I have failed many time at interviewing at technical interviews. I have never failed any 'real' world job project/endeavor. I've built an entire org's codebase and this company is now selling products worldwide successfully. Design,coded,built,tested and deployed code in 4 languages , each element being critical to the company. I am an experienced dev, but couldn't tell you my birthday in a technical interview. All the rest of the interview is fine, I only 'freeze' up at technical questions. Sucks.
Overtonwindow 9y ago
I have only one modest proposal: No. More. Whiteboard interviews!!
wyclif 9y ago
I saw the title to this piece on the HN front page, and thought, "Wow, I'd like to see what a random tech company does with entry level engineers." Too bad this isn't a piece of writing that really discusses that in detail. There is no real actionable content here; it reads more like an HR post or marketing for Clover. A shame.

150 points by dsil 9y ago

> Also, we chose to completely scrap whiteboard coding and assigned homework instead. For an entry level engineer, this removes a bunch of variance and also because it gets you another scenario which happens way often in actual work. You show up with the code and it gets reviewed & discussed on how/why of the implementation. The most talented entry level people I've worked with have struggled with criticism instead of grading (i.e "grade the code" through a bunch of test-cases is different from "why did you do this and explain").

hcoura 9y ago
This also opens space for people from different backgrounds (degrees). In most cases you are not looking for the people with the most level of CS knowledge, but rather someone that can think logically, that learns/ramps fast and have at least some explanation, even if wrong, for his choices and is able to take the observations/review as a learning opportunity.
taurath 9y ago
I’m a senior engineer with an anxiety disorder that has trouble with white boarding. It’s honestly probably the most cruel environment possible for me because when I work I’m on the same team.
- socanxdev-thrwy 9y ago
  My social anxiety makes interviewing very difficult. I freeze up to a single-track-mind. Every spare bit of processing available is used up on guessing social situation around me. I recently had an old boss try to bring me on to a new job with him, but wasn't able to get me through the interview phase, despite knowing I'm capable, because my interviews with the rest of the team went so bad. I've been a software engineer over ten years. All of the positions that I have been offered have come from especially kind/empathetic teams, and the interviews usually focused more on conversation than quizzing. It requires an unfortunate amount of rejection before coming across my more selective criteria, but I guess that is what it is.
  - zip1234 9y ago
    Our coding test is over Skype and a collaborative editor. Is that a scenario when you would freeze up? Genuinely curious as there were a few cases where it seemed that people were way nervous.
    - socanxdev-thrwy 9y ago
      I personally would. Any situation where I'm being judged by strangers in real-time sets off social anxiety. Some things that help me in that situation are when there's some light/friendly conversation preceding the test, or when there's a limited the us-vs-them size, like only one or two interviewers at a time.
    - taurath 9y ago
      Yes very much so. The problem is that it’s such a high pressure environment, something that is the exact opposite of ideal thinking position. The focus is rarely on past accomplishments and a “bad day” of white boarding basically makes you look like an imposter. I feel like someone could get everything they needed to know about how well I code by drilling into specifics about what I’ve done.
    - zerkten 9y ago
      I think there is a greater chance that I would freeze up in this situation than I would on the job with a new problem, or an executive breathing down my neck. I don't think I'd react that way consistently, but I know it's further down the discomfort continuum. In the test: * I'm under pressure to impress, but also be my true self. I've had little contact with you so I'm not sure how to gauge all the feedback when I have increased sensitivity. * Sometimes don't have a real calibration for the level of the test. I've done the recommended prep, but it doesn't feel indicative of the test, or the interviewer inserts and additional level in before we've achieved the basics of the test. * What if I want to rough something out, but feel I'm exposing what I know would be a weakness from my pre-test assessment? I've just gone and made a hash of something I knew I needed to know. I'd say "how you collaborate" would be the biggest factor. If you start out with some code and fix up something, have me fix up something, and progressively raise the difficulty, then I'm much less likely to freeze up. I don't believe I'm particularly unique, but I don't find many tests delivered in this manner. It's more a case of "here you go for your practical test, show me what you can do kiddo" which just demonstrates how little programmers know about assessment, interviewing, and psychology. I'm going to state the following because I think the question was directed to one individual who identified as having a particular disorder. I don't believe I have any specific disorder that would cause me to freeze up. I'm an introvert, tested under Myers-Briggs as an INTP (if you consider that any value), and am OK in networking events. In jobs I've always tended to end up in regular conversations with skip-level managers etc.
- andrewflnr 9y ago
  So here's a crazy idea. What if the "interviewer" doesn't know the answer, and the problem is designed to be big enough to require both of you to solve it? That puts you and them on an equal basis in a more realistic scenario. Plus it helps calibrate the results by generating data on how existing employees perform. Problems: the existing employee ends with even more latitude to wreck the interview. You need to ensure that the interviewee is taking the lead. You probably also need to record the whole thing for objective review by someone else, which you'll have to disclose, which could make it awkward again. Ed: I've also heard of processes where you put multiple interviewees on a task together. That has some similar potential benefits.
  - taurath 9y ago
    Sounds wonderful to me, but I think the “point” of the solo whiteboard experience is to reduce outside factors as much as possible. Of course that is really just a bit of a pipe dream since you’re hugely disadvantaging good coders with bad solo whiteboarding skills - pretending that interviews can be purely objective really does the process harm imo.
- 13of40 9y ago
  Sloppy handwriting and a not-100%-progressive coding style is my bane when it comes to whiteboard coding. For the last two senior level dev jobs I've gotten, I just brought a laptop with Visual Studio set up and told the interviewer I sucked at whiteboard coding and could I please do it in an IDE.
  - taurath 9y ago
    That’s definitely something I feel like I should try. I have failed interviews despite having optimal solutions because the way I went about it seemed haphazard (I was thinking in terms of 2 or 3 things at once and bouncing around).
pjc50 9y ago
How do you check whether they've done it themselves? Rely on the discussion?
- smoe 9y ago
  Not OP, but we do home work assignment instead of white board coding as well. While I can't be absolutely sure that they've done it themselves, a big part of the in person interview is based on the homework, asking them how they would extend it, add specific features, potential performance problems they see, etc. Those answers are hard to fake if you didn't do it yourself We had one case where a candidate used some open source code snippets in his solution, not declaring and removing the comments and license headers. Still it was very obvious in the difference of coding style.

cabaalis 9y ago

We hire mostly junior developers. I give a simple whiteboard interview with 3 questions, and encourage them to solve it with mostly pseudo-code but have a few "extra points" things that I'm looking for. Absolutely avoided any kind of "trick" questions. I like the homework idea. I don't know about "coming back" but maybe send it to them a couple days prior to the tech interview.

jamestimmins 9y ago
The one downside I've found with homework problems is that oftentimes the set of requirements (or "suggested features") doesn't line up with the amount of time you're expected to spend on it. I've personally been in the situation of receiving a list of 10-12 required features, with a stack that the recruiters know I am unfamiliar with, and then been told to only spend four hours on it. While this was an extreme case, I find that the unreasonable time expectation is not uncommon. That is quite frustrating as an applicant.
- NetOpWibby 9y ago
  I love the "this shouldn't take more than an hour or two" assurances.
- walshemj 9y ago
  They might be looking for how good you are at time boxing

deleted 9y ago

[deleted]

pfarnsworth 9y ago

It depends on how much effort we want to put into training entry level programmers. Different companies have different requirements. If we intend to train the programmer from the ground up, then we can probably look for programmers who are smart, resourceful and self-sufficient. We can train the rest, like good coding, etc. If our expectations are higher, then what I look for is code sensibility. Do they cut and paste the code without thinking "hey, this is kind of dumb to do"? Do they see repeated code and think "this can be collapsed into something more succinct"? This is the hardest thing to train for, and the most time consuming from a mentor point of view, so if we have a high bar for entry level programmers, then code sensibility is one of the factors on top of the 3 above that I look for. I've rejected interns that worked for me from a permanent position because after 4 months, they still had terrible code sensibility. I had to pour through their code with a fine tooth comb because it was riddled with tiny bugs, and it needed constant rewrites because they just didn't improve over the course of the internship. I've had other interns that just "got it" and wrote great code from the start, or only needed a few reviews and then "got it". Those are the ones that will be productive quickly and won't hamper the rest of the team with lengthy reviews.

bg4 9y ago

We give homework and it has been a successful approach. Even if it's a trivial assignment one can gain a lot of insight on the candidate based on the overall professionalism of their submission.

kinkrtyavimoodh 9y ago

Every few days an article of this type is written and posted on HN. And all the comments are people either proposing another (same old) approach to hiring or people replying to those comments explaining why they'd hate that approach. This is then done for every possible combination of approaches (homework questions, whiteboard interview, pair programming, fizzbuzz questions, algorithmic questions, work-history questions, career-progression questions, mini-project, magical incantations, and so on...) It may sound cynical but at this point it feels like no one has any new ideas about it and the same content keeps getting recycled in the form of yet another article and yet another comments section. Half of these articles are from company blogs which exist largely as just a marketing tool. Since there is no actual new thought, everyone just cargo-cults the Google approach.

Kluny 9y ago
I've never done hiring, but I've chosen roommates, and I feel like it's the same. You make a checklist of stuff you're looking for and red flags, do a reference check or whatever, but ultimately, the decision always comes down to some gut impulse. My gut has always been right so far, but I guess the day it burns me is the day I start trying to find some optimal solution to protect myself from letting an asshole move in.
- johnrob 9y ago
  Anecdotally, most if not all of the times I’ve been burned has been when I ignored my gut feeling.
  - dudul 9y ago
    I have the same anecdote :) the interview all checked out, everybody gave a thumb up, but my guts were telling me "it's not gonna work". I forced myself to ignore it, put it on the account of unfair bias or something and hired the candidates. After 6 to 8 months they were gone after a disastrous tenure. It actually shook me a little after the 2nd time, because I thought I was doing the right thing not dismissing people just because of gut feeling.
- 9y ago

Kequc 9y ago

I cannot count how many interviews I've done where I've shown up with 10+ years of industry experience and been given an interview catered toward juniors. If anything there is too much emphasis on interviewing entry level software engineers. There are others out there with a proven track record. I'm just underselling myself, shooting too low. At this point do I need to be applying for management?

zip1234 9y ago
What do you mean catered towards juniors? I interview people a lot and part of the interview, junior or not, involves a short coding test. The reason it is done is because some people can skate along and make an impressive resume but there is nothing to validate that resume. How do I know someone did the work that they are claiming? Not everyone has a github, and even then it can take time to validate someone's work.
- walshemj 9y ago
  can't you call there previous employers for a reference?
  - u801e 9y ago
    Some companies have a policy of only confirming that they worked there and how long.
mikeokner 9y ago
It's tricky for interviewers because a lot of candidates claim to be "senior" yet can't even solve fizzbuzz on a whiteboard. Look for jobs with "Lead" or "Architect" in the title, but beyond that, expect that you'll have to go through some sort of stupid process as a bullshit screen.
- monocasa 9y ago
  Can confirm, have interviewed people for a senior position with a wonderful resume with 20+ years of experience, who seem to have never seen a computer before. Flip the bit endianess of a word was one of the problems, and I saw a guy who literally (supposedly) did embedded software for the space shuttle just grind for 45 minutes and somehow not make any progress. Like it was impressive. I would have expected a random walk to make more progress.

B-Con 9y ago

> Notably, “potential” is nowhere on the scale. Many posts talk about hiring entry level engineers on their “potential,” but the term is seldom defined. Most people use “potential” to describe a sense that, even though a candidate currently cannot be an autonomous engineer, someday in the future they could be. We did not attempt to predict the future for any of our candidates but rather evaluated them on what they presented through the interviews. Well... > Some of the qualities we looked for in our associates in particular were: > Resilience: Learning on the job is hard and we assumed that the associates would make mistakes and struggle through difficult concepts. We needed people who could endure these struggles and bounce back ready for the next challenge. > Willingness to learn and the initiative to do so: Clover would assist the Associates in their growth, and provide teachers and mentors to help along the way, but any incoming Associate would need to be responsible for their own growth. > Humility: This is an important trait for all Clover engineers, but we paid special attention to it in our Associates. They would have to learn from those around them, be respectful of others, and be able to take difficult feedback with grace. > ... > Deciding the technical skills to evaluate was a long process. We expected to teach our Associates most of the technical skills they would need to do their job, but we couldn’t accept candidates that were completely blank slates. It really sounds like this is measuring the candidate's "potential". I would argue they defined "potential" as "resilience, willingness to learn, and humility", which they feel is an atypical definition. (Personally, I think that's a decent set of criteria to look for in entry-level engineers.)

zip1234 9y ago
Resilience, willingness to learn, and humility can be the potential, but I would also say that they are the candidates attitude. Yes, I would say that attitude makes a big difference.

drakonka 9y ago

I was pretty nervous at my interview; no test was mentioned until the day, at which time (after the rest of the interview) I was presented with two sheets of xml-style configuration code for the company's proprietary build system. The task was super simple - basically just duplicating some of the configuration and renaming some values to create another job, but at the time I just froze and couldn't see it. I managed to get some sort of answer out, but wasn't happy with my response. They claimed that they were meant to give this to me in advance as a homework question and somewhere the ball got dropped, but I had a feeling they wanted it to be a more on the spot thing considering the problem was so simple. Thankfully I had the wherewithal to ask if I could take the problem home with me after the interview. When I got home and looked at the problem with fresh eyes I facepalmed so hard. I wrote up the correct answer and emailed it over to the company recruiter, hoping she would pass it along to my interviewers, but didn't get my hopes up. I guess she did, since I ended up getting the job. I still cringe a bit about how I just blanked on that stupid question. Thankfully I got ramped up really quickly once I actually started the job and had no problem figuring things out after that.

AlexCoventry 9y ago

It'll be interesting to see how it works out. Without some kind of real-time test of facility with abstract symbolic reasoning, I would expect them to get burned. It's way too easy for people to fool themselves and others about that kind of capability. If they asked the subjects to modify the homework slightly during the debriefing, that might be enough.

makecheck 9y ago

This is generally fine but “homework” is a terrible idea (trend?) because it isn’t looking from the perspective of the new hire: 1. Unemployment can be extremely stressful and there is a very good chance the person needs a job NOW. Every little thing you do to dilly-dally instead of making an offer is another hurdle that may send the applicant elsewhere. Act as soon as possible when you see a résumé you like (get on the phone, find out what you want to know, make a decision; if you bring them to interview and they’re struggling, give feedback immediately; don’t have a system where you wait 3 weeks to tell them yes/no). 2. Any effort you expect an applicant to spend will be greatly multiplied. That person is not talking only to you, they’re looking at LOTS of job postings!! If every potential employer starts assigning “homework”, it becomes a major hurdle and you can bet they’ll look a lot closer at an employer that might provide a paycheck in a couple weeks instead of “homework”. 3. You don’t need complex problems to understand if somebody can code. Let them use whatever language their résumé claims they know, give them something to do for a few minutes and see what happens. If they get stuck, give them a solid hint and again, see what happens. You’re trying to assess things like (a) if they appear to know what their résumé claims, (b) if they are the kind of person you can interact with to get somewhere on a problem.

tptacek 9y ago
I don't think you've thought this through carefully. What do we think the median interval from "established intent to interview" to "offer" is at medium-to-large software companies in SFBA? If I had to guess, it's on the order of a month. Nobody's hiring homework process puts a dent in that timeline. On the other hand, when done well, "homework" makes it easier for engineering teams to make confident decisions, which expedites offers. How much effort do you think it takes a typical candidate to get through an interview gauntlet at a tech company with more than (say) 50 employees? It's at least multiple full days worth of aggregate effort, almost always including one actual, on-site full day. To the extent that homework offsets on-site interviews, it reduces the amount of effort candidates need to put in.
- walshemj 9y ago
  Is it your ignoring the fact that some good candidates will just look at the job add see the home work coding tests and think "fuck you pay me" and ignore the opportunity. Who needs a full 1 day of interviews plus time for phone interviews for an entry level job.

convolvatron 9y ago

entry level is bar far the easiest to interview for. with experienced people they always think they are at or near the top of the scale, but dont understand that you have a different scale entirely. with a junior person, ask them to describe a project they worked on. if they 'i dunno, school stuff', then you're done. if you can engage with them meaningfully about their school work, or if they've done personal projects, have developed any kind of insight, show a spark of excitement or creativity then you're done - hired. just keep an eye on them for a while to make sure they're engaged and absorbing culture. invest a good amount of time in their success (mentorship, pair debugging, whiteboard talks). if they dont start contributing more than they are costing in time after a few months, find them another role or fire them.

mynameishere 9y ago
I love your attitude, but when I got out of school, nobody gave a flying eff about personal projects. This was before Google Analytics, so I had to track IPs myself, but I think something like .1 percent of companies looked at my website--and that might have been a false positive. Even the company that eventually hired me didn't look at my website.
- convolvatron 9y ago
  if you've spent years programming and dont have anything to say about it, there isn't anything for me to go on. you obviously dont have any real interest in it, and you're below the bar where i'm willing to invest the time to see if you're teachable. what would you suggest instead? gpa? top school? test? those dont seem particularly relevant. edit: oh, maybe i wasn't clear. school projects are fine grist, you just have to be able to say something interesting about them
  - mynameishere 9y ago
    I'm not arguing with you. Just saying the attitude is (or was) rare in my experience.

grad_ml 9y ago

I would say, try to test the fundamentals. Do they understands how memory is accessed, have sense what kind of algorithm is used (though it would be too much to code in 45 minutes), ambitious, have general idea to solve a problem, can they see issues if things are not performant... Having a conversation, comfortable conversation is the key, imo.

walshemj 9y ago
For a general developer role that seems excessive not sure what you mean by algorithem here? I would assume you'd be asking about how the address and data busses work which varies by cpu and architecture (von Neumann vs Harvard) presumably you'd be happy if some one explained how PIC memory access works ?
- grad_ml 9y ago
  With algorithms, I mean if the problem demands some optimization, be it network flow, string compression or some kind of resource allocation (something like register allocation in compilers), one should have the vague idea which direction to pursue or approach. This requires somehow rigorous course-work in school. Assuming everyone takes standard CS machine architecture courses, having the breadth in multiple domain is, essential, imo for a well rounded engineer. Now knowing(or having no idea) how memory access, hierarchy etc works, they won't know when things just get stuck.

kelnos 9y ago

One thing I'd be interested in, but feel like I never see, is a post-post-mortem somewhere down the line, say a year or two later. This gets a little tricky because you don't want to air your company's dirty laundry or say negative things that could be easily traced to point to a specific person, but... In theory, I like the approach they've taken to hiring associates, but at the end of the day, what really matters is: are the people they end up hiring successful at their jobs? It might take a year or more to really answer that question. I don't feel like we're very good at this. As 'kinkrtyavimoodh points out in another comment, we see a lot of "how to interview engineer" type posts. But we never find out if, in the end, they were effective. This article claims that everything went really well because they had (or believed they had) such high-quality candidates that it was difficult to choose at the end, but they don't actually know that. They won't find this out after evaluating actual on-the-job performance. It might have been hard to pick from their candidate pool because their process failed to tease out flaws that are only going to become apparent later. We just don't know, and likely will never know.

rdiddly 9y ago

You with your sensible guidelines, your calibrated metrics, your well-defined endpoints. Where's all the VOODOO!??! Where's the WITCHCRAFT!??!?!

kelvin0 9y ago

I have failed many time at interviewing at technical interviews. I have never failed any 'real' world job project/endeavor. I've built an entire org's codebase and this company is now selling products worldwide successfully. Design,coded,built,tested and deployed code in 4 languages , each element being critical to the company. I am an experienced dev, but couldn't tell you my birthday in a technical interview. All the rest of the interview is fine, I only 'freeze' up at technical questions. Sucks.

Overtonwindow 9y ago

I have only one modest proposal: No. More. Whiteboard interviews!!

wyclif 9y ago

I saw the title to this piece on the HN front page, and thought, "Wow, I'd like to see what a random tech company does with entry level engineers." Too bad this isn't a piece of writing that really discusses that in detail. There is no real actionable content here; it reads more like an HR post or marketing for Clover. A shame.