7 ms·

Three things: First, those are questions for a variety of different positions all mixed together. Product manager candidates get asked different questions than

Three things: First, those are questions for a variety of different positions all mixed together. Product manager candidates get asked different questions than engineering candidates. Second, yes, Microsoft tracks the effectiveness of interviewers based on the outcome of the interview loops and the performance of the people who get hired. Third, all of the questions listed, no matter how trivial they seem, are dispositive. I've interviewed engineering candidates with PhD's from top tier universities who couldn't reverse a linked list when asked, or who couldn't even explain basic concepts in their putative focus areas. You have to ask the seemingly stupid stuff -- it's a continual surprise. Full disclosure: no, I don't work there now. Yes, I used to, a very long time ago.

timr 17y ago
"Third, all of the questions listed, no matter how trivial they seem, are dispositive. I've interviewed engineering candidates with PhD's from top tier universities who couldn't reverse a linked list when asked, or who couldn't even explain basic concepts in their putative focus areas. You have to ask the seemingly stupid stuff -- it's a continual surprise." While there are certainly idiots with PhDs, if you've got a candidate with a PhD from a top-tier institution who "can't reverse a linked-list", it's most likely because (s)he is under-prepared at the art of technical interviewing. Idiocy is clearly not impossible, but the conclusion that the candidate is an idiot should be on the low-prior-probability event list. One thing that drives me absolutely crazy about technical interviews today is that most interviewers have completely lost the bubble on what they're trying to accomplish. It's become a bizarre, nerdy form of Kabuki theater, wherein candidates are madly trying to cram their heads full of list- and string-algorithm esoterica, while hoping that they're not presented questions so unfamiliar that they can't derive the answer in under 30 minutes, at a whiteboard. If the interviewer doesn't take this into account (and few do), the interview becomes little more than a random, high-pass screen, wherein lots of smart people are eliminated from positions based on bad luck, and little else. Joel (on Software) has it right, but it seems like few people are listening: you want to make sure the candidate is smart, and can get things done. That's it. The goal is not to see if they're the next human incarnation of Alan Turing, and it's certainly not to see if they can derive fiendishly difficult algorithms during a whiteboard lecture (call me crazy, but I'm reasonably sure that Turing didn't come up with his theories while talking continuously in front of a whiteboard, while some asperger-y geek sneered at him from across the room.) When you're interviewing, you want to make sure that the candidate can write code, that they're reasonably smart, and (IMHO) that they're not an asshole. It seems to me that our industry has taken "coding puzzle" and turned it into "crappy intelligence test", while largely ignoring the "not an asshole" part of the equation -- the part that's usually the most important in real life.
- ambition 17y ago
  "Gets-things-done" people prepare for technical interviews, such that they don't flub simple questions. The danger for the employer is that lots of Computer Science PhDs end up not writing much code during their research, and may not be a good fit at many tech employers. "Interview for Smart and Gets Things Done" is an accurate explanatory abstraction over what good interviewers do, but it's not prescriptive enough to tell people what questions to ask and how to judge the results.
  - timr 17y ago
    "'Gets-things-done' people prepare for technical interviews, such that they don't flub simple questions." So, let's be clear: you're admitting that you're screening for a trait that you assume is correlated with the trait that you actually want. I don't grant the assumption, but it should at least be explicitly stated. "The danger for the employer is that lots of Computer Science PhDs end up not writing much code during their research, and may not be a good fit at many tech employers." How many computer science PhDs have you hired, let alone interviewed? I'd wager that it's not enough for you to be able to make this judgment with any confidence. And even if you're right, how does asking questions about linked-list reversal address the question of the tendency of a person to do practical work? Aside from tech interviews, I've never once in my life had to write a linked-list reversing routine. Look, I'm not saying "don't ask coding questions" -- I'm saying that we need to start being reasonable. Don't assume that a candidate is a no-hire simply because that they haven't pre-memorized the algorithms for the questions that you're asking. It's ridiculous that we're screening people based on the number of silly tricks that they can memorize from interview question websites.
    - ambition 17y ago
      I think by and large we agree: It's wrong to expect memorized answers or to ask questions that are so narrow that they only test whether a candidate spent time studying. I'd even go so far as to say that algorithmic questions are probably not good indicators for skill at many kinds of work we would call "programming"---i.e. Programming skill is more heterogeneous than many interviewers admit. We shouldn't expect a jQuery wiz to nail low-level data structure questions, and we shouldn't expect a bit-twiddling video codec developer to really grok method chaining in 30 minutes. The trouble with "Be reasonable" is that's it's the advice equivalent of a tautology. Of course we should be reasonable when interviewing. But I don't think there's widespread agreement about how to operationalize that. I'd be curious for more detail about how you would do it---you seem to have strong, well-informed feelings on this issue. To my knowledge, there's basically no publicly-available research on tech interview factors and how they correlate with on-the-job performance. The good big employers do this research internally and keep it to themselves. The rest of us are stuck with assumptions, intuitions, logic and argument. So unfortunately I don't think we'll be able to get the debate into the realm of interpreting real data anytime soon.
arethuza 17y ago
I don't know how it works in the US, but in the UK a PhD is usually awarded based purely on research - there is relatively little cutting edge research that I am aware of that hinges on manipulating linked lists. :-)

timr 17y ago

"Third, all of the questions listed, no matter how trivial they seem, are dispositive. I've interviewed engineering candidates with PhD's from top tier universities who couldn't reverse a linked list when asked, or who couldn't even explain basic concepts in their putative focus areas. You have to ask the seemingly stupid stuff -- it's a continual surprise." While there are certainly idiots with PhDs, if you've got a candidate with a PhD from a top-tier institution who "can't reverse a linked-list", it's most likely because (s)he is under-prepared at the art of technical interviewing. Idiocy is clearly not impossible, but the conclusion that the candidate is an idiot should be on the low-prior-probability event list. One thing that drives me absolutely crazy about technical interviews today is that most interviewers have completely lost the bubble on what they're trying to accomplish. It's become a bizarre, nerdy form of Kabuki theater, wherein candidates are madly trying to cram their heads full of list- and string-algorithm esoterica, while hoping that they're not presented questions so unfamiliar that they can't derive the answer in under 30 minutes, at a whiteboard. If the interviewer doesn't take this into account (and few do), the interview becomes little more than a random, high-pass screen, wherein lots of smart people are eliminated from positions based on bad luck, and little else. Joel (on Software) has it right, but it seems like few people are listening: you want to make sure the candidate is smart, and can get things done. That's it. The goal is not to see if they're the next human incarnation of Alan Turing, and it's certainly not to see if they can derive fiendishly difficult algorithms during a whiteboard lecture (call me crazy, but I'm reasonably sure that Turing didn't come up with his theories while talking continuously in front of a whiteboard, while some asperger-y geek sneered at him from across the room.) When you're interviewing, you want to make sure that the candidate can write code, that they're reasonably smart, and (IMHO) that they're not an asshole. It seems to me that our industry has taken "coding puzzle" and turned it into "crappy intelligence test", while largely ignoring the "not an asshole" part of the equation -- the part that's usually the most important in real life.

ambition 17y ago
"Gets-things-done" people prepare for technical interviews, such that they don't flub simple questions. The danger for the employer is that lots of Computer Science PhDs end up not writing much code during their research, and may not be a good fit at many tech employers. "Interview for Smart and Gets Things Done" is an accurate explanatory abstraction over what good interviewers do, but it's not prescriptive enough to tell people what questions to ask and how to judge the results.
- timr 17y ago
  "'Gets-things-done' people prepare for technical interviews, such that they don't flub simple questions." So, let's be clear: you're admitting that you're screening for a trait that you assume is correlated with the trait that you actually want. I don't grant the assumption, but it should at least be explicitly stated. "The danger for the employer is that lots of Computer Science PhDs end up not writing much code during their research, and may not be a good fit at many tech employers." How many computer science PhDs have you hired, let alone interviewed? I'd wager that it's not enough for you to be able to make this judgment with any confidence. And even if you're right, how does asking questions about linked-list reversal address the question of the tendency of a person to do practical work? Aside from tech interviews, I've never once in my life had to write a linked-list reversing routine. Look, I'm not saying "don't ask coding questions" -- I'm saying that we need to start being reasonable. Don't assume that a candidate is a no-hire simply because that they haven't pre-memorized the algorithms for the questions that you're asking. It's ridiculous that we're screening people based on the number of silly tricks that they can memorize from interview question websites.
  - ambition 17y ago
    I think by and large we agree: It's wrong to expect memorized answers or to ask questions that are so narrow that they only test whether a candidate spent time studying. I'd even go so far as to say that algorithmic questions are probably not good indicators for skill at many kinds of work we would call "programming"---i.e. Programming skill is more heterogeneous than many interviewers admit. We shouldn't expect a jQuery wiz to nail low-level data structure questions, and we shouldn't expect a bit-twiddling video codec developer to really grok method chaining in 30 minutes. The trouble with "Be reasonable" is that's it's the advice equivalent of a tautology. Of course we should be reasonable when interviewing. But I don't think there's widespread agreement about how to operationalize that. I'd be curious for more detail about how you would do it---you seem to have strong, well-informed feelings on this issue. To my knowledge, there's basically no publicly-available research on tech interview factors and how they correlate with on-the-job performance. The good big employers do this research internally and keep it to themselves. The rest of us are stuck with assumptions, intuitions, logic and argument. So unfortunately I don't think we'll be able to get the debate into the realm of interpreting real data anytime soon.

arethuza 17y ago

I don't know how it works in the US, but in the UK a PhD is usually awarded based purely on research - there is relatively little cutting edge research that I am aware of that hinges on manipulating linked lists. :-)