7 ms·

Why do big companies ask unrealistic software engineering interview questions?

41 points by tomaszs 3y ago

nunoonun 3y ago
Isn't all about proving that the candidate "took the time to study all our questions"? It's about showing that your company is better because people take 6 months studying to apply there! I thought this was a well known.
farnulfo 3y ago
The expert : seven perpendicular red lines https://youtu.be/BKorP55Aqvg?si=dzXY9hRaVkoU3p7z https://youtu.be/BKorP55Aqvg?si=dzXY9hRaVkoU3p7z
valval 3y ago
For largely the same reasons why graduating from top schools is given more credit than any old college, even if the level of teaching is similar. Getting in takes determination, intelligence, and discipline. Same with getting good at Leetcode.
- 4death4 3y ago
  This feels like a retcon. The real reason is most likely that Google started interviewing people this way and then that's the culture that developed in the industry. Once a culture develops, it's very hard to change.
  - iamflimflam1 3y ago
    This is the correct answer. Microsoft used to ask things like "Why are manholes square" or "How many dentists are there in USA". For a while it got trendy to ask those sort of brainteasers in interviews (thankfully it didn't seem to stick). It is very hard to change established processed - especially when the people doing the interviews have gone through the process. They had to suffer, so why should the people after them get an easier ride?
    - FirmwareBurner 3y ago
      >They had to suffer, so why should the people after them get an easier ride? People in the middle ages and WW1 had it pretty bad, why should people after them get an easier ride? Because humanity progresses when we make things easier for the next generations instead to clinging to the 'crabs in a bucket' mentality of perpetuating the cycle of collective suffering and suckage.
    - janosdebugs 3y ago
      I hold a very controvertial take on the dentist question: this and similar questions select candidates who can come up with convincing lines that lack any basis in facts (i.e. bullshit). This seems to be a good foundation for a terrible engineering/company culture.
  - marginalia_nu 3y ago
    It's also noteworthy that if you're building a search engine, these types of questions are actually pretty relevant. As a field Search demands both a breadth and depth in CS fundamentals in a way that rare in most other places.
    - FirmwareBurner 3y ago
      Most Googlers joining Google today aren't gonna be tasked with the elbow grease of building a search engine from scratch, not even close. You'll be a small cog in a large machine, engaging in theatrics justifying in length why your 10h/week of serializing/deserializing JSONS was so difficult and important to the company, while you rest and vest playing videogames at home, asking on Reddit if everyone working in FAANGs is as bored as you.
      - marginalia_nu 3y ago
        That's true, but these practices date from back when Google was a search company. My point being there existed a time when Google asking these questions made a lot of sense.
  - valval 3y ago
    Might be, although frankly we’ll never know.
- epolanski 3y ago
  That still doesn't say anything about the candidate besides the fact that he/she farmed algorithm questions. It's an extremely poor predictor of a candidate's quality. It should only be used in those companies where the number of applicants is way larger than the number of open positions where you can accept the trade off of losing few great applicants if you can also lose the many more bad ones.
  - valval 3y ago
    It took me a while in life to realise that some people are just made to succeed. These people are typically good to great at anything they put their minds to. Just because you say those questions are an extremely poor predictor of a candidate’s quality doesn’t actually make it so. It might just be one of the best and most cost effective ways of finding good candidates.
    - epolanski 3y ago
      You got it the other way around. It's a way to get rid of many false positives even if you lose some false negative. Again, companies like Google can afford to lose amazing devs that can't be bothered to farm leetcode if the trade off is getting rid of those that would not be able to farm those effectively.
pydry 3y ago
Cargoogle culting
- drewcoo 3y ago
  Because "cdrgoogle culting" is just culting . . .
your_friend 3y ago
That's the thing I was struggling with, I can solve many of problems for the avg company – but I don't know how to solve a leetcode problem. I bet there aren't as many companies solving the complex problems on a scale. But if we look from the perspective of the company – maybe it's just an filter to get someone very dedicated and nerdy about programming. Less churn, less questions.
- yitchelle 3y ago
  The thing is that for problems that are mildly complex, it is rarely that it is solved by a single person. So getting a person to solve complex problem under interview conditions is totally unrealistic. If they are really using this as a filter, they are really doing themselves a disservice.
- Kelteseth 3y ago
  Thank god I never had these interview questions at the companies I interviewed for, here in Germany. Most talks were simply a 30min chat with the actual developer there. Mostly about the job, what they do, and some of their current issues. Then they would ask me how I would tackle them.
  - sgu999 3y ago
    It's roughly the same in France, aside from some startup bros who have been told leetcode and brainteasers are the way of unicorns. I think it's in part due to the higher trust we put on credentials like Eng degrees.
    - guappa 3y ago
      I studied with people from many countries and I think the french people were very qualified students in general. There were students from Greece and Iran who, after a bachelor degree in computer science, could not find the highest value in an array.
  - zwnow 3y ago
    Yup, can sign that. Never had a coding interview so far... It is all about sympathy and being knowledgable. Had the most success by being brutaly honest about not knowing certain technologies. But I also show a lot of interest once I do not know something.
huqedato 3y ago
Don't publish anymore on Medium, people!
- sebastianconcpt 3y ago
  I agree and is not because they are making money out of you (that's fine if you agree) it's mostly because they aren't censorship proof or ideologically neutral.
  - FrustratedMonky 3y ago
    Medium started expressly for the money. How to let authors monetize. To criticize it for doing what it was created to do, seems odd. Sure, there are a lot of software packages out there so any dev can create a new Medium. Who cares? The point was to pull a lot of people to one site to monetize it. They would browse around and find other authors, which is not possible if every dev out there creates their own site. Just like Twitch isn't it? Are you saying it is being censored? To what end? What ideology cares about programming. "Those Democrats and their Objects are using the wrong patterns again". "Republicans and their Emacs, holding on to the past".
    - sebastianconcpt 3y ago
      I already mentioned that monetizing is fine if you agree so no point in being triggered by that. Regarding to censorship your reduction to the absurd attempt is making a non-point here. The cultural war is overt, well understood, widely understood at this point and no human discipline is immune to the ideological mind viruses. These might not change the patterns you code but certainly boycott hiring you without feedback, unconfessably, for the revolutionary goals of advancing a power grab agenda based on either true self-deception or lies.
      - FrustratedMonky 3y ago
        Ok. So where in Medium is there censorship? I actually want to know. Is there a group in Medium management that is removing stories? Which direction? As far as I can tell, it might actually lean right, but I can't tell. This is first I've heard Medium criticized for censorship, and while flippant, I am wondering how/why? For monetizing. Maybe I was reacting to a few different posts. I only replied to one, sorry it was yours, but it seemed like a common theme that devs should just roll their own Medium. And I thought that missed the point.
epolanski 3y ago
For the same reason companies use agile or OKRs without ever bothering to do any analysis/thinking on the effects.
_aaed 3y ago
Can't view all the text without signing in to Medium. Even after trying to sign in with Google, I'm repeatedly taken back to the home page. Does anyone have an archive link?
- layer8 3y ago
  Hou can always create your own: https://archive.is/Xk2F4 https://archive.is/Xk2F4
rco8786 3y ago
As always, it's an imperfect proxy. Nothing more, nothing less. But nobody has found a more reliable proxy yet.
- alephnan 3y ago
  > nobody has found a more reliable proxy yet. Companies have not found a process at scale where they can train their employees to systematically gauge candidates. Small and startup companies where leadership is still involved in the interview process can gauge candidates based on their own technical intuition. So it's false to say NOBODY has a proxy for good software engineering skills. The founders can also look at the candidate's public code contributions. The problem is that this method does not scale. If you trusted employees to hire based on feel and intuition, you open up the door for people to bring in incompetent friends. The real problem here is incentives. Founders have a financial incentive to not hire bad candidates. Employees don't, and that's why a standardized process is needed.
  - phreeza 3y ago
    > Small and startup companies where leadership is still involved in the interview process can gauge candidates based on their own technical intuition. So it's false to say NOBODY has a proxy for good software engineering skills. Is there any proof that this method actually works better though? I've heard plenty of stories of people in smallish (30 FTE or so) startups working alongside some really disastrous hires.
    - mike_hearn 3y ago
      Sure. I built a team of ~35 this way and it worked out pretty well. There were very few dud hires, and of those that happened on investigation it was always because the process I put in place wasn't followed (on one occasion it wasn't followed by me!). It got noticed; we had customers telling us they had to deal with us because they couldn't match the talent we'd been able to attract. We had employees leave and come back because they wanted to work in a good team again. The process wasn't anything special and the questions weren't particularly hard. It was very heavily standardized though. No system is perfect. You'll get bad hires no matter what, and miss good hires. The question is only how many.
woliveirajr 3y ago
More or less the same reason why psychologists use the Rorschach test [0] Sometimes they are not worried about the specific answer, they're trying to find out your (software engineer) unconscious mind reveals during the process. [0] https://en.m.wikipedia.org/wiki/Rorschach_test https://en.m.wikipedia.org/wiki/Rorschach_test
- alephnan 3y ago
  > trying to find out your (software engineer) unconscious mind reveals during the process. For many decades, Floyd's Cycle Detection algorithm was unknown in academic circles as the most efficient method of cycle detection. It is not uncommon to encounter this problem in an interview. There's nothing revealing about the unconscious mind and process, other than that the candidate remembered the solution to this problem. If it was so readily obvious to derive and arrive at a solution, you would have expected the army of PhDs and academics to have done so.
prakhar897 3y ago
"Create an account to read the full story." No Thanks.
mike_hearn 3y ago
If you'd like an article giving an explanation for this that doesn't require upgrading to a paid Medium subscription, try this one I wrote a few years ago: https://blog.plan99.net/in-defence-of-the-technical-interview-966f54a58927 https://blog.plan99.net/in-defence-of-the-technical-intervie... It is, natch, hosted on Medium, so you will see a banner, but unlike this paywalled article it's dismissable. Just click the X to make it go away and then you can read the whole thing for free. The summary is that many interviewers ask "unrealistic" algorithmic questions because: 1. They fit in the short amount of time available. 2. They get people writing programs that cover all the basic features of the language. 3. They don't ask people to do excessive amounts of work (e.g. takehome assignments) 4. They wash out people who lack basic skills you'd expect programmers to have, like actually starting a new project and being able to compile/run it in their self-chosen editor. 5. They are general and don't tend to require knowledge of specific frameworks or even specific languages. The questions are unrealistic because they're designed to be fast ways to extract information in an interview setting, not to actually be an accurate sample of the daily work (which in the time available may only cover a fraction of the skills required of a working programmer).
- alephnan 3y ago
  A fizzbuzz question would sufficiently address points 1 thru 5. The issue at hand is that the "unrealistic" algorithm questions aren't fizzbuzz.
  - andruby 3y ago
    There are problems between fizzbuzz and the leet-code algorithms that are small enough to be solvable in 5 to 20 minutes. I like using one of them to complement other non-code questions.
  - mike_hearn 3y ago
    Fizzbuzz was the leetcode of yesteryear. It's exactly the same concept, just a different unrealistic question. The leetcode issue is to a large extent caused by, literally, leetcode.com the website. When I first started doing technical interviewing this website didn't exist, so if you developed a question that wasn't too easy or too hard, covered the bases, fitted well into 45 minutes, that you could calibrate the difficulty of on the fly etc - basically a good question - then you could make it last quite a long time. At some point people started sharing questions and pre-canned solutions online, and many candidates will cheat by looking up answers and copying them from another screen if they can. Also some shady recruiters started leaking questions to candidates ahead of time. That forced interviewers to start constantly inventing new questions. One of the points I used to make in interview training is that questions are like programs, you have to design them. Some are better than others, they can vary in efficiency etc. Often you can't really know how a new question will work out until you try it a few times. Making a good interview experience for the candidate is hard work and it's easy to screw up, lots of interviewers do and it leads to a lot of bitterness (e.g. questions that are far too hard or specific or don't fit in the available time). Unfortunately if you're constantly having to invent new questions to keep ahead of the people sharing them and grinding the process, then the quality will inherently get way more variable. By the time you've calibrated the question it's been leaked already. And there are a finite number of programs that it's reasonable to ask someone to write in a short amount of time, so by now these giant databases of interview questions and candidates memorizing the answers at scale makes it harder to get accurate information, which was the only goal in the end. Because indeed the specific tasks themselves don't matter much.
hipadev23 3y ago
I couldn’t read the entire article due to Medium, but this author sounds like the kind of guy who’d invent a jump to conclusions mat
commandersaki 3y ago
Couldn't read the article due to pay/registration wall. If this is about screening potential software engineers with programming questions during an interview, hasn't this always been a thing even before Google popularised it? I started my career after the dot-com era but it was in Systems Administration, so I didn't get to experience any programming style interviews until I changed careers and attempted an interview in 2008 with Google. But hasn't this been the standard modus operandi for big tech companies like Microsoft, Sun, Oracle, IBM, etc. even in the 90s? I recall reading this [1] article from Casey Muratori about programming questions for a Software Engineering intern at Microsoft. I'm not against this style of interviewing, but I do also think that some questions can be absurd or unnecessarily tricky. I've had my fair share of programming interview questions, and I found my solve rate to be around 3/7 for my last interview with Google back in 2015. Some of the questions posed were really tricky, and I just don't have the ability to solve in a timely manner without properly experimenting with the problem. From my impression and perspective, interviewers would sometimes choose questions with high coolness/leetness factor instead of choosing something more practical for a 30-50 minute session. [1]: https://www.computerenhance.com/p/the-four-programming-questions-from https://www.computerenhance.com/p/the-four-programming-quest...
RcouF1uZ4gsC 3y ago
Same reason the NFL combine has athletes do the bench press and the 40 yard dash. They aren’t going to do a bench press in an NFL game. Neither are they going to a straight 40 yard dash. However, those activities tend to reveal information about their abilities that are useful in an actual game. Similarly, while you may not be doing those coding problems in day to day development, those coding problems can (ideally) reveal information about your knowledge and abilities in hat does apply to daily coding.
annabyrd 3y ago
These tests and questions are often in place so people feel they can manage the risk/uncertainty of hiring. Some studies show that previous performance is the key indicator of future performance. Interviewing and testing in the hiring process are negligible at best.
can3p 3y ago
The answer is usually self selection and calibration at scale. Self selection is about people looking like minded colleagues. If everybody is crazy about solving riddles and participates in programming competitions that's what they would be looking for. Now, about the scale. Anything run at scale needs standardisation. You need to hire 100 senior developers. How do you know they have more or less same level? If every single interview is hand crafted, you'll either need to get all devs envolved (and not everybody is good at coming up with interview questions) or get standard questions and answers to allow a smaller group of people to deal with candidates and coding puzzles fit great there because they're slef contained and isolated. Every realistic question has multiple facets and that's what you get at the system design interview step. Another problem I've observed was that the more you give the same puzzles to candidates, the easier they look to you. What the means is that you as an interviewer either need to keep your self in check regarding your reactions or you're risking giving a bad interview score in situations where it's not really warranted or there is a chance that the next question you take will be the one you find more complex (to be equal to position level in your head) and that would push the level of questions up. That's another observation I had when some interview questions would become so complex that I knew some of the existing devs would fail the interview for sure. Does that give good results on the other end? I don't know, but what we definitely know is that there is the whole industry around leetcode to train peeople to pass these challenges specifically and that means that they only thing the interviewer know is that the candidate has put some effort to prepare for the interview meaning they're motivated to join the company. And maybe it's not the worst data point! Some companies actually explicitly mention this fact on their hiring pages. To add to it, big corps have different ways to make interviews objective in whatever sense they think and that by definition reduces the personal impact. "Why did you ask this question? We've never done it before, we'll need to have a group call to calibrate". After a handful conversations like this you'll just stick to the standard process. Is there a way to come up with a more human approach? Personal recommendations with some skin in the game I guess. I'm sure in some niche areas like browser engines all good devs know each other well and no interview is often necessary.
ochronus 3y ago
Because they have a different problem compared to smaller ones: a huge influx of applicants, and this is an easy way to filter and standardize process across a loooot of interviewers. Is it perfect? Hell no. Does it result in false positives/negatives? Sure. Is it OK for them, considering the volume? Yup.
cturner 3y ago
Could not read the whole article but get the gist. Time was that I would have agreed. But I have changed position. Candidates: you need to appreciate how difficult it is to hire good people, the dishonesty of many candidates, and how limited the tools are for identifying talent and filtering out weasels. It is true: party trick Big O questions have low utility in most jobs. It is true: there is talent out there which sucks at Big-O questions. It is true: there are people who are good at little else than the party tricks. Nevertheless: those questions provide an objective judgement about a candidate. That objective judgement is valuable. You spent several years doing a degree to make yourself employment-compatible. You will be able to achieve this standard in far less time, and there are cash rewards at the end of it. Accept that you will need to burn 1-2 months to develop this capability, and that you will feel awesome once you have. Get /Cracking the Coding Interview/. I was not impressed by the object-oriented design chapter, but seek to master the other first-11 chapters. Take what you need from the remaining chapters. Develop a set of flashcards. Find problems online. Work intensively for at least two weeks but ideally a month. Then book some interviews at firms where you don't mind if you fail. Some of the FAANGs have awesome pre-interview study material. Try to get into process with a firm like that. Keep training on your flashcards and doing a few problems a day to keep in form. Expect to struggle with early interviews. Learn from those experiences. Revise what you made a mess of, keep working with your flashcards until you have mastered your weak areas. Now start applying at the firms where you want to succeed.
- guappa 3y ago
  Yes the candidates are aware that interviews are terrible. But I seem to understand from your comment that we should just all accept that they are terrible and keep doing things the same way?
  - deleted 3y ago
    [deleted]
  - karmakurtisaani 3y ago
    Pretty much yes. Unless you're the head of HR at a big corporation, you have pretty much no say on how it's done.
neonate 3y ago
https://archive.md/Xk2F4 https://archive.md/Xk2F4
dave333 3y ago
I'd far rather be subjected to 15 bizarre questions than be subjected to 5 rounds of interviews with everyone. Some questions you will get right and a few you may have heard before. Add a little luck and you have a job.

41 points by tomaszs 3y ago

Isn't all about proving that the candidate "took the time to study all our questions"? It's about showing that your company is better because people take 6 months studying to apply there! I thought this was a well known.

farnulfo 3y ago

The expert : seven perpendicular red lines https://youtu.be/BKorP55Aqvg?si=dzXY9hRaVkoU3p7z https://youtu.be/BKorP55Aqvg?si=dzXY9hRaVkoU3p7z

valval 3y ago

For largely the same reasons why graduating from top schools is given more credit than any old college, even if the level of teaching is similar. Getting in takes determination, intelligence, and discipline. Same with getting good at Leetcode.

4death4 3y ago
This feels like a retcon. The real reason is most likely that Google started interviewing people this way and then that's the culture that developed in the industry. Once a culture develops, it's very hard to change.
- iamflimflam1 3y ago
  This is the correct answer. Microsoft used to ask things like "Why are manholes square" or "How many dentists are there in USA". For a while it got trendy to ask those sort of brainteasers in interviews (thankfully it didn't seem to stick). It is very hard to change established processed - especially when the people doing the interviews have gone through the process. They had to suffer, so why should the people after them get an easier ride?
  - FirmwareBurner 3y ago
    >They had to suffer, so why should the people after them get an easier ride? People in the middle ages and WW1 had it pretty bad, why should people after them get an easier ride? Because humanity progresses when we make things easier for the next generations instead to clinging to the 'crabs in a bucket' mentality of perpetuating the cycle of collective suffering and suckage.
  - janosdebugs 3y ago
    I hold a very controvertial take on the dentist question: this and similar questions select candidates who can come up with convincing lines that lack any basis in facts (i.e. bullshit). This seems to be a good foundation for a terrible engineering/company culture.
- marginalia_nu 3y ago
  It's also noteworthy that if you're building a search engine, these types of questions are actually pretty relevant. As a field Search demands both a breadth and depth in CS fundamentals in a way that rare in most other places.
  - FirmwareBurner 3y ago
    Most Googlers joining Google today aren't gonna be tasked with the elbow grease of building a search engine from scratch, not even close. You'll be a small cog in a large machine, engaging in theatrics justifying in length why your 10h/week of serializing/deserializing JSONS was so difficult and important to the company, while you rest and vest playing videogames at home, asking on Reddit if everyone working in FAANGs is as bored as you.
    - marginalia_nu 3y ago
      That's true, but these practices date from back when Google was a search company. My point being there existed a time when Google asking these questions made a lot of sense.
- valval 3y ago
  Might be, although frankly we’ll never know.
epolanski 3y ago
That still doesn't say anything about the candidate besides the fact that he/she farmed algorithm questions. It's an extremely poor predictor of a candidate's quality. It should only be used in those companies where the number of applicants is way larger than the number of open positions where you can accept the trade off of losing few great applicants if you can also lose the many more bad ones.
- valval 3y ago
  It took me a while in life to realise that some people are just made to succeed. These people are typically good to great at anything they put their minds to. Just because you say those questions are an extremely poor predictor of a candidate’s quality doesn’t actually make it so. It might just be one of the best and most cost effective ways of finding good candidates.
  - epolanski 3y ago
    You got it the other way around. It's a way to get rid of many false positives even if you lose some false negative. Again, companies like Google can afford to lose amazing devs that can't be bothered to farm leetcode if the trade off is getting rid of those that would not be able to farm those effectively.

pydry 3y ago

Cargoogle culting

drewcoo 3y ago
Because "cdrgoogle culting" is just culting . . .

your_friend 3y ago

That's the thing I was struggling with, I can solve many of problems for the avg company – but I don't know how to solve a leetcode problem. I bet there aren't as many companies solving the complex problems on a scale. But if we look from the perspective of the company – maybe it's just an filter to get someone very dedicated and nerdy about programming. Less churn, less questions.

yitchelle 3y ago
The thing is that for problems that are mildly complex, it is rarely that it is solved by a single person. So getting a person to solve complex problem under interview conditions is totally unrealistic. If they are really using this as a filter, they are really doing themselves a disservice.
Kelteseth 3y ago
Thank god I never had these interview questions at the companies I interviewed for, here in Germany. Most talks were simply a 30min chat with the actual developer there. Mostly about the job, what they do, and some of their current issues. Then they would ask me how I would tackle them.
- sgu999 3y ago
  It's roughly the same in France, aside from some startup bros who have been told leetcode and brainteasers are the way of unicorns. I think it's in part due to the higher trust we put on credentials like Eng degrees.
  - guappa 3y ago
    I studied with people from many countries and I think the french people were very qualified students in general. There were students from Greece and Iran who, after a bachelor degree in computer science, could not find the highest value in an array.
- zwnow 3y ago
  Yup, can sign that. Never had a coding interview so far... It is all about sympathy and being knowledgable. Had the most success by being brutaly honest about not knowing certain technologies. But I also show a lot of interest once I do not know something.

huqedato 3y ago

Don't publish anymore on Medium, people!

sebastianconcpt 3y ago
I agree and is not because they are making money out of you (that's fine if you agree) it's mostly because they aren't censorship proof or ideologically neutral.
- FrustratedMonky 3y ago
  Medium started expressly for the money. How to let authors monetize. To criticize it for doing what it was created to do, seems odd. Sure, there are a lot of software packages out there so any dev can create a new Medium. Who cares? The point was to pull a lot of people to one site to monetize it. They would browse around and find other authors, which is not possible if every dev out there creates their own site. Just like Twitch isn't it? Are you saying it is being censored? To what end? What ideology cares about programming. "Those Democrats and their Objects are using the wrong patterns again". "Republicans and their Emacs, holding on to the past".
  - sebastianconcpt 3y ago
    I already mentioned that monetizing is fine if you agree so no point in being triggered by that. Regarding to censorship your reduction to the absurd attempt is making a non-point here. The cultural war is overt, well understood, widely understood at this point and no human discipline is immune to the ideological mind viruses. These might not change the patterns you code but certainly boycott hiring you without feedback, unconfessably, for the revolutionary goals of advancing a power grab agenda based on either true self-deception or lies.
    - FrustratedMonky 3y ago
      Ok. So where in Medium is there censorship? I actually want to know. Is there a group in Medium management that is removing stories? Which direction? As far as I can tell, it might actually lean right, but I can't tell. This is first I've heard Medium criticized for censorship, and while flippant, I am wondering how/why? For monetizing. Maybe I was reacting to a few different posts. I only replied to one, sorry it was yours, but it seemed like a common theme that devs should just roll their own Medium. And I thought that missed the point.

epolanski 3y ago

For the same reason companies use agile or OKRs without ever bothering to do any analysis/thinking on the effects.

_aaed 3y ago

Can't view all the text without signing in to Medium. Even after trying to sign in with Google, I'm repeatedly taken back to the home page. Does anyone have an archive link?

layer8 3y ago
Hou can always create your own: https://archive.is/Xk2F4 https://archive.is/Xk2F4

rco8786 3y ago

As always, it's an imperfect proxy. Nothing more, nothing less. But nobody has found a more reliable proxy yet.

alephnan 3y ago
> nobody has found a more reliable proxy yet. Companies have not found a process at scale where they can train their employees to systematically gauge candidates. Small and startup companies where leadership is still involved in the interview process can gauge candidates based on their own technical intuition. So it's false to say NOBODY has a proxy for good software engineering skills. The founders can also look at the candidate's public code contributions. The problem is that this method does not scale. If you trusted employees to hire based on feel and intuition, you open up the door for people to bring in incompetent friends. The real problem here is incentives. Founders have a financial incentive to not hire bad candidates. Employees don't, and that's why a standardized process is needed.
- phreeza 3y ago
  > Small and startup companies where leadership is still involved in the interview process can gauge candidates based on their own technical intuition. So it's false to say NOBODY has a proxy for good software engineering skills. Is there any proof that this method actually works better though? I've heard plenty of stories of people in smallish (30 FTE or so) startups working alongside some really disastrous hires.
  - mike_hearn 3y ago
    Sure. I built a team of ~35 this way and it worked out pretty well. There were very few dud hires, and of those that happened on investigation it was always because the process I put in place wasn't followed (on one occasion it wasn't followed by me!). It got noticed; we had customers telling us they had to deal with us because they couldn't match the talent we'd been able to attract. We had employees leave and come back because they wanted to work in a good team again. The process wasn't anything special and the questions weren't particularly hard. It was very heavily standardized though. No system is perfect. You'll get bad hires no matter what, and miss good hires. The question is only how many.

woliveirajr 3y ago

More or less the same reason why psychologists use the Rorschach test [0] Sometimes they are not worried about the specific answer, they're trying to find out your (software engineer) unconscious mind reveals during the process. [0] https://en.m.wikipedia.org/wiki/Rorschach_test https://en.m.wikipedia.org/wiki/Rorschach_test

alephnan 3y ago
> trying to find out your (software engineer) unconscious mind reveals during the process. For many decades, Floyd's Cycle Detection algorithm was unknown in academic circles as the most efficient method of cycle detection. It is not uncommon to encounter this problem in an interview. There's nothing revealing about the unconscious mind and process, other than that the candidate remembered the solution to this problem. If it was so readily obvious to derive and arrive at a solution, you would have expected the army of PhDs and academics to have done so.

prakhar897 3y ago

"Create an account to read the full story." No Thanks.

mike_hearn 3y ago

If you'd like an article giving an explanation for this that doesn't require upgrading to a paid Medium subscription, try this one I wrote a few years ago: https://blog.plan99.net/in-defence-of-the-technical-interview-966f54a58927 https://blog.plan99.net/in-defence-of-the-technical-intervie... It is, natch, hosted on Medium, so you will see a banner, but unlike this paywalled article it's dismissable. Just click the X to make it go away and then you can read the whole thing for free. The summary is that many interviewers ask "unrealistic" algorithmic questions because: 1. They fit in the short amount of time available. 2. They get people writing programs that cover all the basic features of the language. 3. They don't ask people to do excessive amounts of work (e.g. takehome assignments) 4. They wash out people who lack basic skills you'd expect programmers to have, like actually starting a new project and being able to compile/run it in their self-chosen editor. 5. They are general and don't tend to require knowledge of specific frameworks or even specific languages. The questions are unrealistic because they're designed to be fast ways to extract information in an interview setting, not to actually be an accurate sample of the daily work (which in the time available may only cover a fraction of the skills required of a working programmer).

alephnan 3y ago
A fizzbuzz question would sufficiently address points 1 thru 5. The issue at hand is that the "unrealistic" algorithm questions aren't fizzbuzz.
- andruby 3y ago
  There are problems between fizzbuzz and the leet-code algorithms that are small enough to be solvable in 5 to 20 minutes. I like using one of them to complement other non-code questions.
- mike_hearn 3y ago
  Fizzbuzz was the leetcode of yesteryear. It's exactly the same concept, just a different unrealistic question. The leetcode issue is to a large extent caused by, literally, leetcode.com the website. When I first started doing technical interviewing this website didn't exist, so if you developed a question that wasn't too easy or too hard, covered the bases, fitted well into 45 minutes, that you could calibrate the difficulty of on the fly etc - basically a good question - then you could make it last quite a long time. At some point people started sharing questions and pre-canned solutions online, and many candidates will cheat by looking up answers and copying them from another screen if they can. Also some shady recruiters started leaking questions to candidates ahead of time. That forced interviewers to start constantly inventing new questions. One of the points I used to make in interview training is that questions are like programs, you have to design them. Some are better than others, they can vary in efficiency etc. Often you can't really know how a new question will work out until you try it a few times. Making a good interview experience for the candidate is hard work and it's easy to screw up, lots of interviewers do and it leads to a lot of bitterness (e.g. questions that are far too hard or specific or don't fit in the available time). Unfortunately if you're constantly having to invent new questions to keep ahead of the people sharing them and grinding the process, then the quality will inherently get way more variable. By the time you've calibrated the question it's been leaked already. And there are a finite number of programs that it's reasonable to ask someone to write in a short amount of time, so by now these giant databases of interview questions and candidates memorizing the answers at scale makes it harder to get accurate information, which was the only goal in the end. Because indeed the specific tasks themselves don't matter much.

hipadev23 3y ago

I couldn’t read the entire article due to Medium, but this author sounds like the kind of guy who’d invent a jump to conclusions mat

commandersaki 3y ago

Couldn't read the article due to pay/registration wall. If this is about screening potential software engineers with programming questions during an interview, hasn't this always been a thing even before Google popularised it? I started my career after the dot-com era but it was in Systems Administration, so I didn't get to experience any programming style interviews until I changed careers and attempted an interview in 2008 with Google. But hasn't this been the standard modus operandi for big tech companies like Microsoft, Sun, Oracle, IBM, etc. even in the 90s? I recall reading this [1] article from Casey Muratori about programming questions for a Software Engineering intern at Microsoft. I'm not against this style of interviewing, but I do also think that some questions can be absurd or unnecessarily tricky. I've had my fair share of programming interview questions, and I found my solve rate to be around 3/7 for my last interview with Google back in 2015. Some of the questions posed were really tricky, and I just don't have the ability to solve in a timely manner without properly experimenting with the problem. From my impression and perspective, interviewers would sometimes choose questions with high coolness/leetness factor instead of choosing something more practical for a 30-50 minute session. [1]: https://www.computerenhance.com/p/the-four-programming-questions-from https://www.computerenhance.com/p/the-four-programming-quest...

RcouF1uZ4gsC 3y ago

Same reason the NFL combine has athletes do the bench press and the 40 yard dash. They aren’t going to do a bench press in an NFL game. Neither are they going to a straight 40 yard dash. However, those activities tend to reveal information about their abilities that are useful in an actual game. Similarly, while you may not be doing those coding problems in day to day development, those coding problems can (ideally) reveal information about your knowledge and abilities in hat does apply to daily coding.

annabyrd 3y ago

These tests and questions are often in place so people feel they can manage the risk/uncertainty of hiring. Some studies show that previous performance is the key indicator of future performance. Interviewing and testing in the hiring process are negligible at best.

can3p 3y ago

The answer is usually self selection and calibration at scale. Self selection is about people looking like minded colleagues. If everybody is crazy about solving riddles and participates in programming competitions that's what they would be looking for. Now, about the scale. Anything run at scale needs standardisation. You need to hire 100 senior developers. How do you know they have more or less same level? If every single interview is hand crafted, you'll either need to get all devs envolved (and not everybody is good at coming up with interview questions) or get standard questions and answers to allow a smaller group of people to deal with candidates and coding puzzles fit great there because they're slef contained and isolated. Every realistic question has multiple facets and that's what you get at the system design interview step. Another problem I've observed was that the more you give the same puzzles to candidates, the easier they look to you. What the means is that you as an interviewer either need to keep your self in check regarding your reactions or you're risking giving a bad interview score in situations where it's not really warranted or there is a chance that the next question you take will be the one you find more complex (to be equal to position level in your head) and that would push the level of questions up. That's another observation I had when some interview questions would become so complex that I knew some of the existing devs would fail the interview for sure. Does that give good results on the other end? I don't know, but what we definitely know is that there is the whole industry around leetcode to train peeople to pass these challenges specifically and that means that they only thing the interviewer know is that the candidate has put some effort to prepare for the interview meaning they're motivated to join the company. And maybe it's not the worst data point! Some companies actually explicitly mention this fact on their hiring pages. To add to it, big corps have different ways to make interviews objective in whatever sense they think and that by definition reduces the personal impact. "Why did you ask this question? We've never done it before, we'll need to have a group call to calibrate". After a handful conversations like this you'll just stick to the standard process. Is there a way to come up with a more human approach? Personal recommendations with some skin in the game I guess. I'm sure in some niche areas like browser engines all good devs know each other well and no interview is often necessary.

ochronus 3y ago

Because they have a different problem compared to smaller ones: a huge influx of applicants, and this is an easy way to filter and standardize process across a loooot of interviewers. Is it perfect? Hell no. Does it result in false positives/negatives? Sure. Is it OK for them, considering the volume? Yup.

cturner 3y ago

Could not read the whole article but get the gist. Time was that I would have agreed. But I have changed position. Candidates: you need to appreciate how difficult it is to hire good people, the dishonesty of many candidates, and how limited the tools are for identifying talent and filtering out weasels. It is true: party trick Big O questions have low utility in most jobs. It is true: there is talent out there which sucks at Big-O questions. It is true: there are people who are good at little else than the party tricks. Nevertheless: those questions provide an objective judgement about a candidate. That objective judgement is valuable. You spent several years doing a degree to make yourself employment-compatible. You will be able to achieve this standard in far less time, and there are cash rewards at the end of it. Accept that you will need to burn 1-2 months to develop this capability, and that you will feel awesome once you have. Get /Cracking the Coding Interview/. I was not impressed by the object-oriented design chapter, but seek to master the other first-11 chapters. Take what you need from the remaining chapters. Develop a set of flashcards. Find problems online. Work intensively for at least two weeks but ideally a month. Then book some interviews at firms where you don't mind if you fail. Some of the FAANGs have awesome pre-interview study material. Try to get into process with a firm like that. Keep training on your flashcards and doing a few problems a day to keep in form. Expect to struggle with early interviews. Learn from those experiences. Revise what you made a mess of, keep working with your flashcards until you have mastered your weak areas. Now start applying at the firms where you want to succeed.

guappa 3y ago
Yes the candidates are aware that interviews are terrible. But I seem to understand from your comment that we should just all accept that they are terrible and keep doing things the same way?
- deleted 3y ago
  [deleted]
- karmakurtisaani 3y ago
  Pretty much yes. Unless you're the head of HR at a big corporation, you have pretty much no say on how it's done.

neonate 3y ago

https://archive.md/Xk2F4 https://archive.md/Xk2F4

dave333 3y ago

I'd far rather be subjected to 15 bizarre questions than be subjected to 5 rounds of interviews with everyone. Some questions you will get right and a few you may have heard before. Add a little luck and you have a job.