5 ms·

Well, empirically we're getting quite a lot of people attempting it! :) Three hours is pretty lenient (you could blaze through it in less time), though the last

by balajis 12y ago

Well, empirically we're getting quite a lot of people attempting it! :) Three hours is pretty lenient (you could blaze through it in less time), though the last question is challenging. We think the questions are kind of fun as well. Without something like this we'd just have 100s-1000s of unstructured resumes in different formats to rank against each other. This process is more fair and (IMO) allows someone without a college degree, or from outside the US, to compete on a level playing field.

jay_kyburz 12y ago
Do you mind if people do the test with no interest in the job?
- balajis 12y ago
  Sure, go ahead! We just ask that you not share the questions with others after completing it.
7Figures2Commas 12y ago
> Well, empirically we're getting quite a lot of people attempting it! > Without something like this we'd just have 100s-1000s of unstructured resumes in different formats to rank against each other. So instead of hundreds or thousands of unstructured resumes in different formats, you have hundreds or thousands of completed programming tests. Even assuming that you automatically filter out submissions that fail testing, given that there are a lot of ways to solve a problem, you'd still be stuck evaluating potentially hundreds of correct submissions. More importantly, a test doesn't seem to be aligned with your top priority: > ...but the most important requirement is significant independent programming experience as demonstrated by your GitHub account, personal projects, academic publications, or startup success. Are you assuming that all of the folks most accomplished in the real world are going to complete your programming test successfully? Since you ultimately seem to be interested in folks who can build dashboards, I would think you'd be far better off asking candidates to build a dashboard prototype using sample data you provide. There are lot of really smart developers out there with advanced degrees who could solve programming quizzes all day long but couldn't design and implement a useful dashboard application.
- dlss 12y ago
  > You'd still be stuck evaluating potentially hundreds of correct submissions. I'm guessing the MVP of hackerrank was the web code editor + lint/findbugs/whatever for filtering. A lot of people have spent a lot of time writing open source projects that convert code to metrics, so making a reasonably prioritized list is probably fairly easy. > There are lot of really smart developers out there with advanced degrees who could solve programming quizzes all day long but couldn't design and implement a useful dashboard application. Agree! I think this is a great critique of HackerRank as a business idea -- while they are new, it's just a hurdle that weeds the busier coders out of your hiring pool... if they become established there will be books on cramming for it, and will no longer separate the wheat from the chaff. (ie https://web.archive.org/web/20130114163457/https://raganwald.posterous.com/i-dont-hire-unlucky-people https://web.archive.org/web/20130114163457/https://raganwald... meets http://en.wikipedia.org/wiki/Goodhart's_law http://en.wikipedia.org/wiki/Goodhart's_law)
  - lifeisstillgood 12y ago
    To be honest I think it's the other way round - if you can do programming tests all day long you will succeed at most coding jobs. However there are a lot of developers who can't tackle an interview that with a nights sleep and no pressure can easily code a intuitive and effective architecture. Sorry - programming tests raise far more false negatives than false positives (and this from a person who considers himself a false negative in most tests)
    - dlss 12y ago
      I'm not sure what kind of coding work you do, but I assure you the vast majority of web development work has little resemblance to coding questions. You are almost never designing an algorithm: that was already done for you by whoever designed the library you are using. Majority of the tasks IME: - code reading & refactoring - figuring out why two libraries aren't playing nice - bug triangulation - optimization (not the algorithmic type, but rather the fix-the-auto-generated-SQL-query / frontend asset packing kind) - repetitive/simple/obvious bits of jQuery Maybe there's some kind of online test that could exercise those skills, but if you ask me the main performance changers for modern web development are, they're: really knowing the stack (problem anticipation, what to use, how to use the various tools to quickly get search keywords), google fu, and most importantly having done it a couple hundred times (muscle memory).
abalone 12y ago
> Well, empirically we're getting quite a lot of people attempting it! I think you missed my point. It's that the people submitting to a multi-hour exam just to submit a resume are more desperate / value their time less than those that can stand on their experience. The fact that there a "quite a lot" of submissions only worsens the probability that the multi-hour effort will be wasted.
zodiac 12y ago
>> allows someone ... from outside the US, to compete on a level playing field. Just curious, why do you say that the normal process doesn't allow people outside the US to compete on a level playing field? Higher barrier to on-site interviews?
emsy 12y ago
The questions were pretty good, because they were close to the real world and you couldn't google them so easily. The programming tasks were rather unbalanced. The first two were super easy, while you can easily get stuck on the last one, even as a good programmer. I knew that I wouldn't be able to answer it today, so I skipped it after a short while. If I'd plan such a test, I'd try to get an example of the structure and comprehensibility of my future employees work.

by balajis 12y ago

jay_kyburz 12y ago

Do you mind if people do the test with no interest in the job?

balajis 12y ago
Sure, go ahead! We just ask that you not share the questions with others after completing it.

7Figures2Commas 12y ago

> Well, empirically we're getting quite a lot of people attempting it! > Without something like this we'd just have 100s-1000s of unstructured resumes in different formats to rank against each other. So instead of hundreds or thousands of unstructured resumes in different formats, you have hundreds or thousands of completed programming tests. Even assuming that you automatically filter out submissions that fail testing, given that there are a lot of ways to solve a problem, you'd still be stuck evaluating potentially hundreds of correct submissions. More importantly, a test doesn't seem to be aligned with your top priority: > ...but the most important requirement is significant independent programming experience as demonstrated by your GitHub account, personal projects, academic publications, or startup success. Are you assuming that all of the folks most accomplished in the real world are going to complete your programming test successfully? Since you ultimately seem to be interested in folks who can build dashboards, I would think you'd be far better off asking candidates to build a dashboard prototype using sample data you provide. There are lot of really smart developers out there with advanced degrees who could solve programming quizzes all day long but couldn't design and implement a useful dashboard application.

dlss 12y ago
> You'd still be stuck evaluating potentially hundreds of correct submissions. I'm guessing the MVP of hackerrank was the web code editor + lint/findbugs/whatever for filtering. A lot of people have spent a lot of time writing open source projects that convert code to metrics, so making a reasonably prioritized list is probably fairly easy. > There are lot of really smart developers out there with advanced degrees who could solve programming quizzes all day long but couldn't design and implement a useful dashboard application. Agree! I think this is a great critique of HackerRank as a business idea -- while they are new, it's just a hurdle that weeds the busier coders out of your hiring pool... if they become established there will be books on cramming for it, and will no longer separate the wheat from the chaff. (ie https://web.archive.org/web/20130114163457/https://raganwald.posterous.com/i-dont-hire-unlucky-people https://web.archive.org/web/20130114163457/https://raganwald... meets http://en.wikipedia.org/wiki/Goodhart's_law http://en.wikipedia.org/wiki/Goodhart's_law)
- lifeisstillgood 12y ago
  To be honest I think it's the other way round - if you can do programming tests all day long you will succeed at most coding jobs. However there are a lot of developers who can't tackle an interview that with a nights sleep and no pressure can easily code a intuitive and effective architecture. Sorry - programming tests raise far more false negatives than false positives (and this from a person who considers himself a false negative in most tests)
  - dlss 12y ago
    I'm not sure what kind of coding work you do, but I assure you the vast majority of web development work has little resemblance to coding questions. You are almost never designing an algorithm: that was already done for you by whoever designed the library you are using. Majority of the tasks IME: - code reading & refactoring - figuring out why two libraries aren't playing nice - bug triangulation - optimization (not the algorithmic type, but rather the fix-the-auto-generated-SQL-query / frontend asset packing kind) - repetitive/simple/obvious bits of jQuery Maybe there's some kind of online test that could exercise those skills, but if you ask me the main performance changers for modern web development are, they're: really knowing the stack (problem anticipation, what to use, how to use the various tools to quickly get search keywords), google fu, and most importantly having done it a couple hundred times (muscle memory).

abalone 12y ago

> Well, empirically we're getting quite a lot of people attempting it! I think you missed my point. It's that the people submitting to a multi-hour exam just to submit a resume are more desperate / value their time less than those that can stand on their experience. The fact that there a "quite a lot" of submissions only worsens the probability that the multi-hour effort will be wasted.

zodiac 12y ago

>> allows someone ... from outside the US, to compete on a level playing field. Just curious, why do you say that the normal process doesn't allow people outside the US to compete on a level playing field? Higher barrier to on-site interviews?

emsy 12y ago

The questions were pretty good, because they were close to the real world and you couldn't google them so easily. The programming tasks were rather unbalanced. The first two were super easy, while you can easily get stuck on the last one, even as a good programmer. I knew that I wouldn't be able to answer it today, so I skipped it after a short while. If I'd plan such a test, I'd try to get an example of the structure and comprehensibility of my future employees work.