6 ms·
Your analysis is extremely flawed. I really appreciate your sharing the data with us and I like your service. But, this is a poorly done and a far from subtle
by mendicantB 10y ago
Your analysis is extremely flawed.
I really appreciate your sharing the data with us and I like your service. But, this is a poorly done and a far from subtle plug of your business at the cost of LinkedIn.
1) Technical ability vs # of endorsements
Jesus. Hiding stats that you don't like through aggregations? And please read up on Simspons Paradox, which is clearly the case here just by looking at your plot. Try a basic t-test, or rather some statistical rigor, the next time you try to make conclusions from data.
2) Most endorsed vs Language of Choice
As pointed out, this is not the way to frame your problem. By obfuscating what's happening in your histogram (which isn't technically constructued right either) you are again hiding what you dont like through aggregation. By the way, language matters greatly here, and you'd have benefitted by standardization.
3) Your conclusion
"After running some significance testing, though" and not posting your results or methodology, which is at best questionable after reading your analysis.
Again, I enjoy your service, but blog posts on technical ability that are ironically lacking in technical ability don't really make me want to come back.
PS: A little birdie told me that endorsements are quite strong in predictive power for jobs :)
- closed 10y agoAgreed. This post would be okay if there wasn't so much "we crunched the numbers" talk, and then very perplexing reporting.
- minimaxir 10y ago> And please read up on Simspons Paradox, which is clearly the case here just by looking at your plot. What is the instance of Simpson's paradox (https://en.wikipedia.org/wiki/Simpson's_paradox https://en.wikipedia.org/wiki/Simpson's_paradox) in the scatterplot? There are skews on both X and Y axes, but I don't see disparate trends. Would faceting by preferred language/experience/LinkedIn age show different trends in this context?
- mendicantB 10y agoIf you breakup the plot by any number of categories of technical ability, then there are trends. But to your question, what I was suggesting was that the aggregations done, especially with categorical data that is averaged, are very susceptible to this. And those clusters are reminiscent of situations like this: https://www.researchgate.net/figure/256074671_fig3_Visualization-alone-does-not-always-suffice-A-shows-the-bivariate-relationship-between https://www.researchgate.net/figure/256074671_fig3_Visualiza... I'd more so like to see this analyzed against who got to the next round (their binary signal), or yes against preferred language, which I suspect will be much more telling. The takeaway from that plot is, there is more to the story.
- fnl 10y agoHere's another, even worse example: > It turns out that people’s interview language of choice matched their most endorsed language on LinkedIn just under 50% of the time, so, you know, just slightly worse than flipping a coin. A coin has 2 sides. How many programming languages are there, again?
- huac 10y agoAnd what's the distribution over the languages, in terms of 'preferred interview programming language'?
- leeny 10y agoPeople overwhelmingly chose one language to interview in, so there wasn't much fo a distribution. Unless you mean which languages are most popular?
- xapata 10y ago> which languages are most popular? Is there any other (reasonable) way to interpret that question?
- leeny 10y agoWasn't sure if you were asking that independently of endorsements. Right now, the most heavily used languages on interviewing.io are Java, Python, JavaScript, and C++ (in that order). Ruby, C#, and Go are in the middle. At the bottom are Perl and PHP.
- stingraycharles 10y agoYeah reading that made me realise the whole article is just spam.
- usgroup 10y agolanguage of choice == linkedin endorsed lang -> {TRUE,FALSE} Yet if there was no relationship between the two entities and there were N languages, one would expect the random probability of TRUE to be N/N^2 = 1/N. Although, the writer doesn't seem to allege that he's comparing to random or anything like that.
- pmiller2 10y agoDoes your little birdie have data?
- arcticfox 10y agoSeriously, not sure how you can criticize someone for their methodology and then throw in a super weird appeal to authority with no support
- deleted 10y ago[deleted]
- morgante 10y agoSeriously, I'm not sure how they can even pretend to have reached that conclusion. It would require interviewing random LinkedIn users without endorsements (including ones with no technical ability at all) and then rating them. I certainly think that LinkedIn endorsements aren't exactly a meaningful metric. But they're not meant to be. They're just a simple tool for finding people who can even nominally program.
- Spooky23 10y agoIMO, abusing endorsements like this isn't very useful. At best, it's sort of replaces some attribute of a reference. Endorsement is really feedback to validate the field that somebody works in and is probably not a bozo. If you're interviewing a mid-career programmer, and her endorsements are all for accounting and financial related things, you may have a fit issue to explore. Likewise, if you have a candidate with 200 connections and 3 endorsements, and that's atypical for the industry, you may want to focus more heavily on real reference checking.
- leeny 10y agoAuthor here. I appreciate the notes and am happy to revisit and make corrections when needed. To respond to your points: 1. As a sanity check, I did do a t-test of technical ability vs. # of endorsements before publishing. There is no statistically significant relationship between the 2. (P < 0.335) 2. What do you mean by "language matters here" (re the histogram)?
- closed 10y agoDo you mean that you fit a simple linear model, of the form below? ability = b0 + b1*endorsements + error And when you say t-test, are you saying you did a t-test for the parameter b1? Usually when people refer to a t-test, without more information, they are saying they tested the difference of means between two groups. (or one mean against a number). See, for example, the Wikipedia article on t-tests: https://en.m.wikipedia.org/wiki/Student's_t-test https://en.m.wikipedia.org/wiki/Student's_t-test
- minimaxir 10y ago> Do you mean that you fit a simple linear model, of the form below? That would be the form of the best-fit line in the scatterplot. (and it would make sense to assume that the t-test refers to b1 != 0, as there is only one group)
- closed 10y agoEdit: on second thought, you're probably right. I think I was too off the cuff in responding. Left original response below. If by best fit you mean minimizing sum squared error, that's fair. But to be sure, if someone said t-test, and they only had one group, I would first guess they were doing a one-sample t-test. Even with two dependent variables and one group, I would think over whether they did a dependent t-test. I figured it was a simple linear model (in this case a correlation) because they mentioned that they tested the relationship, and it makes sense, but it seems important to sanity check the use of the term t-test, which can be highly ambiguous (and I have seen used in very surprising ways).
- 10y ago
- nazgul17 10y agoAs someone interested in data science, I am curious to know more: could you elaborate on your points?
- mendicantB 10y agoFirst 200 pages https://www.amazon.com/All-Statistics-Statistical-Inference-Springer/dp/0387402721 https://www.amazon.com/All-Statistics-Statistical-Inference-...
- snoman 10y agoSassy, but I'll allow it.