8 ms·
Unpopular opinion… but IMO almost all of Karpathy’s fame an influence come from being an incredible educator and communicator. Relative to his level of fame, h
by dontreact 3y ago
Unpopular opinion… but IMO almost all of Karpathy’s fame an influence come from being an incredible educator and communicator.
Relative to his level of fame, his actual level of contribution as far as pushing forward AI, I’m not so sure about.
I deeply appreciate his educational content and I’m glad that it has led to a way for him to gain influence and sustain a career. Hopefully he’s rich enough from that that he can focus 100% on educational stuff!
- georgehill 3y ago> Relative to his level of fame, his actual level of contribution as far as pushing forward AI, I’m not so sure about. Are you sure about your perspective? https://scholar.google.com/citations?view_op=view_citation&hl=en&user=l8WuQJgAAAAJ&citation_for_view=l8WuQJgAAAAJ:UeHWp8X0CEIC https://scholar.google.com/citations?view_op=view_citation&h...
- the_arun 3y agoMay be this is what you wanted to share? https://scholar.google.com/citations?user=l8WuQJgAAAAJ&hl=en https://scholar.google.com/citations?user=l8WuQJgAAAAJ&hl=en
- iaseiadit 3y agoImageNet was very influential, but this just shows he was eighth author on a twelve author paper from almost a decade ago. Is there better evidence of sustained contributions to the field?
- laborcontract 3y agoHm, well, I see on his resume that he was a founder of OpenAI, recruited to be Tesla's head of AI, went back to OpenAI, and also has the most viewed educational videos in this space. So, he has made theoretical contributions to the space, contributions to prominent private organizations in the space, and broadly educated others about the space. What more are you looking for?
- roflulz 3y agothe commenter is probably a junior boot camp web dev...
- paganel 3y agoTesla fumbled big on AI, and as for his work at OpenAI, he just left, had he been good enough they would have made him a financial offer that would have made him continue. But, I'll give him that, he seems to be a really good teacher.
- literalAardvark 3y agoI doubt he left because he wasn't being compensated fairly. People just get bored and go do something else for a while sometimes. Or he's got some beef.
- hef19898 3y agoNot everyone is purely motivated by money so. I know that the moment I decided to quit or switch jobs, no, and I mean litterally no, amount of money would change my mind. Me changing can never be used as an appraisal of my old organisation so. Disclaimer: regarding money, if I get enough in max a year to rezire forever after that, I might be tempted. Which won't happen, because a) I'd just leave a year later anyway and b) nobody would pay me high 7 figures just to not quit.
- rangestransform 3y agomany SOTA papers for multi-camera deep fusion/birds eye view perception in autonomous driving were based on copying teslas homework after their AI day 2022 talk
- slater- 3y agoYou left out a few prepositions like off the space, toward the space, from the space, and down the space.
- sjwhevvvvvsj 3y agoHe’s first author on a ton of those papers. That’s a tenure worthy CV almost anywhere. Gimme a break.
- iaseiadit 3y agoOP shared a link to just the Imagenet paper. Agree he had a decent overall track record at Stanford, but that’s not how tenure works — it might have got his foot in the door as an assistant professor somewhere. He chose a much more lucrative path.
- nblgbg 3y agoIts IMHO too. His contribution to educational content is incredible, and very few individuals have the ability to explain things the way he does. However, I am also unsure about his contribution to the field itself. It is a side effect of working in the industry on the product side. You don't have a chance to publish papers, and you don't want to reveal your secrets or bugs to everyone.
- deleted 3y ago[deleted]
- abadpoli 3y agoEducation and communication is important. It brings new people into the field, and helps grow those that are already part of the field, both of which are essential to long term growth and progress. Using phrases like “actual contribution” to refer to non-educational acts is entirely dismissive to the role that great educators play to in the march for progress. Where would you be today if such education was unavailable? He contributed to pushing forward AI, no “actual” about it. The loss of a great educator should be viewed with just as much sadness as the loss of a great engineer.
- dontreact 3y agoHis job at Tesla or OpenAI wasn’t as an educator though. I think a clearer version of my point is that most of his impact has come from activities he has done “on the side” and hasn’t gotten paid for from his job. I’m hopeful it can be his main gig now given that YouTube creators seem to be making more money.
- p1esk 3y agohis actual level of contribution as far as pushing forward AI He did pioneering research in image captioning - aligning visual and textual semantic spaces - the conceptual foundation of modern image generators. He also did an excellent analysis of RNNs - one of the first and best explanations of what happens under the hood of a language model.
- whatshisface 3y agoA hackernews comment section is one of the least legitimate forums imaginable for the public reading of somebody's resume. Congress, maybe.
- Judgmentality 3y agoHonestly I have more faith in hacker news users than congress.
- solardev 3y agoI wish more of us would run for Congress. I'd much rather have a government of technocrats of various stripes than ex lawyers and rich business types. IMO governments, like websites, should be boring but effective, focused on small day to day improvements, not all flash and empty marketing chasing cultural trends...
- smegger001 3y agoI don't know, while lawyers and MBA's are not who I would choose to run the country, I am not sure the I would pick people with the motto "run fast and break things" in charge either.
- jonasdegendt 3y agoBut what if we implemented agile and scrum? :^) Imagine the retros!
- smegger001 3y ago>But what if we implemented agile and scrum? thats what i am afraid of.
- int_19h 3y agoIt would be an improvement over "just break things", no?
- VirusNewbie 3y ago>Relative to his level of fame, his actual level of contribution as far as pushing forward AI, I’m not so sure about. He lead a team of one of the most common uses of DNNs, if that isn't 'pushing AI forward', I think you're confused. It's certainly pushing it forward quite a bit more than the publishing game where 99% of the papers are ignored by the people actually building real applications of AI.
- bertil 3y agoI would argue that's far more valuable.
- deleted 3y ago[deleted]
- johnnyanmac 3y ago>his actual level of contribution as far as pushing forward AI, I’m not so sure about. I mean, I don't know why people still try to devalue educating the masses. Anyone who's had to knowledge share know how hard it is to make a concise but approachable explanation for someone who knows relatively little about the field. In addition, he's still probably in a standing well above the 80% mark in terms of technical prowess. even without influencer fame I'm sure he can get into any studio he wishes.
- havercosine 3y agoDisagreeing here! I think we often overlook the value of excellent educational materials. Karpathy has truly revitalized the AI field, which is often cluttered with overly complex and dense mathematical descriptions. Take CS 231, for example, which stands as one of Stanford's most popular AI/ML courses. Think about the number of students who have taken this class from around 2015 to 2017 and have since advanced in AI. It's fair to say a good chunk of credit goes back to that course. Instructors who break it down, showing you how straightforward it can be, guiding you through each step, are invaluable. They play a crucial role in lowering the entry barriers into the field. In the long haul, it's these newcomers, brought into AI by resources like those created by Karpathy, who will drive some of the most significant breakthroughs. For instance, his "Hacker's Guide to Neural Networks," now almost a decade old, provided me with one of the clearest 'aha' moments in understanding back-propagation.
- redundantly 3y agoPeople like the grandparent think innovation and advancement happens in isolation.
- dontreact 3y agoI don’t think we disagree. Education is crucial and the value is enormous, but this hasn’t been what he was paid for in the past. I am hopeful that he finds a way to make this his job more directly than at Tesla or OpenAI as the whole world will benefit.
- sitkack 3y agoWhat a tone def elitist thing to say, you have no tact.
- simondotau 3y agoTu quoque.
- magoghm 3y agoIn 2015 he wrote this blog post about "The Unreasonable Effectiveness of Recurrent Neural Networks": https://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://karpathy.github.io/2015/05/21/rnn-effectiveness/ That blog post inspired Alec Radford at Open AI to do the research that produced the "Unsupervised sentiment neuron": https://openai.com/research/unsupervised-sentiment-neuron https://openai.com/research/unsupervised-sentiment-neuron Open AI decided to see what happened if they scaled up that model by leveraging the new Transformer architecture invented at Google, and they created something called GPT: https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf https://cdn.openai.com/research-covers/language-unsupervised...
- imjonse 3y agoAlso in that article he says "In fact, I’d go as far as to say that The concept of attention is the most interesting recent architectural innovation in neural networks." when the initial attention paper was less than a year old, and two years before the transformer paper.
- jatins 3y agoI read that post recently and it felt prescient to someone who has not been deeply involved in ML Even the HN discussion around this had comments like "this feels my baby learning to speak.." which are the same comparisons people were saying when LLMs hit mainstream in 2022
- sigmoid10 3y agoI had forgotten it's existence by now, but I remember reading this post all those years back. Damn. I also remember thinking that this would be so cool if RNNs didn't suck at long contexts, even with an attention mechanism. In some sense, the only thing he needed was the transformer architecture and a "fuck, let's just do it" compute budget to end up at ChatGPT. He was always at the frontier of this field.
- arugulum 3y agoIs it stated somewhere that Radford was inspired by that blog post?
- noufalibrahim 3y agoI think the ability to teach is a direct outcome of the ability to think and articulate ideas clearly. This is a meta skill that will make a person effective in any area of work. I'd say that that his work on AI has been significant and his ability to teach has contributed to that greatly.
- _giorgio_ 3y agoThat's incredibly impolite and totally without foundation. You make him look like a peasant :-) What do you know about his work? He's been leading the vision team at Tesla, implementing in the field all the papers that were available in the subject of autonomous driving and vision (he explicitly wrote that). He has not published about it surely due to obligations with Tesla.
- paganel 3y ago[flagged]
- _giorgio_ 3y agoLOL, that's quite the critique. Autonomous driving, especially in all weather and road conditions, presents challenges that are almost insurmountable, with complexities akin to those of Artificial General Intelligence (AGI). Good luck tackling that. The main issue lies in Tesla's decision to rely solely on vision-based systems, despite engineers advocating for the inclusion of LIDAR technology as well (which, to my knowledge, is only incorporated in one Tesla model). This decision was made by Elon Musk. Your words seem to convey a sense of bitterness and resentment. I'm not losing time discussing "peasant vs AI" because you're a person with a very limited "vision". LOL again.
- VirusNewbie 3y agoMillions of people use autopilot every day. He lead autopilot, not just fully autonomous driving.
- camillomiller 3y agoIt’s like saying Rick Rubin didn’t do much for music because he doesn’t play any instrument.
- sashank_1509 3y agoWould have to agree. Looking at Karpathys research career it’s hard to pin point something and say he’s the inventor of so and so. There are plenty of other researchers for whol you can easily say he’s the inventor of so and so and they have much lesser fame than Karpathy, for example Kaiming He for ResNet, John Schulman for PPO etc. I don’t see that as an issue though, just a natural consequence of his great work in teaching neural networks!
- Simon_ORourke 3y ago> Relative to his level of fame, his actual level of contribution as far as pushing forward AI, I’m not so sure about. I'd agree with that, however I've always wondered how easy it is for folks at that level to get hands on keyboards and not wind up spending their days polishing slide decks for talks instead.
- deleted 3y ago[deleted]