8 ms·

Self-driving as a case study for AGI

144 points by askytb 3y ago

arielzj 3y ago
Sure, it could be the case that AGI is developed progressively and slowly, the way we're seeing Waymo build autonomous vehicles. But that's just one way amongst many, and you could see it arrive suddenly through very different means, as is possible to imagine with scaled up LLMs. I really wish people would consider all the possibilities, and assign their relative probability weights to them. Is Karpathy 100% sure it will be like self-driving cars? 50%?
Animats 3y ago
Just the quiet disappearance of jobs, as has happened in so many other areas.
- freediver 3y ago
  I think historically this has never happened without being followed by appearance of (more) new, previously not existing, jobs. Industrial revolution brought machine operators, computer revolution brought computer operators etc..
  - deleted 3y ago
    [deleted]
- yeknoda 3y ago
  This is often said, but isn’t labor force participation at the lowest level ever? And aren’t a good fraction of jobs so called bull shit jobs that could be eliminated with often a positive net result, particularly gov etc.
YZF 3y ago
I think what Andrej is describing is more "automation" than AGI. His discussion of self-driving is more analogous to robots building cars in a Tesla factory displacing workers than anything AGI. We've already had "self driving" trains where we got rid of the human train driver. Nothing "AGI" about that. The evolution of getting cars to self drive not necessarily making the entity controlling the car more human-like intelligent. It's more like meeting in between the human driver and the factory robot +/- some technology. So how to define AGI? I'm not sure economic value factors here. I would lean towards a definition around problem solving. When computers can solve general problems as well as humans, that's AGI. You want to find a drug for cancer, or drive a car, or prove a math theorem, or write a computer program to accomplish something, or whatever problems humans solve all the time. (EDIT: or reason about what problems need to be solved as part of addressing other problems.) There's already classes of problems, like chess, where computers outperform humans. But I mean calculators did that for arithmetic a long time ago. The "G" part is whether or not we have a generalized computer that excels at everything.
- riffraff 3y ago
  Yeah, it seems as if he has forgotten the G. I recall Norvig's AI book preaching decades ago that "intelligent" does not mean able to do everything, and that for an agent to be useful it was enough to solve a small problem. Which in my mind is where the G came from. And yet we now suddenly go back to the old narrow definition? I still see no path from LLMs and autonomous driving to AGI.
  - m-i-l 3y ago
    > "Yeah, it seems as if he has forgotten the G. ... I still see no path from LLMs and autonomous driving to AGI." That is exactly my view too. While LLMs and autonomous driving can be exceptionally good at what they do, they are also incredibly specialist, they completely lack anything along the lines of what you might call "common sense". For example, (at least last time I looked) autonomous driving largely works off object detection at discreet time intervals, so objects can pop into and out of existence, whereas humans develop a sense of "object permanence" from a young age (i.e. know that just because something is no longer visible doesn't mean it is no longer there), and many humans also know about the laws of physics (i.e. know that if an object has a certain trajectory then there are probabilities and constraints on what can happen next).
    - 05 3y ago
      It doesn’t look like it works off discrete frames.. https://waymo.com/research/stinet-spatio-temporal-interactive-network-for-pedestrian-detection-and-trajectory-prediction/ https://waymo.com/research/stinet-spatio-temporal-interactiv...
      - m-i-l 3y ago
        Thanks, interesting read (it was a while ago I looked into this). I think the point still remains though - a self driving car doesn't have any general knowledge which can be applied to other areas, e.g. what a pedestrian is, or why a pedestrian who sees you is unlikely to step out in front of you. And similarly, the ordered tokens that an LLM outputs sometimes appear "stupid" because it has no "common sense".
freediver 3y ago
I too did not realize Waymo is operating in SF and looked for some videos on YT and found this. It paints not so rosey picture about it: https://www.youtube.com/watch?v=-Rxvl3INKSg https://www.youtube.com/watch?v=-Rxvl3INKSg
- carbocation 3y ago
  I use Waymo to get to work, and I prefer it over human drivers (even though it takes longer). It drives safely.
- alooPotato 3y ago
  Ha yeah it would have been a super interesting segment if the car just picked her up and dropped her off where she was going with no incident.
- omeze 3y ago
  I use Waymos every week, Im sure there were early problems with navigation but its been flawless every time Ive rode for the past few months
  - freediver 3y ago
    Gld to hear that. It is pretty amaziing we’ve gone this far.
SnazzyJeff 3y ago
IMO a chatbot is sufficient to demonstrate AGI. I think the chinese room problem is as good as anything. Of course, there isn't much money in teaching a bot that only knows english chinese. EDIT, Wikipedia page for context: https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
- CuriouslyC 3y ago
  People like to shit on the Turing test, but if you step back from the subjective judgement angle, and instead imagine that the person performing the Turing test is a scientist trying to collect evidence that the agent that it is communicating with is _NOT_ intelligent/human, it is actually still very relevant. Tools like statistical analysis of output and responses to jailbreak prompts and recursive/self referential prompts designed to confuse machines and generate emotional responses from humans could be used to generate probability of human/not human in a much more rigorous way.
  - skybrian 3y ago
    The actual Turing Test is a party game like Werewolf. If the humans are skilled then they should be able to authenticate by picking a subject that the AI can’t compete on. This would be a very difficult game to build a computer opponent for and nobody really tries.
    - krisoft 3y ago
      > picking a subject that the AI can’t compete on Isn’t that the point? If there are no more such subjects then the AI reached humans level cognition.
      - skybrian 3y ago
        Yes, but people underestimate what that involves. Serious players would study previous games for weaknesses, looking for subject areas that include unpublished knowledge that isn’t in the training set. Casual talk about “passing the Turing Test” sets a much lower bar.
CuriouslyC 3y ago
I don't things like "full self driving" are meaningful (and probably also AGI), because in reality it isn't a binary thing, rather it's a spectrum of power based on error rate and problem space coverage. Waymo self driving works within a defined subset of the problem space, we can stick a goalpost in the sand in term of the known problem space and error rates and say that represents "full self driving" but the reality is the problem space is less bounded than we'd like to think. We might find what we think of as full self driving and AGI turn out to be highly detailed facades when new areas of problem space are explored. For example, imagine a full self driving car trying to get out of a city that's flooding due to heavy rains, while having to compete with people fleeing to higher ground on foot. People can generalize that way but FSD is gonna take a shit, and if you don't know how to drive in that situation so are you.
- kylebenzle 3y ago
  That's a great point! In aviation we could easily call major jet liners "full self flying" if they wanted to market them as such but we still require TWO highly trained technicians in the piolets seats at all times!
- blovescoffee 3y ago
  The very beginning of the article discusses what "full self driving" means and also points out how important it is to define terms. I'm not sure your comment is a fair response to this particular article.
- akira2501 3y ago
  > Waymo self driving works within a defined subset of the problem space "works" includes a failure mode of "alert a human and ask them to take over." > when new areas of problem space are explored. The problem space is that the "rules of the road" are both legal, technical and social. All of which have internal conflicts as well as conflicts among each other. Anyone who has driven in severe weather has realized this in one way or another. > For example, imagine a full self driving car trying to get out of a city that's flooding due to heavy rains, while having to compete with people fleeing to higher ground on foot. Why do I find this easier to imagine in the fictional setting of Elysium than on the real Earth?
threeseed 3y ago
The question is with what hardware. Humans are able to move our heads to infer depth and resolve issues like occlusion. No amount of AGI can solve those if we say take a Tesla and the cameras are low quality, fixed and limited in number. And the same hardware question applies to a lot of use cases for AGI.
- kylebenzle 3y ago
  Can there be more than one high quality camera, or is that not allowed?
- arlecks 3y ago
  Simple test which I expect they have tried: Can a human drive successfully with only the video from the cameras?
  - akira2501 3y ago
    Tom Scott's got us covered. https://www.youtube.com/watch?v=UNHhfykh5Qc https://www.youtube.com/watch?v=UNHhfykh5Qc https://www.youtube.com/watch?v=7ZK_fnS62Lk https://www.youtube.com/watch?v=7ZK_fnS62Lk https://www.youtube.com/watch?v=hSmtKMx1CB4 https://www.youtube.com/watch?v=hSmtKMx1CB4
jackblemming 3y ago
Something tells me Karpathy rarely uses the “FSD” in his Tesla. He barely mentioned Tesla FSD in the blog despite being a key leader in the project. Perhaps he’d like to forget about it all together.. Maybe Elon really screwed the project by forcing the use of video cams and Karpathy is still salty about it.
- kajecounterhack 3y ago
  I think most folks use FSD for what it was intended for, L2 driving. It's a horrible name but it's not crazy to think maybe they'll get their shit together and figure out how to get to L4. They might just learn they need to add back modalities they neglected previously, or explore some new ones.
- enslavedrobot 3y ago
  Actually if you read the blog he addresses Waymo and Tesla's strategy. He says the barriers to Tesla scaling are software and Waymo needs to scale hardware. He then implies that software will win the scaling race
  - seanmcdirmid 3y ago
    LIDAR is rapidly falling in price, which will enable lots of interesting applications even if Tesla’s bet pays off.
  - steveBK123 3y ago
    Tesla "its just another software update away" pumps are the most delusional. Tesla cameras aren't even class leading in the automative industry, let alone at the cars price point. They are worse than the iPhone you handed down to your tween 5 years ago. What makes anyone think that their quality and placement will ever be adequate even with new CPUs and better software?
    - enslavedrobot 3y ago
      I think Karpathy is referring to the building of cars when he refers to scaling hardware. I find it strange that you accuse Karpathy of delusional pumping.
ShamelessC 3y ago
Never thought I would see Karpathy praising Tesla's competition so much.
- nurettin 3y ago
  It's his life's work. Also, Elon needs his stocks pumped before selling out at the end of January.
  - ShamelessC 3y ago
    What?
    - iknowstuff 3y ago
      Haha they’re seething so much, so eager to post, they couldn’t even read your comment fully before spewing out the reply
a-dub 3y ago
"AGI: An autonomous system that surpasses human capabilities in the majority of economically valuable work." -- what an obscenely depressing reduction of a fascinating field of inquiry. who the hell snuck in and redefined the science of thinking machines to this sad and reductive get rich quick crap?
- Mythrandir 3y ago
  Preach,my friend! This is the most reductive and disgusting distillation of the human experience I've read here recently...and I've followed quite a few EA threads as their founders were imprisoned ;)
- ActorNightly 3y ago
  Its not really about "get rich", its about giving people the ability to bypass the middleman for pretty much anything that they rely on.
  - Radim 3y ago
    The "middleman" being other people, i.e. themselves from another's perspective. "These other people are useless, let's bypass them. But not me! I simply gain the ability to get anything I want." The lack of second-order thinking is hilarious.
    - ben_w 3y ago
      This is why UBI gets discussed so often at the same time as AGI/ASI. If we're all redundant, how do we live? On a pension that starts at ${debatable from conception to adulthood}. Who provides the production on which the pension is spent? The AI. Even assuming UBI is great (small scale tests say so, but have necessary limits so we can't be sure), there's going to be a huge mess with most attempts to roll out such a huge change.
      - hiAndrewQuinn 3y ago
        Assuming AGI doesn't kill us all, I would imagine the argument for UBI will become much easier to defend once it causes 100x, 1000x, 10000x etc growth in the economy. Our job is mostly to hang on until one of those two outcomes occurs.
etwigg 3y ago
Whenever I am trying to figure what is true and/or good for me, there are some people who want to help me and others who want to help themselves (sell me cigarettes, negative sum politics, etc). This is the battle where AGI seems spooky - eventually people are not able to tell which way is up. We should consider the OODA loop of a person's self-determination separately from the menial tasks a person undertakes to make a living. Automating a task is totally different than breaking a person's ability to self-orient.
- riffraff 3y ago
  Do you believe this is new? It seems to me to just be another iteration of dealing with uncertain information: our neighbors may lie, our leaders may lie, newspaper may lie, radio may lie, TV may lie, blogs may lie, social networks may lie, pictures are photoshopped, videos are deepfaked.. At each iteration we had some problems but we adapted, it's one thing we're good at.
  - etwigg 3y ago
    If you wanted to buy some thinking, you used to have to pay a human! Humans are social creatures, we delegate some of our thinking to our social fabric, and some of that fabric are already avatars on screens. This is definitely new!
3asdf123 3y ago
I now come to realize that if you don't want to drive it's better to have public transport. For the real fun part of owning a personal vehicle: sport car, road-trip... I doubt you would want a robot to take over.
- seanmcdirmid 3y ago
  You can already experience self driving cars by taking an Uber or a taxi, or being chauffeured if you are richer. None of that is new, the self driving aspect just promises to make those experiences perhaps more accessible (or at the very least, not less accessible than they are now). For example, I took a taxi to and from work every day when I lived in Beijing, which came to about 100 kuai/day for a 20-30 minute drive each way, which is affordable to a lot of people (although only possible due to cheap labor). I wouldn’t mind being driven to work here in the states, although it isn’t really economically feasible (and perhaps should be replaced with direct public transit if that was time competitive, which it isn’t, but could be).
  - 3asdf123 3y ago
    Yeah which I think the only remaining good application of self-driving car ( taxi ) doesn't bring that much convenient, since taxi here is already somewhat reasonably priced. I can't speak for the the US experience. Also the article touches briefly on drivers going into jobless. A lot of drivers where i'm from seems to be retiring middle-old age working in taxi. I think it's a good job fit for them and I don't know how the new self-driving industry can provide the same thing (?)
    - dchftcs 3y ago
      A significant portion of taxi fares go to the driver, as opposed to the maintenance of cars. There are other marginal benefits such as making the driver's seat available, eliminating the driver's commute and as well as the risk of criminal driver behaviour, that probably offsets some of the drawbacks of having one fewer human being dealing with rare, complex non-driving situations such as a pregnant woman having to give birth in the car. The economic benefit is significant to companies building self-driving cars, good enough to pursue if the tech is within reach. But to your point it's indeed much less of an improvement compared to various historical automation technologies that create >10x incremental efficiency gains.
29athrowaway 3y ago
As a driver you not only see cars. You apply theory of mind to each car, assigning a personality to each moving car, even if you can't see the driver. Let's see you see a car full of bumps, marks and broken lights. You might think: this car has crashed before, I am going to avoid it. Or a car with racing parts and decals and tinted windows, you know that car likes to accelerate faster than usual and may be unsafe to be around. Or you see a SUV with baby on board stickers, you'll know that if you are going to crash you may try to crash that car last because it has babies inside, etc... So humans don't see objects they see the whole situation, unconsciously even.
- iknowstuff 3y ago
  Hardly an absolute necessity. But also, if FSD 12 is truly end to end, nothing is preventing it from learning that as well.
  - 29athrowaway 3y ago
    Is it optional? if you see a car zig zagging, or a car is going in the wrong direction, or there's a driving school sign, etc. As a human you can tell something is going on.
    - iknowstuff 3y ago
      Nothing preventing a model from learning any of those.
nl 3y ago
I think I broadly agree with how Karpathy thinks AGI will roll out with the exception of this bit: > Some people get really upset about it, and do the equivalent of putting cones on Waymos in protest, whatever the equivalent of that may be. Of course, we’ve come nowhere close to seeing this aspect fully play out just yet, but when it does I expect it to be broadly predictive. I think the equivalent of putting cones on Waymos in protest will involve large scale protests and civil unrest in some places. I think people will die (inadvertently?) because companies will act to put inadequately tested self-preservation modes in their hardware device to protect against aggressive and organized vandalism.
wodenokoto 3y ago
> but automate it we did (ahead of many other sectors of the economy), and society has noticed and is responding to it I was quite surprised by this sentence, as I thought we didn't have self driving cars. Have I been sleeping under a rock?
- dave4420 3y ago
  We have self driving cars in an extremely limited number of cities (just one or two) and you can’t buy them, you can only use them like an Uber. He’s overselling how mature the technology is.
  - thrw4w4y0 3y ago
    [dead]
  - motoxpro 3y ago
    How is he over selling it when he says explicitly that waymo exists in two cities? It's solved in that it works and they have to scale horizonatally.
    - mlazos 3y ago
      lol why do you think they picked those cities? No snow or ice or much inclement weather other than rain. It is not just scaling horizontally.
      - minwcnt5 3y ago
        I mean a year or two ago plenty of people said the tech was a dead end because it didn't handle rain and fog. They will add snow and ice capability as they expand.
    - staticman2 3y ago
      He wrote about self driving "And yet, overall it’s almost like no one cares." As if it's puzzling a product which hardly works anywhere isn't something people care about?
gwoolhurme 3y ago
As others pointed out he seems to be talking more about about automation which... sure that's a fine discussion to have, but what bugged me more than anything is the overselling of Waymo/FSD. I understand a lot of this is also on a spectrum but it seems a bit irresponsible of Karpathy not to mention crashes Waymo has faced or other problems FSD systems have faced. It's not just an issue of scaling up, sensors, etc. There is more engineering work that needs to be put in clearly. It's fine to bring it up in his example of reactions to economical forces, but let's be completely honest about the whole thing.
maaaaattttt 3y ago
Lately, whilst playing Zelda TotK (works for BotW as well), I was thinking that a good test to see if you have AGI would be letting it solve all the shrines. They require real world knowledge, sometimes rather "deep" logic and the ability to use previously unseen capabilities. Of course the AGI should not be able to have a million tries at it, RL style. Just use the "shrine test" as a regular test set. I believe one would have a pretty nice virtual proxy for a general intelligence test. From the article, I find it strange that AGI often de facto implies "super intelligence". It should be 2 distinct concepts. I find that GPT-4 is close to a general intelligence, but far from a super intelligence. Succeeding at just general intelligence would be amazing, but I don't believe it means super intelligence is just a step away. This also brings me to a point I don't see discussed a lot which is simulation (NOT in "we live in a simulation" sense). Let's say I have AGI, it passes the above mentioned shrine test, or any other accepted test. Now I'd like to tell it "find a way to travel faster than light" for example. The AGI would first be limited by our current knowledge, but could potentially find a new way. In order to find a new way it would probably need to conduct experiments and adjust its knowledge based on these experiments. If the AGI cannot run on a good enough simulation, then what it can discover will be rather limited, at least time-wise most likely quality wise. I'm thinking this falls back to Wolfram's computational irreducibility. Even if we managed a super general intelligence, it will be limited by physics of the world we live in sooner rather than later.
- ImHereToVote 3y ago
  If find it amazing how GPT-4 is good at even abstract "reasoning" as long as you present the problem as a story. Some problems can't plausibly be presented as a story ofc, also there is no way to automatically convert something into a story.
  - 1oooqooq 3y ago
    So you are doing all the work by "coding" in a weird language that actually takes more brain power than regular high level (as in direct business logic, not C or python) programming languages?
snowpid 3y ago
Self driving is also a good example from a regulation viewpoint and societal interaction. Unfortunately the article is very America - centric and ignores e.g. Mercedes progress, German regulation and the competition in China.
- buzzert 3y ago
  What is the Mercedes progress?
  - snowpid 3y ago
    Level 3 with guarantee. More than Tesla offers in the US which is mentioned in the article.
sgt101 3y ago
So this makes me wonder if I were running society what questions should I ask... - what Automation initiatives never hit "take off"? I mean, like for Nuclear Fusion, human interplanetary exploration, and Quantum Computing there's some chance that the technology simply remains beyond us "forever", I guess that "forever" means more than the lifetime of the people who start the journey... or maybe actually just beyond humans full stop. We should admit there is a non-zero chance that FSD is one of these failing quests, even if a rational observer would have to say that that chance does seem to be shrinking and close enough to 0 to instill some confidence. Perhaps domestic robotics, auto-doctors, robot-manufacturing, programming, drug development will playout to automation - but maybe not. - how do we consider the utilisation of the resources to do this? FSD has been very expensive so far, it's consumed lots of investment capital and lots of human creativity. Was that investment rational given where we stand? If society had held off and invested minimally from 2000->2024 how much would that have delayed the technology in reality? Or is it the other way round? Has the FSD investment facilitated the development of other technologies and created a 1->1 acceleration (for every year of 2000->2024 it's brought FSD a year closer than it would have been, so a cold start this year would mean FSD by 2050 or similar, whereas if we keep going then we can expect FSD by e.g. 2026) - how do we value these outcomes? Are these unalloyed goods, or are some worse than the status-quo? It could be argued that the development of some technologies left the world worse off than before - smoking, social media, personal automobiles (I know this is politically charged but I am just using examples others have raised before). Can we choose rationally, especially if a large scale intervention and development process is required to realise these outcomes?
jocoda 3y ago
Until recently I thought self driving was not going to happen. But in the early days of the car's history someone had to walk in front of the car as it went along, waving a red flag to warn people of this mechanical monstrosity. And now we have substantial societal adaptations, both legal and structural to support ubiquitous vehicular transport. Similar changes are on the way to support self driving. Our environment will be adapted to make it easier to implement self driving. And for that we won't need AGI. Jaywalking is a crime thanks to the car. Who knows what we're not going to be allowed to do soon because of self driving.
- steveBK123 3y ago
  By that definition though, if we were anywhere close... I'd expect a peer AI power & authoritarian regime like China, which also focusses on EVs, to have some tier 2 city with robotaxi-only mandate by now and a working model for what it looks like. Yet there are no signs of that. If anything they appear to be behind us.
steveBK123 3y ago
Seeing the guy who couldn't deliver FSD decide to make analogies from FSD to AGI does actually give me confidence we are decades away. Yes, yes, I know there's like 1-2 companies that have highly modified vehicles that are pretty good, in a limited geofenced area, in good weather, at low speed, driving conservatively, local roads, most of the time. This is not "FSD". They've been making very impressive incremental improvements every few years for sure. I had a Tesla for nearly 5 years and it was "wow" at first, and then "heh, I guess it's a little better" every year after that. But when can I get in a taxi at JFK or on 5th Ave and get robotaxied through city streets, urban highway, off into the far suburbs? Could be a decade, if it happens. Just because we were able to make horses faster doesn't mean we flew horses to the moon. Apply the same "sorta kinda almost" definition to AGI and yeah sure, maybe in 10 years. Really really actually solved? Hah.
- gitfan86 3y ago
  AGI has become a philosophical term in the way you are using it. Which is fine to discuss philosophy, but to the point of the article, AI enabled automation is beginning to have significant impact on the economy due to the new functionality.
  - steveBK123 3y ago
    Where have LLM AIs made measurable economic impacts that weren't already using some form of AI to start with (translation, legal, content farms, marketing, robotics)?
    - gitfan86 3y ago
      SubReddits for copywriting and graphic design and digital art are full of people talking about how the amount of gig work available has dropped off.
      - steveBK123 3y ago
        Fair examples! Also, not to move the goal posts, but there has been a large economic shift since Q3 2022 with tech & finance belt tightening, inflation normalizing and generally slowing. So these things don't happen in a vacuum (which makes them hard to measure!). For example my previous & current employer have done their first layoffs since pre-pandemic, and neither has anything to do with AI. They just overexpanded and need to shrink.
bilsbie 3y ago
Ten years ago my definition of AGI was that it can learn and improve itself across a wide range of unrelated problems. These days people seem to define if more as artificial super intelligence.
- dopidopHN 3y ago
  That fair, I had exponential growth of knowledge in the bingo cards for AGI as well.
quonn 3y ago
I can, in principle, take someone from a group of uncontacted peoples, put them into New York and let them figure out how to drive and they likely will be able to do it after not too much time. We are not even close to any technology that could figure out driving having never been built for it.
- steveBK123 3y ago
  Likewise, it's remarkable what a home DIY'er can do given a couple power tools and 5 minutes of YouTube watching.
- gitfan86 3y ago
  But that is irrelevant to the point of the article. Maybe it takes 1 million hours of computing to train a model that can generate a logo, but an average human could have learned how to do that in just 50 hours of training with Photoshop. The point of the article is that now for pennies users can generate logos in seconds that would have previously cost hundreds of dollars and days of back and forth with a designer. This dynamic is going to flow through the economy
  - quonn 3y ago
    I just don't like the AGI label.
- tim333 3y ago
  The humans have probably been learning how to negotiate the physical world for a decade or more. Also humans have evolved to be good at that stuff. That self driving tech also has to be designed and trained is sort of a similar deal.
pdimitar 3y ago
I don't trust any discussion on this topic anymore. When I was much younger, "AI" was what "AGI" is now. Now people started using "AGI" for "cars with several sensors and okay algorithms for collision detection" and then you have loud advocates going on obviously logically broken rants about the nature of "actual" intelligence -- and those are philosophical and not scientific. But still, we don't have anything even 1% close to AGI. And no, Chess and Go have NEVER EVER been about AGI. I have no idea how people ever mistook "combinatorics way beyond what the human brain can do" with "intelligent thought" but that super obvious mistake also explains the state of the AI sector these days, I feel. So before long, I guess we'll need another term, probably AGIFRTTWP == Artifical General Intelligence, For Real This Time, We Promise. And then we'll start adding numbers to it. So I am guessing Skynet / Transcendence level of AI will be at about AGIFRTTWP-6502. As for the state of this "industry", what's going on is that people with marketing chops and vested interests hijack word meanings. Nothing new, right? But it also kills my motivation to follow anything in the field. 99.9% are just loud mouths looking for the next investment round with absolutely nothing to show for it. I think I saw on YouTube military-sponsored autonomous cars races 5+ years ago (if not 10) where they did better than what the current breed of "autonomously driving cars" are doing. Will there be even one serious discussion about the general AI that you can put in a robot body and it can learn to clean, cook, repair and chat with you? Of course not, let's focus on yet-another-philosophical debate while pretending it's a scientific one. As a bystander -- not impressed. You all who are in this field should be ashamed of yourselves.
- gitfan86 3y ago
  Sounds like you are looking for an Android in the style of BladeRunner. That would be cool, but I don't understand why you are against LLMs and FSD being labeled as AI. They are using neural networks to generate content and drive cars in ways that humans find valuable.
  - pdimitar 3y ago
    They are semi-valuable. There are tons of posts out there demonstrating GPT writing code that compiles but is utterly wrong, for example. Using NNs is just a first step. To me labeling this as actual intelligence is childishly rushing into conclusions.
tim333 3y ago
He seems to be talking mostly about the impact on society >When your Waymo is driving through the streets of SF, you’ll see many people look at it as an oddity... Then they seem to move on with their lives. >When full autonomy gets introduced in other industries....they might stare and then shrug... Which I guess is ok on a small scale but if AGI starts to replace all human jobs it will have a different effect to Waymo firing some drivers and hiring AI researchers.
photon_collider 3y ago
It seems to me that Andrej is predicting how AGI will impact society by extrapolating from the current societal impacts of self driving. I don’t get the sense he was trying to say that self-driving automation is the exact same as AGI. Mainly that that AGI, like other technologies before it, will displace some jobs and create new ones but this will require companies to figure out how to scale the technology. I do think this is still very optimistic. If indeed AGIs can think and learn on their own it isn’t hard to envision a future where humans aren’t needed at all in the loop.
tdalaa 3y ago
Ah, would have liked to have read this one, but it's now 404. Anyone managed to capture it or have a link to an archive?
- jjri 3y ago
  https://web.archive.org/web/20240122141906/https://karpathy.github.io/2024/01/21/selfdriving-agi/ https://web.archive.org/web/20240122141906/https://karpathy....
kerim-ca 3y ago
https://web.archive.org/web/20240122141906/https://karpathy.github.io/2024/01/21/selfdriving-agi/ https://web.archive.org/web/20240122141906/https://karpathy....

144 points by askytb 3y ago

Sure, it could be the case that AGI is developed progressively and slowly, the way we're seeing Waymo build autonomous vehicles. But that's just one way amongst many, and you could see it arrive suddenly through very different means, as is possible to imagine with scaled up LLMs. I really wish people would consider all the possibilities, and assign their relative probability weights to them. Is Karpathy 100% sure it will be like self-driving cars? 50%?

Animats 3y ago

Just the quiet disappearance of jobs, as has happened in so many other areas.

freediver 3y ago
I think historically this has never happened without being followed by appearance of (more) new, previously not existing, jobs. Industrial revolution brought machine operators, computer revolution brought computer operators etc..
- deleted 3y ago
  [deleted]
yeknoda 3y ago
This is often said, but isn’t labor force participation at the lowest level ever? And aren’t a good fraction of jobs so called bull shit jobs that could be eliminated with often a positive net result, particularly gov etc.

YZF 3y ago

I think what Andrej is describing is more "automation" than AGI. His discussion of self-driving is more analogous to robots building cars in a Tesla factory displacing workers than anything AGI. We've already had "self driving" trains where we got rid of the human train driver. Nothing "AGI" about that. The evolution of getting cars to self drive not necessarily making the entity controlling the car more human-like intelligent. It's more like meeting in between the human driver and the factory robot +/- some technology. So how to define AGI? I'm not sure economic value factors here. I would lean towards a definition around problem solving. When computers can solve general problems as well as humans, that's AGI. You want to find a drug for cancer, or drive a car, or prove a math theorem, or write a computer program to accomplish something, or whatever problems humans solve all the time. (EDIT: or reason about what problems need to be solved as part of addressing other problems.) There's already classes of problems, like chess, where computers outperform humans. But I mean calculators did that for arithmetic a long time ago. The "G" part is whether or not we have a generalized computer that excels at everything.

riffraff 3y ago
Yeah, it seems as if he has forgotten the G. I recall Norvig's AI book preaching decades ago that "intelligent" does not mean able to do everything, and that for an agent to be useful it was enough to solve a small problem. Which in my mind is where the G came from. And yet we now suddenly go back to the old narrow definition? I still see no path from LLMs and autonomous driving to AGI.
- m-i-l 3y ago
  > "Yeah, it seems as if he has forgotten the G. ... I still see no path from LLMs and autonomous driving to AGI." That is exactly my view too. While LLMs and autonomous driving can be exceptionally good at what they do, they are also incredibly specialist, they completely lack anything along the lines of what you might call "common sense". For example, (at least last time I looked) autonomous driving largely works off object detection at discreet time intervals, so objects can pop into and out of existence, whereas humans develop a sense of "object permanence" from a young age (i.e. know that just because something is no longer visible doesn't mean it is no longer there), and many humans also know about the laws of physics (i.e. know that if an object has a certain trajectory then there are probabilities and constraints on what can happen next).
  - 05 3y ago
    It doesn’t look like it works off discrete frames.. https://waymo.com/research/stinet-spatio-temporal-interactive-network-for-pedestrian-detection-and-trajectory-prediction/ https://waymo.com/research/stinet-spatio-temporal-interactiv...
    - m-i-l 3y ago
      Thanks, interesting read (it was a while ago I looked into this). I think the point still remains though - a self driving car doesn't have any general knowledge which can be applied to other areas, e.g. what a pedestrian is, or why a pedestrian who sees you is unlikely to step out in front of you. And similarly, the ordered tokens that an LLM outputs sometimes appear "stupid" because it has no "common sense".

freediver 3y ago

I too did not realize Waymo is operating in SF and looked for some videos on YT and found this. It paints not so rosey picture about it: https://www.youtube.com/watch?v=-Rxvl3INKSg https://www.youtube.com/watch?v=-Rxvl3INKSg

carbocation 3y ago
I use Waymo to get to work, and I prefer it over human drivers (even though it takes longer). It drives safely.
alooPotato 3y ago
Ha yeah it would have been a super interesting segment if the car just picked her up and dropped her off where she was going with no incident.
omeze 3y ago
I use Waymos every week, Im sure there were early problems with navigation but its been flawless every time Ive rode for the past few months
- freediver 3y ago
  Gld to hear that. It is pretty amaziing we’ve gone this far.

SnazzyJeff 3y ago

IMO a chatbot is sufficient to demonstrate AGI. I think the chinese room problem is as good as anything. Of course, there isn't much money in teaching a bot that only knows english chinese. EDIT, Wikipedia page for context: https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room

CuriouslyC 3y ago
People like to shit on the Turing test, but if you step back from the subjective judgement angle, and instead imagine that the person performing the Turing test is a scientist trying to collect evidence that the agent that it is communicating with is _NOT_ intelligent/human, it is actually still very relevant. Tools like statistical analysis of output and responses to jailbreak prompts and recursive/self referential prompts designed to confuse machines and generate emotional responses from humans could be used to generate probability of human/not human in a much more rigorous way.
- skybrian 3y ago
  The actual Turing Test is a party game like Werewolf. If the humans are skilled then they should be able to authenticate by picking a subject that the AI can’t compete on. This would be a very difficult game to build a computer opponent for and nobody really tries.
  - krisoft 3y ago
    > picking a subject that the AI can’t compete on Isn’t that the point? If there are no more such subjects then the AI reached humans level cognition.
    - skybrian 3y ago
      Yes, but people underestimate what that involves. Serious players would study previous games for weaknesses, looking for subject areas that include unpublished knowledge that isn’t in the training set. Casual talk about “passing the Turing Test” sets a much lower bar.

CuriouslyC 3y ago

I don't things like "full self driving" are meaningful (and probably also AGI), because in reality it isn't a binary thing, rather it's a spectrum of power based on error rate and problem space coverage. Waymo self driving works within a defined subset of the problem space, we can stick a goalpost in the sand in term of the known problem space and error rates and say that represents "full self driving" but the reality is the problem space is less bounded than we'd like to think. We might find what we think of as full self driving and AGI turn out to be highly detailed facades when new areas of problem space are explored. For example, imagine a full self driving car trying to get out of a city that's flooding due to heavy rains, while having to compete with people fleeing to higher ground on foot. People can generalize that way but FSD is gonna take a shit, and if you don't know how to drive in that situation so are you.

kylebenzle 3y ago
That's a great point! In aviation we could easily call major jet liners "full self flying" if they wanted to market them as such but we still require TWO highly trained technicians in the piolets seats at all times!
blovescoffee 3y ago
The very beginning of the article discusses what "full self driving" means and also points out how important it is to define terms. I'm not sure your comment is a fair response to this particular article.
akira2501 3y ago
> Waymo self driving works within a defined subset of the problem space "works" includes a failure mode of "alert a human and ask them to take over." > when new areas of problem space are explored. The problem space is that the "rules of the road" are both legal, technical and social. All of which have internal conflicts as well as conflicts among each other. Anyone who has driven in severe weather has realized this in one way or another. > For example, imagine a full self driving car trying to get out of a city that's flooding due to heavy rains, while having to compete with people fleeing to higher ground on foot. Why do I find this easier to imagine in the fictional setting of Elysium than on the real Earth?

threeseed 3y ago

The question is with what hardware. Humans are able to move our heads to infer depth and resolve issues like occlusion. No amount of AGI can solve those if we say take a Tesla and the cameras are low quality, fixed and limited in number. And the same hardware question applies to a lot of use cases for AGI.

kylebenzle 3y ago
Can there be more than one high quality camera, or is that not allowed?
arlecks 3y ago
Simple test which I expect they have tried: Can a human drive successfully with only the video from the cameras?
- akira2501 3y ago
  Tom Scott's got us covered. https://www.youtube.com/watch?v=UNHhfykh5Qc https://www.youtube.com/watch?v=UNHhfykh5Qc https://www.youtube.com/watch?v=7ZK_fnS62Lk https://www.youtube.com/watch?v=7ZK_fnS62Lk https://www.youtube.com/watch?v=hSmtKMx1CB4 https://www.youtube.com/watch?v=hSmtKMx1CB4

jackblemming 3y ago

Something tells me Karpathy rarely uses the “FSD” in his Tesla. He barely mentioned Tesla FSD in the blog despite being a key leader in the project. Perhaps he’d like to forget about it all together.. Maybe Elon really screwed the project by forcing the use of video cams and Karpathy is still salty about it.

kajecounterhack 3y ago
I think most folks use FSD for what it was intended for, L2 driving. It's a horrible name but it's not crazy to think maybe they'll get their shit together and figure out how to get to L4. They might just learn they need to add back modalities they neglected previously, or explore some new ones.
enslavedrobot 3y ago
Actually if you read the blog he addresses Waymo and Tesla's strategy. He says the barriers to Tesla scaling are software and Waymo needs to scale hardware. He then implies that software will win the scaling race
- seanmcdirmid 3y ago
  LIDAR is rapidly falling in price, which will enable lots of interesting applications even if Tesla’s bet pays off.
- steveBK123 3y ago
  Tesla "its just another software update away" pumps are the most delusional. Tesla cameras aren't even class leading in the automative industry, let alone at the cars price point. They are worse than the iPhone you handed down to your tween 5 years ago. What makes anyone think that their quality and placement will ever be adequate even with new CPUs and better software?
  - enslavedrobot 3y ago
    I think Karpathy is referring to the building of cars when he refers to scaling hardware. I find it strange that you accuse Karpathy of delusional pumping.

ShamelessC 3y ago

Never thought I would see Karpathy praising Tesla's competition so much.

nurettin 3y ago
It's his life's work. Also, Elon needs his stocks pumped before selling out at the end of January.
- ShamelessC 3y ago
  What?
  - iknowstuff 3y ago
    Haha they’re seething so much, so eager to post, they couldn’t even read your comment fully before spewing out the reply

a-dub 3y ago

"AGI: An autonomous system that surpasses human capabilities in the majority of economically valuable work." -- what an obscenely depressing reduction of a fascinating field of inquiry. who the hell snuck in and redefined the science of thinking machines to this sad and reductive get rich quick crap?

Mythrandir 3y ago
Preach,my friend! This is the most reductive and disgusting distillation of the human experience I've read here recently...and I've followed quite a few EA threads as their founders were imprisoned ;)
ActorNightly 3y ago
Its not really about "get rich", its about giving people the ability to bypass the middleman for pretty much anything that they rely on.
- Radim 3y ago
  The "middleman" being other people, i.e. themselves from another's perspective. "These other people are useless, let's bypass them. But not me! I simply gain the ability to get anything I want." The lack of second-order thinking is hilarious.
  - ben_w 3y ago
    This is why UBI gets discussed so often at the same time as AGI/ASI. If we're all redundant, how do we live? On a pension that starts at ${debatable from conception to adulthood}. Who provides the production on which the pension is spent? The AI. Even assuming UBI is great (small scale tests say so, but have necessary limits so we can't be sure), there's going to be a huge mess with most attempts to roll out such a huge change.
    - hiAndrewQuinn 3y ago
      Assuming AGI doesn't kill us all, I would imagine the argument for UBI will become much easier to defend once it causes 100x, 1000x, 10000x etc growth in the economy. Our job is mostly to hang on until one of those two outcomes occurs.

etwigg 3y ago

Whenever I am trying to figure what is true and/or good for me, there are some people who want to help me and others who want to help themselves (sell me cigarettes, negative sum politics, etc). This is the battle where AGI seems spooky - eventually people are not able to tell which way is up. We should consider the OODA loop of a person's self-determination separately from the menial tasks a person undertakes to make a living. Automating a task is totally different than breaking a person's ability to self-orient.

riffraff 3y ago
Do you believe this is new? It seems to me to just be another iteration of dealing with uncertain information: our neighbors may lie, our leaders may lie, newspaper may lie, radio may lie, TV may lie, blogs may lie, social networks may lie, pictures are photoshopped, videos are deepfaked.. At each iteration we had some problems but we adapted, it's one thing we're good at.
- etwigg 3y ago
  If you wanted to buy some thinking, you used to have to pay a human! Humans are social creatures, we delegate some of our thinking to our social fabric, and some of that fabric are already avatars on screens. This is definitely new!

3asdf123 3y ago

I now come to realize that if you don't want to drive it's better to have public transport. For the real fun part of owning a personal vehicle: sport car, road-trip... I doubt you would want a robot to take over.

seanmcdirmid 3y ago
You can already experience self driving cars by taking an Uber or a taxi, or being chauffeured if you are richer. None of that is new, the self driving aspect just promises to make those experiences perhaps more accessible (or at the very least, not less accessible than they are now). For example, I took a taxi to and from work every day when I lived in Beijing, which came to about 100 kuai/day for a 20-30 minute drive each way, which is affordable to a lot of people (although only possible due to cheap labor). I wouldn’t mind being driven to work here in the states, although it isn’t really economically feasible (and perhaps should be replaced with direct public transit if that was time competitive, which it isn’t, but could be).
- 3asdf123 3y ago
  Yeah which I think the only remaining good application of self-driving car ( taxi ) doesn't bring that much convenient, since taxi here is already somewhat reasonably priced. I can't speak for the the US experience. Also the article touches briefly on drivers going into jobless. A lot of drivers where i'm from seems to be retiring middle-old age working in taxi. I think it's a good job fit for them and I don't know how the new self-driving industry can provide the same thing (?)
  - dchftcs 3y ago
    A significant portion of taxi fares go to the driver, as opposed to the maintenance of cars. There are other marginal benefits such as making the driver's seat available, eliminating the driver's commute and as well as the risk of criminal driver behaviour, that probably offsets some of the drawbacks of having one fewer human being dealing with rare, complex non-driving situations such as a pregnant woman having to give birth in the car. The economic benefit is significant to companies building self-driving cars, good enough to pursue if the tech is within reach. But to your point it's indeed much less of an improvement compared to various historical automation technologies that create >10x incremental efficiency gains.

29athrowaway 3y ago

As a driver you not only see cars. You apply theory of mind to each car, assigning a personality to each moving car, even if you can't see the driver. Let's see you see a car full of bumps, marks and broken lights. You might think: this car has crashed before, I am going to avoid it. Or a car with racing parts and decals and tinted windows, you know that car likes to accelerate faster than usual and may be unsafe to be around. Or you see a SUV with baby on board stickers, you'll know that if you are going to crash you may try to crash that car last because it has babies inside, etc... So humans don't see objects they see the whole situation, unconsciously even.

iknowstuff 3y ago
Hardly an absolute necessity. But also, if FSD 12 is truly end to end, nothing is preventing it from learning that as well.
- 29athrowaway 3y ago
  Is it optional? if you see a car zig zagging, or a car is going in the wrong direction, or there's a driving school sign, etc. As a human you can tell something is going on.
  - iknowstuff 3y ago
    Nothing preventing a model from learning any of those.

nl 3y ago

I think I broadly agree with how Karpathy thinks AGI will roll out with the exception of this bit: > Some people get really upset about it, and do the equivalent of putting cones on Waymos in protest, whatever the equivalent of that may be. Of course, we’ve come nowhere close to seeing this aspect fully play out just yet, but when it does I expect it to be broadly predictive. I think the equivalent of putting cones on Waymos in protest will involve large scale protests and civil unrest in some places. I think people will die (inadvertently?) because companies will act to put inadequately tested self-preservation modes in their hardware device to protect against aggressive and organized vandalism.

wodenokoto 3y ago

> but automate it we did (ahead of many other sectors of the economy), and society has noticed and is responding to it I was quite surprised by this sentence, as I thought we didn't have self driving cars. Have I been sleeping under a rock?

dave4420 3y ago
We have self driving cars in an extremely limited number of cities (just one or two) and you can’t buy them, you can only use them like an Uber. He’s overselling how mature the technology is.
- thrw4w4y0 3y ago
  [dead]
- motoxpro 3y ago
  How is he over selling it when he says explicitly that waymo exists in two cities? It's solved in that it works and they have to scale horizonatally.
  - mlazos 3y ago
    lol why do you think they picked those cities? No snow or ice or much inclement weather other than rain. It is not just scaling horizontally.
    - minwcnt5 3y ago
      I mean a year or two ago plenty of people said the tech was a dead end because it didn't handle rain and fog. They will add snow and ice capability as they expand.
  - staticman2 3y ago
    He wrote about self driving "And yet, overall it’s almost like no one cares." As if it's puzzling a product which hardly works anywhere isn't something people care about?

gwoolhurme 3y ago

As others pointed out he seems to be talking more about about automation which... sure that's a fine discussion to have, but what bugged me more than anything is the overselling of Waymo/FSD. I understand a lot of this is also on a spectrum but it seems a bit irresponsible of Karpathy not to mention crashes Waymo has faced or other problems FSD systems have faced. It's not just an issue of scaling up, sensors, etc. There is more engineering work that needs to be put in clearly. It's fine to bring it up in his example of reactions to economical forces, but let's be completely honest about the whole thing.

maaaaattttt 3y ago

Lately, whilst playing Zelda TotK (works for BotW as well), I was thinking that a good test to see if you have AGI would be letting it solve all the shrines. They require real world knowledge, sometimes rather "deep" logic and the ability to use previously unseen capabilities. Of course the AGI should not be able to have a million tries at it, RL style. Just use the "shrine test" as a regular test set. I believe one would have a pretty nice virtual proxy for a general intelligence test. From the article, I find it strange that AGI often de facto implies "super intelligence". It should be 2 distinct concepts. I find that GPT-4 is close to a general intelligence, but far from a super intelligence. Succeeding at just general intelligence would be amazing, but I don't believe it means super intelligence is just a step away. This also brings me to a point I don't see discussed a lot which is simulation (NOT in "we live in a simulation" sense). Let's say I have AGI, it passes the above mentioned shrine test, or any other accepted test. Now I'd like to tell it "find a way to travel faster than light" for example. The AGI would first be limited by our current knowledge, but could potentially find a new way. In order to find a new way it would probably need to conduct experiments and adjust its knowledge based on these experiments. If the AGI cannot run on a good enough simulation, then what it can discover will be rather limited, at least time-wise most likely quality wise. I'm thinking this falls back to Wolfram's computational irreducibility. Even if we managed a super general intelligence, it will be limited by physics of the world we live in sooner rather than later.

ImHereToVote 3y ago
If find it amazing how GPT-4 is good at even abstract "reasoning" as long as you present the problem as a story. Some problems can't plausibly be presented as a story ofc, also there is no way to automatically convert something into a story.
- 1oooqooq 3y ago
  So you are doing all the work by "coding" in a weird language that actually takes more brain power than regular high level (as in direct business logic, not C or python) programming languages?

snowpid 3y ago

Self driving is also a good example from a regulation viewpoint and societal interaction. Unfortunately the article is very America - centric and ignores e.g. Mercedes progress, German regulation and the competition in China.

buzzert 3y ago
What is the Mercedes progress?
- snowpid 3y ago
  Level 3 with guarantee. More than Tesla offers in the US which is mentioned in the article.

sgt101 3y ago

So this makes me wonder if I were running society what questions should I ask... - what Automation initiatives never hit "take off"? I mean, like for Nuclear Fusion, human interplanetary exploration, and Quantum Computing there's some chance that the technology simply remains beyond us "forever", I guess that "forever" means more than the lifetime of the people who start the journey... or maybe actually just beyond humans full stop. We should admit there is a non-zero chance that FSD is one of these failing quests, even if a rational observer would have to say that that chance does seem to be shrinking and close enough to 0 to instill some confidence. Perhaps domestic robotics, auto-doctors, robot-manufacturing, programming, drug development will playout to automation - but maybe not. - how do we consider the utilisation of the resources to do this? FSD has been very expensive so far, it's consumed lots of investment capital and lots of human creativity. Was that investment rational given where we stand? If society had held off and invested minimally from 2000->2024 how much would that have delayed the technology in reality? Or is it the other way round? Has the FSD investment facilitated the development of other technologies and created a 1->1 acceleration (for every year of 2000->2024 it's brought FSD a year closer than it would have been, so a cold start this year would mean FSD by 2050 or similar, whereas if we keep going then we can expect FSD by e.g. 2026) - how do we value these outcomes? Are these unalloyed goods, or are some worse than the status-quo? It could be argued that the development of some technologies left the world worse off than before - smoking, social media, personal automobiles (I know this is politically charged but I am just using examples others have raised before). Can we choose rationally, especially if a large scale intervention and development process is required to realise these outcomes?

jocoda 3y ago

Until recently I thought self driving was not going to happen. But in the early days of the car's history someone had to walk in front of the car as it went along, waving a red flag to warn people of this mechanical monstrosity. And now we have substantial societal adaptations, both legal and structural to support ubiquitous vehicular transport. Similar changes are on the way to support self driving. Our environment will be adapted to make it easier to implement self driving. And for that we won't need AGI. Jaywalking is a crime thanks to the car. Who knows what we're not going to be allowed to do soon because of self driving.

steveBK123 3y ago
By that definition though, if we were anywhere close... I'd expect a peer AI power & authoritarian regime like China, which also focusses on EVs, to have some tier 2 city with robotaxi-only mandate by now and a working model for what it looks like. Yet there are no signs of that. If anything they appear to be behind us.

steveBK123 3y ago

Seeing the guy who couldn't deliver FSD decide to make analogies from FSD to AGI does actually give me confidence we are decades away. Yes, yes, I know there's like 1-2 companies that have highly modified vehicles that are pretty good, in a limited geofenced area, in good weather, at low speed, driving conservatively, local roads, most of the time. This is not "FSD". They've been making very impressive incremental improvements every few years for sure. I had a Tesla for nearly 5 years and it was "wow" at first, and then "heh, I guess it's a little better" every year after that. But when can I get in a taxi at JFK or on 5th Ave and get robotaxied through city streets, urban highway, off into the far suburbs? Could be a decade, if it happens. Just because we were able to make horses faster doesn't mean we flew horses to the moon. Apply the same "sorta kinda almost" definition to AGI and yeah sure, maybe in 10 years. Really really actually solved? Hah.

gitfan86 3y ago
AGI has become a philosophical term in the way you are using it. Which is fine to discuss philosophy, but to the point of the article, AI enabled automation is beginning to have significant impact on the economy due to the new functionality.
- steveBK123 3y ago
  Where have LLM AIs made measurable economic impacts that weren't already using some form of AI to start with (translation, legal, content farms, marketing, robotics)?
  - gitfan86 3y ago
    SubReddits for copywriting and graphic design and digital art are full of people talking about how the amount of gig work available has dropped off.
    - steveBK123 3y ago
      Fair examples! Also, not to move the goal posts, but there has been a large economic shift since Q3 2022 with tech & finance belt tightening, inflation normalizing and generally slowing. So these things don't happen in a vacuum (which makes them hard to measure!). For example my previous & current employer have done their first layoffs since pre-pandemic, and neither has anything to do with AI. They just overexpanded and need to shrink.

bilsbie 3y ago

Ten years ago my definition of AGI was that it can learn and improve itself across a wide range of unrelated problems. These days people seem to define if more as artificial super intelligence.

dopidopHN 3y ago
That fair, I had exponential growth of knowledge in the bingo cards for AGI as well.

quonn 3y ago

I can, in principle, take someone from a group of uncontacted peoples, put them into New York and let them figure out how to drive and they likely will be able to do it after not too much time. We are not even close to any technology that could figure out driving having never been built for it.

steveBK123 3y ago
Likewise, it's remarkable what a home DIY'er can do given a couple power tools and 5 minutes of YouTube watching.
gitfan86 3y ago
But that is irrelevant to the point of the article. Maybe it takes 1 million hours of computing to train a model that can generate a logo, but an average human could have learned how to do that in just 50 hours of training with Photoshop. The point of the article is that now for pennies users can generate logos in seconds that would have previously cost hundreds of dollars and days of back and forth with a designer. This dynamic is going to flow through the economy
- quonn 3y ago
  I just don't like the AGI label.
tim333 3y ago
The humans have probably been learning how to negotiate the physical world for a decade or more. Also humans have evolved to be good at that stuff. That self driving tech also has to be designed and trained is sort of a similar deal.

pdimitar 3y ago

I don't trust any discussion on this topic anymore. When I was much younger, "AI" was what "AGI" is now. Now people started using "AGI" for "cars with several sensors and okay algorithms for collision detection" and then you have loud advocates going on obviously logically broken rants about the nature of "actual" intelligence -- and those are philosophical and not scientific. But still, we don't have anything even 1% close to AGI. And no, Chess and Go have NEVER EVER been about AGI. I have no idea how people ever mistook "combinatorics way beyond what the human brain can do" with "intelligent thought" but that super obvious mistake also explains the state of the AI sector these days, I feel. So before long, I guess we'll need another term, probably AGIFRTTWP == Artifical General Intelligence, For Real This Time, We Promise. And then we'll start adding numbers to it. So I am guessing Skynet / Transcendence level of AI will be at about AGIFRTTWP-6502. As for the state of this "industry", what's going on is that people with marketing chops and vested interests hijack word meanings. Nothing new, right? But it also kills my motivation to follow anything in the field. 99.9% are just loud mouths looking for the next investment round with absolutely nothing to show for it. I think I saw on YouTube military-sponsored autonomous cars races 5+ years ago (if not 10) where they did better than what the current breed of "autonomously driving cars" are doing. Will there be even one serious discussion about the general AI that you can put in a robot body and it can learn to clean, cook, repair and chat with you? Of course not, let's focus on yet-another-philosophical debate while pretending it's a scientific one. As a bystander -- not impressed. You all who are in this field should be ashamed of yourselves.

gitfan86 3y ago
Sounds like you are looking for an Android in the style of BladeRunner. That would be cool, but I don't understand why you are against LLMs and FSD being labeled as AI. They are using neural networks to generate content and drive cars in ways that humans find valuable.
- pdimitar 3y ago
  They are semi-valuable. There are tons of posts out there demonstrating GPT writing code that compiles but is utterly wrong, for example. Using NNs is just a first step. To me labeling this as actual intelligence is childishly rushing into conclusions.

tim333 3y ago

He seems to be talking mostly about the impact on society >When your Waymo is driving through the streets of SF, you’ll see many people look at it as an oddity... Then they seem to move on with their lives. >When full autonomy gets introduced in other industries....they might stare and then shrug... Which I guess is ok on a small scale but if AGI starts to replace all human jobs it will have a different effect to Waymo firing some drivers and hiring AI researchers.

photon_collider 3y ago

It seems to me that Andrej is predicting how AGI will impact society by extrapolating from the current societal impacts of self driving. I don’t get the sense he was trying to say that self-driving automation is the exact same as AGI. Mainly that that AGI, like other technologies before it, will displace some jobs and create new ones but this will require companies to figure out how to scale the technology. I do think this is still very optimistic. If indeed AGIs can think and learn on their own it isn’t hard to envision a future where humans aren’t needed at all in the loop.

tdalaa 3y ago

Ah, would have liked to have read this one, but it's now 404. Anyone managed to capture it or have a link to an archive?

jjri 3y ago
https://web.archive.org/web/20240122141906/https://karpathy.github.io/2024/01/21/selfdriving-agi/ https://web.archive.org/web/20240122141906/https://karpathy....

kerim-ca 3y ago

https://web.archive.org/web/20240122141906/https://karpathy.github.io/2024/01/21/selfdriving-agi/ https://web.archive.org/web/20240122141906/https://karpathy....