9 ms·

Tesla has bet the company on robotaxis, but their vision only tech stack doesn’t seem capable of solving it, which is a problem because Tesla has repeatedly pr

Tesla has bet the company on robotaxis, but their vision only tech stack doesn’t seem capable of solving it, which is a problem because Tesla has repeatedly promised FSD is right around the corner, or less than a year away. It’s hard to believe Karpathy would step down if he felt they were close to solving the problem anytime soon. This announcement comes after a 4 month sabbatical where Karpathy said he wanted to take some time off to “sharpen my technical edge,” which makes it sound like this is the result of frustration with the technical approach instead of burnout.

impulser_ 4y ago
Yeah, but Google's vision + lidar tech also doesn't seem any better at solving it either. They have been working on this problem the longest and they aren't even confident enough to produce a product with it. Google is probably the leader in AI and AI research. They are also the leader in data and mapping. They have billions of cash to play with. Yet it seem like they haven't gotten any closer at solving this problem as well. They are just going about it better but not trying to selling it. Any reason why everyone seems to be stuck on this problem?
- realitysballs 4y ago
  Answer me this; How is AI supposed to confidently distinguish a real stop sign from someone/something holding up a picture of a stop sign? Yes, this is a weird edge case, but I think it gets at the core issue being that it takes way more sophistication to release this tech into the wild then ppl would like to admit.
- Barrin92 4y ago
  >Any reason why everyone seems to be stuck on this problem? ML maximalism focused on the narrow problem of 'solving driving' while not recognizing that any task as complex as driving requires probably something closer to general intelligence, and theoretically the field has been impoverished in favor of "throw more graphics cards at everything".
  - hn_throwaway_99 4y ago
    Couldn't agree more. Especially when it comes to city driving, which would obviously be necessary for robotaxis, when AI zealots promise "it'll be here in a year or two", I always wondered "Have these people ever driven in the city?" I mean, to drive in a city, you basically: 1. Need to understand all standard signage (seems possible with AI). 2. Need to understand all "unstandard" signage (not sure how possible). 3. Need to understand the cop with the thick NY accent yelling at you saying "Can't you see there's been an accident and the road is covered with glass you dufus? Turn the F around." I can certainly see AI solving the problem of driving in specially designed limited access highways (which could also support normal human drivers), and that alone would be a huge benefit, but I never saw how so many were willing to make the leap to "robotaxis that can drive you anywhere in the city."
    - dmd 4y ago
      4. Need to understand the cop who is directing people into lanes by jutting his chin subtly in different directions when you make eye contact with him 5. Need to understand that occluded objects have not vanished from the universe never to be seen again
      - naijaboiler 4y ago
        6. Need to reasonably predict what that human that just made eye contact with you would likely do next, and how that's different from what he might do when he doesn’t make eye contact with you. And all of that differs if you're in NYC or SF or small town, Indiana
        dibujante 4y ago
        7. Need to understand that drunk person staggering along the roadside has been repeatedly slipping off the sidewalk and there's a non-zero chance they trip and fall right in front of you.
        4y ago
- Jabbles 4y ago
  > doesn't seem any better at solving it I thought they had real self-driving taxis in Pheonix that you can order? Real ones, with no safety driver. That definitely sounds "better", even if it is heavily geo-fenced.
  - jowday 4y ago
    They’ve also got real ones with no safety driver in San Francisco right now.
  - impulser_ 4y ago
    Yeah, but I am sure Tesla's software can do the same. Depending on the route, you could probably even do it with comma.ai hardware. When I think of FSD, I think any route under any condition.
    - kadoban 4y ago
      > Yeah, but I am sure Tesla's software can do the same. They absolutely cannot. They won't even try, they require a driver to be there to be ready to take over with no notice. Their software also makes so many basic mistakes that even what they're allowing it to do is dangerously reckless.
  - espadrine 4y ago
    Waymo has superior performance based on their historical statistics. It makes sense, since their lidar sensors capture more of the environment, and directly in 3D. Their AI also seems better QA’ed. The Tesla AI Day[0] surprised me as it showed they only had a simple architecture for a very long time, simply feeding barely processed camera pixels to a DNN and hoping for the best with little more than supervised learning off human feeds. Their big claim to glory was that they rearchitectured it to produce a 2D map of the environment… which I thought they had years ago, and is still a far cry from the 3D modeling that is needed. After all, sure, we humans only take two video feeds as input… But we can appreciate from it the position, intent, and trajectory of a wealth of elements around us, with reasonable probability estimates for multiple possibilities, sometimes pertaining to things that are invisible, such as kids crossing from nowhere when near a school. Cruise also seems to have better tech; they had a barely-watched 2h30 description of their systems[1] which shows they do create a richer environment, evaluate the routing of many objects, and train their systems on a very realistic simulation, not just supervised training, which means it can learn from very low-probability events. They have a whole segment on including the probability that unseen cars may travel from perpendicular roads; Tesla’s creeping hit-or-miss are well-documented on Youtube. [0]: https://www.youtube.com/watch?v=j0z4FweCy4M https://www.youtube.com/watch?v=j0z4FweCy4M [1]: https://www.youtube.com/watch?v=uJWN0K26NxQ https://www.youtube.com/watch?v=uJWN0K26NxQ
- alphabetting 4y ago
  There's been a lot of progress despite AVs not meeting intitial hyped predictions. Waymo and Cruise are operating driverless robotaxis in SF. We're probably a couple years from many major cities having them.
- tootie 4y ago
  Probably because it's really, really, really hard to solve the thousands of edge cases that occur in real-world driving situations. I don't think FSD happens until government gets behind it and starts putting infrastructure behind it. If we start building roads (and cars) to be highly visible to AI one way or another, it all becomes much easier.
  - aaaaaaaaaaab 4y ago
    >If we start building roads (and cars) to be highly visible to AI one way or another, it all becomes much easier. Or we could build 1-dimensional roads, which would make the AI's jobs much easier. Like, we could put down two parallel piece of metal, which vehicles could "hook" onto somehow...
- cco 4y ago
  I paid ~$10 for two rides after signing up as a regular ole user in Mesa AZ. It was great, the first ride was a bit nerve wracking, but the second felt very nice. I certainly wouldn't argue with you that it isn't ready for prime time and wide distribution, but it is interesting to see their progress in San Francisco, a much different driving problem. If it takes them 10 years to get to prod in Mesa, two (maybe three?) in SF, maybe they start shrinking that a lot in metros without winters. ¯\_(ツ)_/¯
- dreamcompiler 4y ago
  Because self-driving has a bunch of tricky edge cases and most of them will kill people. Problems with hundreds of important edge cases cannot be solved by simply throwing more training data at the problem; that's how you solve AI problems in a "dumb" manner, and it works for lots of problems (like recognizing dogs in images) -- but not for self-driving. To solve the self-driving problem we need "smart" A.I., which means we have to approach it with systematic engineering, and the solution will probably involve some combination of better sensors, introspectable neural nets, symbolic A.I., and logical A.I.
  - naijaboiler 4y ago
    And we haven't even adressed that drving is not a purely technical endeavor, it's largely a social one.
    - MonkeyMalarky 4y ago
      A social one where you use experience to make inferences about what is going to happen next. Who hasn't observed an aggressive driver coming up from behind, weaving around other cars and said to themselves something like "that guy is going to cut me off, better ease off the gas so there's a little extra room in front for when he does". That level of situational awareness isn't coded for is it? Edit: I have a whole mental model for other drivers and different approaches for them. Someone driving like a grandma? Pass when available. Nervous/erratic/lost driver? Keep extra distance then pass as soon as possible. Aggressive driver? Relax, give some space and let them get ahead. And so on. I get that stereotyping is bad but ignoring the subtle signals other drivers give off seems like it would be myopic. An AI that doesn't anticipate what others will do on the road will always be reactive rather than proactive.
      - aidenn0 4y ago
        Most self-driving vehicles aren't just coded; they incorporate some form of ML as well. Categorizing patterns of behaviors is well within the reach of ML algorithms, but my understanding is we have far more basic problems to solve first (Tesla's seems to struggle with object tracking over time, which would be a necessary first step to recognizing patterns).
- sonofhans 4y ago
  Driving is a social problem, not a technical one. It's functionally the same as walking down a crowded sidewalk. The car is just a tool, just an extension of our bodies. We can't build a robot which can walk down a sidewalk without running into people either. The sensor tech and mapping fidelity are red herrings. People drive well because only people are good at predicting human behavior.
  - raydiatian 4y ago
    Sort of. Your wording actually assumes down to its core that driving is inherently social, when in fact I think it may be only provisionally social. Driving currently “is social” in a few senses, But of course the important (and obvious) one is that it currently involves accounting for human-operated vehicles. Alternatively, an autonomous vehicle operator in a homogenous network full of other autonomous operators has capabilities and characteristics that greatly simplify failure modes. Maybe even majority autonomous, partially heterogeneous? You can literally slow or stop the whole show to deal with a catastrophic event. It’s still “social” but probably much reduced from the scenario where you’ve got the full scope of human expressivity behind the wheel. The REAL problem is how do we take our roads to the crossover point where those simplified network features become accessible.
    - aidenn0 4y ago
      This assumes that roads are used exclusively by autonomous vehicles, when our roads are not used exclusively by motor vehicles to begin with.
      - raydiatian 4y ago
        > “This assumes our roads are used exclusively by autonomous vehicles” I not only assume it, I say it out loud: “[fully autonomous, …] maybe even majority homogenous, partly heterogenous?” > “When our roads are not used exclusively by motor vehicles to begin with.” I’m not following what you’re saying here in the context of the earlier clause. If you mean not used exclusively by autonomous vehicles, yes, and that’s why I’m pointing out the provisional aspect.
- rsynnott 4y ago
  > Any reason why everyone seems to be stuck on this problem? Because it's really, really difficult. A lot of AI-ish stuff pretty rapidly gets to the point where it _looks_ quite impressive, but struggles to make the jump to actual feasibility. Like, there were convincing demos of voice recognition in the mid-90s. You could buy software to transcribe voice on your home computer, and people did. And, now, well, it's better than in the mid-90s certainly, but you wouldn't trust it to write a transcript, not of anything important. Maybe in 2040 we'll have voice recognition that can produce a perfect transcript, and human transcription will be a quaint old-fashioned concept. But I wouldn't like to bet on it, honestly. And voice recognition is arguably a far, far easier problem.
  - UncleOxidant 4y ago
    I'm old enough to remember all the breathless pronouncements in 2015 about how self-driving cars would be everywhere within 5 years. I was skeptical then and I'm still skeptical that it will ever happen outside of some limited range of well known paths that have been pre-determined to be safe - ie we'll have (and we do have) some self driving cars but they'll essentially be on a closed course and will not leave the course. There are already some small bus lines like that - the bus just goes around a closed circuit picking up and dropping off people always taking the same route.
    - sorry_outta_gas 4y ago
      we're still trying to convert to electric, self dirving is far out there
      - xnx 4y ago
        These are mostly unrelated. Some of Waymo's vehicles are ICE.
        sorry_outta_gas 4y ago
        My mistake, I trying to imply to `mass adpotion of` and bungled it
- turzmo 4y ago
  We often can figure out how to take a product that works 90% and bring it to 99%, and then to 99.9%. The engineering challenges involved in each nine are often vastly more than the percentages indicate, but they're conceivable. With AI we have absolutely no idea how much effort might be required to get to that next level of reliability. We hope that bigger models, or better AI technology might get us there, but there's also a chance that they won't.
- deleted 4y ago
  [deleted]
- relueeuler 4y ago
  Zoox has solved it.
  - AlotOfReading 4y ago
    Has Zoox even gotten the permits to do fleet-scale driverless testing? Last I heard, they were still limited to 2 vehicles on specific streets in foster city. Doesn't seem very solved.
- deleted 4y ago
  [deleted]
- toss1 4y ago
  Google/Alphabet is stuck and will make advances in specific territories but will never get there without a fundamental change in approach. Their approach relies on very detailed mapping/modeling of specific terrain, so they can make a usable case sooner, but outside of the map/model territory, they're literally lost. And maps/models change constantly and rapidly. Tesla is taking a fundamentally more broad and deep approach - working with the fundamental fact that a pair of visual sensors and a compute engine (eyes & brain) can successfully figure out driving in strange areas in real time, ergo, it should be possible without a map/model or lidar. Once they get it solved, it will be solved once and for all. Bigger gamble, bigger payoff. Equipping the car with dozens of eyes is the easy part. The question is whether enough compute power can be brought to bear on solving the recognition problems, and the edge cases. They have obvious issues with failing to recognize large objects like trucks in unexpected orientations, left turns etc. Using millions of miles of live human driver data as a training set is great, except that the average driver is really bad, so it's entirely polluted with bad examples, ESPECIALLY around the edge cases that get people killed. There, examples from professionally trained drivers, who really understand the physics and limits of the car, adhesion, traffic dynamics, etc, are what you want to train on, but that isn't what they have. It is also possible that even if the set of training data would actually be sufficient, the big question will kill them - perhaps the solution requires orders of magnitude more compute power to approach human performance, and they just don't have the hardware to simulate human compute power. So, have they just hit the limits of what their compute power can do? I think Tesla's approach is fundamentally the way to go, as it is a general solution, compared to everyone else's limited map/model approach. But both may require either or both a more specifically programmed higher-level behaviors, and/or something much closer to AGI than exists, something that has actual understanding of the machine-learned objects and relationships, which does not yet exist (if one is known, pleas correct me - I'd love to know about it).
- Animats 4y ago
  Waymo and Cruise routinely have driverless cars on city streets. In California, all collisions, however minor, have to be reported, and DMV posts them on their web site.[1] Most are very minor. Here's a more serious one from last month: "A Cruise autonomous vehicle ("Cruise AV") operating in driverless autonomous mode, was traveling eastbound on Geary Boulevard toward the intersection with Spruce Street. As it approached the intersection, the Cruise AV entered the left hand turn lane, turned the left turn signal on, and initiated a left turn on a green light onto Spruce Street. At the same time, a Toyota Prius traveling westbound in the rightmost bus and turn lane of Geary Boulevard approached the intersection in the right turn lane. The Toyota Prius was traveling approximately 40 mph in a 25 mph speed zone. The Cruise AV came to a stop before fully completing its turn onto Spruce Street due to the oncoming Toyota Prius, and the Toyota Prius entered the intersection traveling straight from the turn lane instead of turning. Shortly thereafter, the Toyota Prius made contact with the rear passenger side of the Cruise AV. The impact caused damage to the right rear door, panel, and wheel of the Cruise AV. Police and Emergency Medical Services were called to the scene, and a police report was filed. The Cruise AV was towed from the scene. Occupants of both vehicles received medical treatment for allegedly minor injuries." Now, this shows the strengths and weaknesses of the system. The Cruise vehicle was making a left turn from Geary onto Spruce. Eastbound Geary at this point has a dedicated left turn lane cut out of a grass median, two through lanes, a right turn bus/taxi lane, and a bus stop lane. It detected cross traffic that shouldn't have been in that lane and was going too fast. So it stopped, and was hit. It did not take evasive action, which might have worked. Or it might have made the situation worse. By not doing so, it did the legally correct thing. The other driver will be blamed for this. But it may not have done the thing most likely to avoid an accident. This is the real version of the trolley problem. [1] https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/autonomous-vehicle-collision-reports/ https://www.dmv.ca.gov/portal/vehicle-industry-services/auto... [2] https://earth.google.com/web/@37.78169591,-122.45337171 https://earth.google.com/web/@37.78169591,-122.45337171 [3] https://patch.com/california/san-francisco/speed-limit-lowered-parts-geary-boulevard https://patch.com/california/san-francisco/speed-limit-lower...
  - ryantgtg 4y ago
    > In California, all collisions, however minor, have to be reported. Just to be clear, this statement only applies to AVs, right?
- uoaei 4y ago
  > Any reason why everyone seems to be stuck on this problem? Because they're all trying visual- or line-of-sight methods only, I call this the "robo-human" fallacy in ML: trying to automate the processes that humans undergo so that you eventually have a drop-in replacement for a human. But that is a myopic and unimaginative approach because you could be re-assessing the system itself and eliminating inefficiencies that lead to poor performance. In the autonomous vehicles space, there is massive potential for self-organizing swarm algorithms to control pelotons of cars, rather than individual cars with no intrinsic sense of the general flow of traffic. You wouldn't need a top-down "commander" style architecture, it could be designed so that cars only talk to their immediate neighbors and emergent patterns keep traffic flowing smooth and fast. I have always been skeptical of the attempts to reduce the amount of information about the road that a car receives. (Moving from stereoscopic to monocular vision to save the cost of one camera seems just stupid.) But people who dream of "smart cities" really seem to see little more than The Jetsons in their mind, and it limits the scope of research to our detriment.
  - dlkf 4y ago
    I’d like to hear an actual response from the people downvoting this comment. It’s an interesting perspective, and while I can imagine legit criticisims (SPOF, privacy concerns, coordination with vehicles outside the system, etc) actually articulating them would yield a more productive discussion than downvoting without comment.
01100011 4y ago
Isn't Tesla supposed to be producing Optimus, their human-like android, next year? Elon has been over-promising(i.e. flat out lying) about self-driving every year since.. 2014(there's a youtube video compilation of it)? It seems like his strategy is to just come up with increasingly grandiose promises every year when he fails to deliver on his past promises. He's trapped in his swirling vortex of bullshit. Very worrying to see Karpathy leaving...
- akmarinov 4y ago
  Elon in 2024: “by 2026 we’ll have actual, real teleportation” Elon in 2026: “by 2028 we’ll have FTL drives” Elon in 2028: “time machine!”
  - gruturo 4y ago
    > Elon in 2028: “time machine!” Well, to be fair, he only has to hit _that_ goal - at which point he can go back in time at his leisure and fix all the others. And he could hit even the time machine goal as late as he wants, and it won't matter.
    - aidenn0 4y ago
      This sounds like solid evidence for the fact that he won't ever get his hands on a time machine.
      - sdenton4 4y ago
        Or it'll be like the time machine in Primer, where you can't go back before the time machine was turned on.
        mstipetic 4y ago
        That kinda only works if we’re the only civilization in the universe, right? What if some other civilization discovered it a long time ago?
glintik 4y ago
> if he felt they were close to solving the problem anytime soon He felt? It's evident enough, that approach they used doesn't allow them to prepare FSD for real life and real streets. I think he just understood, that approach to be changed/improved significantly to reach the goal.
dreamcompiler 4y ago
Uber bet the company on robotaxis and lost. Tesla is still building very good cars that happen to not be able to drive themselves. Just like every other car. If they could lose their obsession with self-driving and just focus on their incredible cars, they'd still make money.
- deleted 4y ago
  [deleted]
- rootusrootus 4y ago
  > If they could lose their obsession with self-driving and just focus on their incredible cars, they'd still make money. They may want to think about that strategy soon. Model 3 is starting to seem dated (not to mention Model S, which is ten years old). There are very competitive alternatives on the market now that have strengths where Tesla is weak, and which are not especially weak in the areas Tesla is strong.
  - notJim 4y ago
    In what sense is the Model 3 seeming dated? It's still one of the best electric cars on important measures like range and efficiency. It also has access to hands down the best charging network and is well loved by its owners, despite the well-documented problems.
    - bhauer 4y ago
      Bjørn Nyland's review of the 2022 Model 3 Performance [1] confirms your view that the Model 3 is absolutely not dated. He tests scores of EVs and still considers the Model 3 the best balance of comfort, features, technology, performance, etc. If anybody here is seriously concerned that the Model 3 may be "dated," I suggest watching the video below. [1] https://www.youtube.com/watch?v=QdMwGJBFUd4 https://www.youtube.com/watch?v=QdMwGJBFUd4
      - ahahahahah 4y ago
        Hahaha. Yes, very honest to link to a massive fanboy as though he's an unbiased source.
rsynnott 4y ago
> but their vision only tech stack doesn’t seem capable of solving it Well, I'm not sure that anyone's tech stack is capable of solving it; the live examples of robotaxis are, well, not something you'd bet your company on (and generally their creators are _not_ betting their companies on them). There was, I think, a decade ago the idea that fully self-driving cars were a near-term inevitability. That's fading, now.
- TheDarkestSoul 4y ago
  I think a lot of that came from the Tesla hype machine creating a strong association between electric and self-driving as being the immediate future of cars in popular consciousness, so when people saw electric becoming a reality they assumed self-driving was right around the corner when in actuality their maturity levels aren't related much at all. Fallacious thinking that may doom a few companies between Lyft, Uber, and Tesla
  - rootusrootus 4y ago
    This is a good point. I think the market has to adjust to a reality where electric cars are just cars that happen to be electric, rather than a hyped up techno-dream. Electric propulsion all by itself is pretty great, we don't need to tie it to FSD dreams.
- pas 4y ago
  the low point on the hype cycle, probably we underestimate the actual capability (because our expectations were based on years of overestimation)
duped 4y ago
> Tesla has bet the company on robotaxis, How so? They're not selling robotaxis or building factories to build them > Tesla has repeatedly promised FSD is right around the corner Which means it's years away and/or "FSD" means "automatic cruise control and lane keep assist" or whatever standard feature from auto manufacturers they've renamed
- clouddrover 4y ago
  > How so? Because they chose to back themselves into that corner. Musk says that Tesla is worth nothing without full self-driving. Certainly it's the only thing left to justify the stock price: https://electrek.co/2022/06/15/elon-musk-solving-self-driving-difference-between-tesla-worth-a-lot-or-nothing/ https://electrek.co/2022/06/15/elon-musk-solving-self-drivin... > Which means it's years away and/or "FSD" means "automatic cruise control and lane keep assist" Well, more precisely it means Musk has been lying about it for nine years straight: https://jalopnik.com/elon-musk-promises-full-self-driving-next-year-for-th-1848432496 https://jalopnik.com/elon-musk-promises-full-self-driving-ne... The lies have been profitable so far. People have bought into the false promises. Perhaps they'll start demanding refunds for the full self-driving they paid for that has still not been delivered.
  - electric_mayhem 4y ago
    Consumer and grid-scale battery storage Solar And I also think they could do some clever stuff with home HVAC, possibly using waste heat from crypto miners as the H. Lastly, afaik they went camera-only in their cheap cars (3/Y) and still use fancy stuff in the S/X. They gave fsd customers new computers once. What’s to stop them from going back to vision+lidar or whatever once the parts are available and retrofitting as needed? The whole ‘vision-only is better’ gag seemed like an obvious ploy to keep being able to ship cars from the beginning of supply chain problems. And yeah, Elon does not appear to be a good person.
    - saurik 4y ago
      https://insideevs.com/news/570053/tesla-models-modelx-no-radar/ https://insideevs.com/news/570053/tesla-models-modelx-no-rad... > Tesla has just confirmed that it has removed radar from the Model S and Model X as of mid-February 2022, moving its entire lineup to what it calls ‘Tesla Vision,’ which is an array of cameras that Tesla says negates the need for radar. The manufacturer did the same for the Model 3 and Model Y in May of last year and even though this prompted some questions from the IIHS, the institute is now fine with it after testing.
ra7 4y ago
> This announcement comes after a 4 month sabbatical where Karpathy said he wanted to take some time off to “sharpen my technical edge,” which makes it sound like this is the result of frustration with the technical approach instead of burnout. I think Karpathy realized (probably way back) that cheap sensors + no HD maps + their (reckless) public testing feedback loop doesn't advance towards L5 self driving and is bailing out. Karpathy has always backed Elon Musk whenever he talks about their technical approach, so it can't be frustration with the approach all of a sudden.
- bitL 4y ago
  Maybe he was just updating himself on the latest DL trends like using transformers in vision and rethinking his original approach?
lazyjones 4y ago
> doesn’t seem capable of solving it, How so? If humans can master driving with 2 eyes looking forward, why would a car with plenty of cameras in all directions not have sufficient sensory input to master it? The problem is the software, not the sensors.
- ra7 4y ago
  Same reason airplanes don't fly by just flapping their wings like birds. There's not always a biological equivalent for solving a problem, especially when you take into account human brain's evolution over millions of years. Sometimes computers need more help.
  - dekhn 4y ago
    It does seem reasonable, though, that if humans can solve safe driving with a brain mass (3 lbs) that computers wouldn't need that much more help. Put another way: every machine learning "expert" who has a kid is amazed to learn how simple unsupervised learning is and how few training examples are required.
  - notsapiensatall 4y ago
    Large birds don't fly by flapping their wings when they have enough lift. Look at one in a thermal updraft sometime, it's quite a majestic sight. If birds had engines on their wings, they would probably fly more like airplanes. But cameras are a poor substitute for human vision, because they can't move or pivot or refocus much.
- wahnfrieden 4y ago
  not much similar between eyes (incl the full sensory processing) and camera sensors you're using elon's own argument btw, are you repeating that knowingly
- giantrobot 4y ago
  You use far more input when you drive than just your eyes. Suggesting otherwise is just obtuse.
lcnmrn 4y ago
The only wait to solve it is to have smart roads. Basically, moving the problem domain from the car to the street.
- macNchz 4y ago
  I've long had this fantasy of a smart road pilot program wherein manufacturers would partner with governments (or toll road owners in some places?) to make self-driving-only smart lanes, where you have to surrender control but the car goes 120+mph. I imagine getting even a handful of popular longer routes enabled for that would be quite popular.
  - tapatio 4y ago
    Makes sense. It's like having a train but you can take your Pullman off the "track" for "last mile" transport.
- dirkg 4y ago
  how is that any different from geofencing. The goal of an AV should be a car that can drive in any country/road. Not just fancy smart roads in western countries
- thatswrong0 4y ago
  I don't see how smart roads would solve the issues of "the unexpected".. weird pedestrian behavior, getting cut off, parked trucks, construction, etc. It seems to me that smart roads would only solve the issue of general routing, and that already seems to be dealt with as far as I can tell.
  - vanilla_nut 4y ago
    It wouldn't. A real "smart road" implementation would require us to literally fence off those roads so pedestrians, deer, children, bicyclists, etc. cannot use them. And they would likely fail in any kind of inclement weather -- fog, snow, rain, etc. Essentially an admission of failure for the self-driving car industry.
pbreit 4y ago
"Tesla has bet the company on robotaxis, but their vision only tech stack doesn’t seem capable of solving it" Both clauses seem wrong.
harles 4y ago
Monocular vision only seems pretty clearly not capable of solving the problem. Stereo/multi view systems have a shot (humans are proof), but Tesla bet against that long ago. I wonder what could’ve been in a proper multi view setup.
- pilaf 4y ago
  Humans are perfectly capable of driving on racing simulators using a flat screen though, where binocular vision makes no difference. And Tesla cars have more than one camera on them. The front-facing camera is actually an array of 3 cameras (the two farthest ones are at about human eyes distance), but they're also equipped with forward and rearward looking side cameras, and back cameras. I think Tesla underestimated how hard vision-only FSD is, but having a single camera (they don't) is not the reason.
  - harles 4y ago
    Driving simulators are a bad example. It’s too easy to learn priors. A better analogy is humans with one eye that still drive - most are taught to induce parallax. Also, I never said they have one camera. Multi camera != multi view.
elif 4y ago
FSD beta tester here. I think they are minimum 3 years away from anything exciting beta.. but the localization, mapping, and visualization are not the reason. I don't think LIDAR would contribute substantially to improvement. The fundamental flaws are in the decision-making being based upon 10-30 second feature memory, ignoring features outright, and only depending on visible road features instead of persisted map data. For instance, near my house there's an intersection where it will try to use a turn-only lane with a red arrow when it's trying to go straight thru a light. 100% of the time. Even if I'm in the correct lane with no traffic around. That's because the turn arrow on the ground is worn off. It is not a perception problem, it is a) ignoring the obvious red turn arrow signal's significance for lane selection deliberately b) makes no attempt to persist or consult map data for 'which lanes go where' c) it completely disregards painted lines on the ground in the "no drive here" striping. Also one block from my house, FSD will stay still indefinitely waiting for trash cans (displayed as trash cans) to clear the leftmost lane so that it will turn left. None of the failures I encounter are due to lack of perception.
- ncmncm 4y ago
  The elephant in the room is fraud. Tesla has been collecting thousands of dollars, each, from car buyers and utterly failing to deliver what it represented, and keeping the money year after year. Would be let GM, Toyota, or Audi do this? Where is the criminal prosecution? Where are the refunds?
  - elif 4y ago
    I wouldn't call it fraud. I bought it knowing it was in beta and with no timeline for production release. They haven't represented it as anything more than that. Being along for the journey is a big part of why I decided to pay.
    - asdajksah2123 4y ago
      Elon Musk has literally said in a public conference keynote that robotaxis would be running in 2019, and earning Tesla owners 30k/yr since 3 years ago, and that it would be financially foolish to buy anything but a Tesla.
moralestapia 4y ago
>which makes it sound like this is the result of frustration with the technical approach instead of burnout LOL no, he was jumping ship already.
lostmsu 4y ago
He contributed some commits to his excellent https://github.com/karpathy/minGPT https://github.com/karpathy/minGPT during that time. BTW, Andrej, if you're reading this, it is not just excellent it is beyond excellent. I do a lot of tinkering with transformers and other models lately, and base them all on minGPT. My fork is now growing into a kind of monorepo for deep learning experimentation, though lately it started looking like a repo of Theseus, and the boat is not as simple anymore :)
HWR_14 4y ago
> This announcement comes after a 4 month sabbatical where Karpathy said he wanted to take some time off to “sharpen my technical edge,” which makes it sound like this is the result of frustration with the technical approach instead of burnout. That's exactly what I would expect someone burning out to say. You feel the burnout so you need time to get over it and feel 100% (regain your technical edge). You're still burnt out after 4 months, so you don't come back. Frustration with the technical approach can also cause burnout.
FireBeyond 4y ago
> but their vision only tech stack Tesla filed with the FCC in May to get authorization for a new radar system.
osuairt 4y ago
Yes, the vision only approach isn't something that you do in robotics. There is usually a hierarchy of sensors, mainly for redundancy. Example: Bumper sensory at the wheel base, sonar / Lidar at the mid, and a camera at the top for advanced sensing. For the sake of cost cutting Tesla has done away with their radar sensors at the front of the vehicle. It would be a substantial cost overhead, but have very real repercussions when it comes to safety, while also providing a "ground truth" to what at least the front facing cameras are seeing. I don't think Lidar is a practical sensor for them to adopt, because it is quite bulky and has limited viewing angles, but I would expect them to have adopted some novel, lower cost radar solution. Apart from the lower cost of the camera, I think Elon's rationale for having a camera only FSD is not valid, has made the problem needlessly complex and unsafe. He believes since we have eyes, and we can drive a car, then it should be sufficient to drive the car, but we only use eyes because these are the sensors we were born with, it is the best we have. In my mind, Elon's approach is like looking at a horse, and saying to yourself, that you want to build a car based on a horse, where instead of wheels, you have four mechanical legs, and those mechanical legs are limited is so many ways, but they should still at least "work", but there is no reason to limit locomotion in that way. The same with the vision system on a FSD, the whole spectrum of light is available, with any number of configurations, providing data at rates and with precision far beyond what a camera system can do.
- agoose77 4y ago
  IIRC there was a presentation from Karpathy talking about the challenges with sensor fusion, particular in resolving divergence between e.g. the vision and the radar stack: https://www.youtube.com/watch?v=NSDTZQdo6H8&t=1949s https://www.youtube.com/watch?v=NSDTZQdo6H8&t=1949s My background is in physics, but I find myself having a growing appreciate for the vision-only stack. It's really challenging building a formal understanding of the world that is robust to outliers that are so numerous as navigating in an urban environment. With vision, you have multiple kinds of information that are highly correlated (colour, spatial distribution, depth, etc) that are self-consistent. Whereas, fusing radar with vision, where object responses to radar are highly geometry & material dependent, is a much harder task. I'm really not an expert, so this reads more as an opinion than an experienced view, but I can see the merits in doubling down on vision.

impulser_ 4y ago

Yeah, but Google's vision + lidar tech also doesn't seem any better at solving it either. They have been working on this problem the longest and they aren't even confident enough to produce a product with it. Google is probably the leader in AI and AI research. They are also the leader in data and mapping. They have billions of cash to play with. Yet it seem like they haven't gotten any closer at solving this problem as well. They are just going about it better but not trying to selling it. Any reason why everyone seems to be stuck on this problem?

realitysballs 4y ago
Answer me this; How is AI supposed to confidently distinguish a real stop sign from someone/something holding up a picture of a stop sign? Yes, this is a weird edge case, but I think it gets at the core issue being that it takes way more sophistication to release this tech into the wild then ppl would like to admit.
Barrin92 4y ago
>Any reason why everyone seems to be stuck on this problem? ML maximalism focused on the narrow problem of 'solving driving' while not recognizing that any task as complex as driving requires probably something closer to general intelligence, and theoretically the field has been impoverished in favor of "throw more graphics cards at everything".
- hn_throwaway_99 4y ago
  Couldn't agree more. Especially when it comes to city driving, which would obviously be necessary for robotaxis, when AI zealots promise "it'll be here in a year or two", I always wondered "Have these people ever driven in the city?" I mean, to drive in a city, you basically: 1. Need to understand all standard signage (seems possible with AI). 2. Need to understand all "unstandard" signage (not sure how possible). 3. Need to understand the cop with the thick NY accent yelling at you saying "Can't you see there's been an accident and the road is covered with glass you dufus? Turn the F around." I can certainly see AI solving the problem of driving in specially designed limited access highways (which could also support normal human drivers), and that alone would be a huge benefit, but I never saw how so many were willing to make the leap to "robotaxis that can drive you anywhere in the city."
  - dmd 4y ago
    4. Need to understand the cop who is directing people into lanes by jutting his chin subtly in different directions when you make eye contact with him 5. Need to understand that occluded objects have not vanished from the universe never to be seen again
    - naijaboiler 4y ago
      6. Need to reasonably predict what that human that just made eye contact with you would likely do next, and how that's different from what he might do when he doesn’t make eye contact with you. And all of that differs if you're in NYC or SF or small town, Indiana
      - dibujante 4y ago
        7. Need to understand that drunk person staggering along the roadside has been repeatedly slipping off the sidewalk and there's a non-zero chance they trip and fall right in front of you.
        4y ago
Jabbles 4y ago
> doesn't seem any better at solving it I thought they had real self-driving taxis in Pheonix that you can order? Real ones, with no safety driver. That definitely sounds "better", even if it is heavily geo-fenced.
- jowday 4y ago
  They’ve also got real ones with no safety driver in San Francisco right now.
- impulser_ 4y ago
  Yeah, but I am sure Tesla's software can do the same. Depending on the route, you could probably even do it with comma.ai hardware. When I think of FSD, I think any route under any condition.
  - kadoban 4y ago
    > Yeah, but I am sure Tesla's software can do the same. They absolutely cannot. They won't even try, they require a driver to be there to be ready to take over with no notice. Their software also makes so many basic mistakes that even what they're allowing it to do is dangerously reckless.
- espadrine 4y ago
  Waymo has superior performance based on their historical statistics. It makes sense, since their lidar sensors capture more of the environment, and directly in 3D. Their AI also seems better QA’ed. The Tesla AI Day[0] surprised me as it showed they only had a simple architecture for a very long time, simply feeding barely processed camera pixels to a DNN and hoping for the best with little more than supervised learning off human feeds. Their big claim to glory was that they rearchitectured it to produce a 2D map of the environment… which I thought they had years ago, and is still a far cry from the 3D modeling that is needed. After all, sure, we humans only take two video feeds as input… But we can appreciate from it the position, intent, and trajectory of a wealth of elements around us, with reasonable probability estimates for multiple possibilities, sometimes pertaining to things that are invisible, such as kids crossing from nowhere when near a school. Cruise also seems to have better tech; they had a barely-watched 2h30 description of their systems[1] which shows they do create a richer environment, evaluate the routing of many objects, and train their systems on a very realistic simulation, not just supervised training, which means it can learn from very low-probability events. They have a whole segment on including the probability that unseen cars may travel from perpendicular roads; Tesla’s creeping hit-or-miss are well-documented on Youtube. [0]: https://www.youtube.com/watch?v=j0z4FweCy4M https://www.youtube.com/watch?v=j0z4FweCy4M [1]: https://www.youtube.com/watch?v=uJWN0K26NxQ https://www.youtube.com/watch?v=uJWN0K26NxQ
alphabetting 4y ago
There's been a lot of progress despite AVs not meeting intitial hyped predictions. Waymo and Cruise are operating driverless robotaxis in SF. We're probably a couple years from many major cities having them.
tootie 4y ago
Probably because it's really, really, really hard to solve the thousands of edge cases that occur in real-world driving situations. I don't think FSD happens until government gets behind it and starts putting infrastructure behind it. If we start building roads (and cars) to be highly visible to AI one way or another, it all becomes much easier.
- aaaaaaaaaaab 4y ago
  >If we start building roads (and cars) to be highly visible to AI one way or another, it all becomes much easier. Or we could build 1-dimensional roads, which would make the AI's jobs much easier. Like, we could put down two parallel piece of metal, which vehicles could "hook" onto somehow...
cco 4y ago
I paid ~$10 for two rides after signing up as a regular ole user in Mesa AZ. It was great, the first ride was a bit nerve wracking, but the second felt very nice. I certainly wouldn't argue with you that it isn't ready for prime time and wide distribution, but it is interesting to see their progress in San Francisco, a much different driving problem. If it takes them 10 years to get to prod in Mesa, two (maybe three?) in SF, maybe they start shrinking that a lot in metros without winters. ¯\_(ツ)_/¯
dreamcompiler 4y ago
Because self-driving has a bunch of tricky edge cases and most of them will kill people. Problems with hundreds of important edge cases cannot be solved by simply throwing more training data at the problem; that's how you solve AI problems in a "dumb" manner, and it works for lots of problems (like recognizing dogs in images) -- but not for self-driving. To solve the self-driving problem we need "smart" A.I., which means we have to approach it with systematic engineering, and the solution will probably involve some combination of better sensors, introspectable neural nets, symbolic A.I., and logical A.I.
- naijaboiler 4y ago
  And we haven't even adressed that drving is not a purely technical endeavor, it's largely a social one.
  - MonkeyMalarky 4y ago
    A social one where you use experience to make inferences about what is going to happen next. Who hasn't observed an aggressive driver coming up from behind, weaving around other cars and said to themselves something like "that guy is going to cut me off, better ease off the gas so there's a little extra room in front for when he does". That level of situational awareness isn't coded for is it? Edit: I have a whole mental model for other drivers and different approaches for them. Someone driving like a grandma? Pass when available. Nervous/erratic/lost driver? Keep extra distance then pass as soon as possible. Aggressive driver? Relax, give some space and let them get ahead. And so on. I get that stereotyping is bad but ignoring the subtle signals other drivers give off seems like it would be myopic. An AI that doesn't anticipate what others will do on the road will always be reactive rather than proactive.
    - aidenn0 4y ago
      Most self-driving vehicles aren't just coded; they incorporate some form of ML as well. Categorizing patterns of behaviors is well within the reach of ML algorithms, but my understanding is we have far more basic problems to solve first (Tesla's seems to struggle with object tracking over time, which would be a necessary first step to recognizing patterns).
sonofhans 4y ago
Driving is a social problem, not a technical one. It's functionally the same as walking down a crowded sidewalk. The car is just a tool, just an extension of our bodies. We can't build a robot which can walk down a sidewalk without running into people either. The sensor tech and mapping fidelity are red herrings. People drive well because only people are good at predicting human behavior.
- raydiatian 4y ago
  Sort of. Your wording actually assumes down to its core that driving is inherently social, when in fact I think it may be only provisionally social. Driving currently “is social” in a few senses, But of course the important (and obvious) one is that it currently involves accounting for human-operated vehicles. Alternatively, an autonomous vehicle operator in a homogenous network full of other autonomous operators has capabilities and characteristics that greatly simplify failure modes. Maybe even majority autonomous, partially heterogeneous? You can literally slow or stop the whole show to deal with a catastrophic event. It’s still “social” but probably much reduced from the scenario where you’ve got the full scope of human expressivity behind the wheel. The REAL problem is how do we take our roads to the crossover point where those simplified network features become accessible.
  - aidenn0 4y ago
    This assumes that roads are used exclusively by autonomous vehicles, when our roads are not used exclusively by motor vehicles to begin with.
    - raydiatian 4y ago
      > “This assumes our roads are used exclusively by autonomous vehicles” I not only assume it, I say it out loud: “[fully autonomous, …] maybe even majority homogenous, partly heterogenous?” > “When our roads are not used exclusively by motor vehicles to begin with.” I’m not following what you’re saying here in the context of the earlier clause. If you mean not used exclusively by autonomous vehicles, yes, and that’s why I’m pointing out the provisional aspect.
rsynnott 4y ago
> Any reason why everyone seems to be stuck on this problem? Because it's really, really difficult. A lot of AI-ish stuff pretty rapidly gets to the point where it _looks_ quite impressive, but struggles to make the jump to actual feasibility. Like, there were convincing demos of voice recognition in the mid-90s. You could buy software to transcribe voice on your home computer, and people did. And, now, well, it's better than in the mid-90s certainly, but you wouldn't trust it to write a transcript, not of anything important. Maybe in 2040 we'll have voice recognition that can produce a perfect transcript, and human transcription will be a quaint old-fashioned concept. But I wouldn't like to bet on it, honestly. And voice recognition is arguably a far, far easier problem.
- UncleOxidant 4y ago
  I'm old enough to remember all the breathless pronouncements in 2015 about how self-driving cars would be everywhere within 5 years. I was skeptical then and I'm still skeptical that it will ever happen outside of some limited range of well known paths that have been pre-determined to be safe - ie we'll have (and we do have) some self driving cars but they'll essentially be on a closed course and will not leave the course. There are already some small bus lines like that - the bus just goes around a closed circuit picking up and dropping off people always taking the same route.
  - sorry_outta_gas 4y ago
    we're still trying to convert to electric, self dirving is far out there
    - xnx 4y ago
      These are mostly unrelated. Some of Waymo's vehicles are ICE.
      - sorry_outta_gas 4y ago
        My mistake, I trying to imply to `mass adpotion of` and bungled it
turzmo 4y ago
We often can figure out how to take a product that works 90% and bring it to 99%, and then to 99.9%. The engineering challenges involved in each nine are often vastly more than the percentages indicate, but they're conceivable. With AI we have absolutely no idea how much effort might be required to get to that next level of reliability. We hope that bigger models, or better AI technology might get us there, but there's also a chance that they won't.
deleted 4y ago
[deleted]
relueeuler 4y ago
Zoox has solved it.
- AlotOfReading 4y ago
  Has Zoox even gotten the permits to do fleet-scale driverless testing? Last I heard, they were still limited to 2 vehicles on specific streets in foster city. Doesn't seem very solved.
deleted 4y ago
[deleted]
toss1 4y ago
Google/Alphabet is stuck and will make advances in specific territories but will never get there without a fundamental change in approach. Their approach relies on very detailed mapping/modeling of specific terrain, so they can make a usable case sooner, but outside of the map/model territory, they're literally lost. And maps/models change constantly and rapidly. Tesla is taking a fundamentally more broad and deep approach - working with the fundamental fact that a pair of visual sensors and a compute engine (eyes & brain) can successfully figure out driving in strange areas in real time, ergo, it should be possible without a map/model or lidar. Once they get it solved, it will be solved once and for all. Bigger gamble, bigger payoff. Equipping the car with dozens of eyes is the easy part. The question is whether enough compute power can be brought to bear on solving the recognition problems, and the edge cases. They have obvious issues with failing to recognize large objects like trucks in unexpected orientations, left turns etc. Using millions of miles of live human driver data as a training set is great, except that the average driver is really bad, so it's entirely polluted with bad examples, ESPECIALLY around the edge cases that get people killed. There, examples from professionally trained drivers, who really understand the physics and limits of the car, adhesion, traffic dynamics, etc, are what you want to train on, but that isn't what they have. It is also possible that even if the set of training data would actually be sufficient, the big question will kill them - perhaps the solution requires orders of magnitude more compute power to approach human performance, and they just don't have the hardware to simulate human compute power. So, have they just hit the limits of what their compute power can do? I think Tesla's approach is fundamentally the way to go, as it is a general solution, compared to everyone else's limited map/model approach. But both may require either or both a more specifically programmed higher-level behaviors, and/or something much closer to AGI than exists, something that has actual understanding of the machine-learned objects and relationships, which does not yet exist (if one is known, pleas correct me - I'd love to know about it).
Animats 4y ago
Waymo and Cruise routinely have driverless cars on city streets. In California, all collisions, however minor, have to be reported, and DMV posts them on their web site.[1] Most are very minor. Here's a more serious one from last month: "A Cruise autonomous vehicle ("Cruise AV") operating in driverless autonomous mode, was traveling eastbound on Geary Boulevard toward the intersection with Spruce Street. As it approached the intersection, the Cruise AV entered the left hand turn lane, turned the left turn signal on, and initiated a left turn on a green light onto Spruce Street. At the same time, a Toyota Prius traveling westbound in the rightmost bus and turn lane of Geary Boulevard approached the intersection in the right turn lane. The Toyota Prius was traveling approximately 40 mph in a 25 mph speed zone. The Cruise AV came to a stop before fully completing its turn onto Spruce Street due to the oncoming Toyota Prius, and the Toyota Prius entered the intersection traveling straight from the turn lane instead of turning. Shortly thereafter, the Toyota Prius made contact with the rear passenger side of the Cruise AV. The impact caused damage to the right rear door, panel, and wheel of the Cruise AV. Police and Emergency Medical Services were called to the scene, and a police report was filed. The Cruise AV was towed from the scene. Occupants of both vehicles received medical treatment for allegedly minor injuries." Now, this shows the strengths and weaknesses of the system. The Cruise vehicle was making a left turn from Geary onto Spruce. Eastbound Geary at this point has a dedicated left turn lane cut out of a grass median, two through lanes, a right turn bus/taxi lane, and a bus stop lane. It detected cross traffic that shouldn't have been in that lane and was going too fast. So it stopped, and was hit. It did not take evasive action, which might have worked. Or it might have made the situation worse. By not doing so, it did the legally correct thing. The other driver will be blamed for this. But it may not have done the thing most likely to avoid an accident. This is the real version of the trolley problem. [1] https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/autonomous-vehicle-collision-reports/ https://www.dmv.ca.gov/portal/vehicle-industry-services/auto... [2] https://earth.google.com/web/@37.78169591,-122.45337171 https://earth.google.com/web/@37.78169591,-122.45337171 [3] https://patch.com/california/san-francisco/speed-limit-lowered-parts-geary-boulevard https://patch.com/california/san-francisco/speed-limit-lower...
- ryantgtg 4y ago
  > In California, all collisions, however minor, have to be reported. Just to be clear, this statement only applies to AVs, right?
uoaei 4y ago
> Any reason why everyone seems to be stuck on this problem? Because they're all trying visual- or line-of-sight methods only, I call this the "robo-human" fallacy in ML: trying to automate the processes that humans undergo so that you eventually have a drop-in replacement for a human. But that is a myopic and unimaginative approach because you could be re-assessing the system itself and eliminating inefficiencies that lead to poor performance. In the autonomous vehicles space, there is massive potential for self-organizing swarm algorithms to control pelotons of cars, rather than individual cars with no intrinsic sense of the general flow of traffic. You wouldn't need a top-down "commander" style architecture, it could be designed so that cars only talk to their immediate neighbors and emergent patterns keep traffic flowing smooth and fast. I have always been skeptical of the attempts to reduce the amount of information about the road that a car receives. (Moving from stereoscopic to monocular vision to save the cost of one camera seems just stupid.) But people who dream of "smart cities" really seem to see little more than The Jetsons in their mind, and it limits the scope of research to our detriment.
- dlkf 4y ago
  I’d like to hear an actual response from the people downvoting this comment. It’s an interesting perspective, and while I can imagine legit criticisims (SPOF, privacy concerns, coordination with vehicles outside the system, etc) actually articulating them would yield a more productive discussion than downvoting without comment.

01100011 4y ago

Isn't Tesla supposed to be producing Optimus, their human-like android, next year? Elon has been over-promising(i.e. flat out lying) about self-driving every year since.. 2014(there's a youtube video compilation of it)? It seems like his strategy is to just come up with increasingly grandiose promises every year when he fails to deliver on his past promises. He's trapped in his swirling vortex of bullshit. Very worrying to see Karpathy leaving...

akmarinov 4y ago
Elon in 2024: “by 2026 we’ll have actual, real teleportation” Elon in 2026: “by 2028 we’ll have FTL drives” Elon in 2028: “time machine!”
- gruturo 4y ago
  > Elon in 2028: “time machine!” Well, to be fair, he only has to hit _that_ goal - at which point he can go back in time at his leisure and fix all the others. And he could hit even the time machine goal as late as he wants, and it won't matter.
  - aidenn0 4y ago
    This sounds like solid evidence for the fact that he won't ever get his hands on a time machine.
    - sdenton4 4y ago
      Or it'll be like the time machine in Primer, where you can't go back before the time machine was turned on.
      - mstipetic 4y ago
        That kinda only works if we’re the only civilization in the universe, right? What if some other civilization discovered it a long time ago?

glintik 4y ago

> if he felt they were close to solving the problem anytime soon He felt? It's evident enough, that approach they used doesn't allow them to prepare FSD for real life and real streets. I think he just understood, that approach to be changed/improved significantly to reach the goal.

dreamcompiler 4y ago

Uber bet the company on robotaxis and lost. Tesla is still building very good cars that happen to not be able to drive themselves. Just like every other car. If they could lose their obsession with self-driving and just focus on their incredible cars, they'd still make money.

deleted 4y ago
[deleted]
rootusrootus 4y ago
> If they could lose their obsession with self-driving and just focus on their incredible cars, they'd still make money. They may want to think about that strategy soon. Model 3 is starting to seem dated (not to mention Model S, which is ten years old). There are very competitive alternatives on the market now that have strengths where Tesla is weak, and which are not especially weak in the areas Tesla is strong.
- notJim 4y ago
  In what sense is the Model 3 seeming dated? It's still one of the best electric cars on important measures like range and efficiency. It also has access to hands down the best charging network and is well loved by its owners, despite the well-documented problems.
  - bhauer 4y ago
    Bjørn Nyland's review of the 2022 Model 3 Performance [1] confirms your view that the Model 3 is absolutely not dated. He tests scores of EVs and still considers the Model 3 the best balance of comfort, features, technology, performance, etc. If anybody here is seriously concerned that the Model 3 may be "dated," I suggest watching the video below. [1] https://www.youtube.com/watch?v=QdMwGJBFUd4 https://www.youtube.com/watch?v=QdMwGJBFUd4
    - ahahahahah 4y ago
      Hahaha. Yes, very honest to link to a massive fanboy as though he's an unbiased source.

rsynnott 4y ago

> but their vision only tech stack doesn’t seem capable of solving it Well, I'm not sure that anyone's tech stack is capable of solving it; the live examples of robotaxis are, well, not something you'd bet your company on (and generally their creators are _not_ betting their companies on them). There was, I think, a decade ago the idea that fully self-driving cars were a near-term inevitability. That's fading, now.

TheDarkestSoul 4y ago
I think a lot of that came from the Tesla hype machine creating a strong association between electric and self-driving as being the immediate future of cars in popular consciousness, so when people saw electric becoming a reality they assumed self-driving was right around the corner when in actuality their maturity levels aren't related much at all. Fallacious thinking that may doom a few companies between Lyft, Uber, and Tesla
- rootusrootus 4y ago
  This is a good point. I think the market has to adjust to a reality where electric cars are just cars that happen to be electric, rather than a hyped up techno-dream. Electric propulsion all by itself is pretty great, we don't need to tie it to FSD dreams.
pas 4y ago
the low point on the hype cycle, probably we underestimate the actual capability (because our expectations were based on years of overestimation)

duped 4y ago

> Tesla has bet the company on robotaxis, How so? They're not selling robotaxis or building factories to build them > Tesla has repeatedly promised FSD is right around the corner Which means it's years away and/or "FSD" means "automatic cruise control and lane keep assist" or whatever standard feature from auto manufacturers they've renamed

clouddrover 4y ago
> How so? Because they chose to back themselves into that corner. Musk says that Tesla is worth nothing without full self-driving. Certainly it's the only thing left to justify the stock price: https://electrek.co/2022/06/15/elon-musk-solving-self-driving-difference-between-tesla-worth-a-lot-or-nothing/ https://electrek.co/2022/06/15/elon-musk-solving-self-drivin... > Which means it's years away and/or "FSD" means "automatic cruise control and lane keep assist" Well, more precisely it means Musk has been lying about it for nine years straight: https://jalopnik.com/elon-musk-promises-full-self-driving-next-year-for-th-1848432496 https://jalopnik.com/elon-musk-promises-full-self-driving-ne... The lies have been profitable so far. People have bought into the false promises. Perhaps they'll start demanding refunds for the full self-driving they paid for that has still not been delivered.
- electric_mayhem 4y ago
  Consumer and grid-scale battery storage Solar And I also think they could do some clever stuff with home HVAC, possibly using waste heat from crypto miners as the H. Lastly, afaik they went camera-only in their cheap cars (3/Y) and still use fancy stuff in the S/X. They gave fsd customers new computers once. What’s to stop them from going back to vision+lidar or whatever once the parts are available and retrofitting as needed? The whole ‘vision-only is better’ gag seemed like an obvious ploy to keep being able to ship cars from the beginning of supply chain problems. And yeah, Elon does not appear to be a good person.
  - saurik 4y ago
    https://insideevs.com/news/570053/tesla-models-modelx-no-radar/ https://insideevs.com/news/570053/tesla-models-modelx-no-rad... > Tesla has just confirmed that it has removed radar from the Model S and Model X as of mid-February 2022, moving its entire lineup to what it calls ‘Tesla Vision,’ which is an array of cameras that Tesla says negates the need for radar. The manufacturer did the same for the Model 3 and Model Y in May of last year and even though this prompted some questions from the IIHS, the institute is now fine with it after testing.

ra7 4y ago

> This announcement comes after a 4 month sabbatical where Karpathy said he wanted to take some time off to “sharpen my technical edge,” which makes it sound like this is the result of frustration with the technical approach instead of burnout. I think Karpathy realized (probably way back) that cheap sensors + no HD maps + their (reckless) public testing feedback loop doesn't advance towards L5 self driving and is bailing out. Karpathy has always backed Elon Musk whenever he talks about their technical approach, so it can't be frustration with the approach all of a sudden.

bitL 4y ago
Maybe he was just updating himself on the latest DL trends like using transformers in vision and rethinking his original approach?

lazyjones 4y ago

> doesn’t seem capable of solving it, How so? If humans can master driving with 2 eyes looking forward, why would a car with plenty of cameras in all directions not have sufficient sensory input to master it? The problem is the software, not the sensors.

ra7 4y ago
Same reason airplanes don't fly by just flapping their wings like birds. There's not always a biological equivalent for solving a problem, especially when you take into account human brain's evolution over millions of years. Sometimes computers need more help.
- dekhn 4y ago
  It does seem reasonable, though, that if humans can solve safe driving with a brain mass (3 lbs) that computers wouldn't need that much more help. Put another way: every machine learning "expert" who has a kid is amazed to learn how simple unsupervised learning is and how few training examples are required.
- notsapiensatall 4y ago
  Large birds don't fly by flapping their wings when they have enough lift. Look at one in a thermal updraft sometime, it's quite a majestic sight. If birds had engines on their wings, they would probably fly more like airplanes. But cameras are a poor substitute for human vision, because they can't move or pivot or refocus much.
wahnfrieden 4y ago
not much similar between eyes (incl the full sensory processing) and camera sensors you're using elon's own argument btw, are you repeating that knowingly
giantrobot 4y ago
You use far more input when you drive than just your eyes. Suggesting otherwise is just obtuse.

lcnmrn 4y ago

The only wait to solve it is to have smart roads. Basically, moving the problem domain from the car to the street.

macNchz 4y ago
I've long had this fantasy of a smart road pilot program wherein manufacturers would partner with governments (or toll road owners in some places?) to make self-driving-only smart lanes, where you have to surrender control but the car goes 120+mph. I imagine getting even a handful of popular longer routes enabled for that would be quite popular.
- tapatio 4y ago
  Makes sense. It's like having a train but you can take your Pullman off the "track" for "last mile" transport.
dirkg 4y ago
how is that any different from geofencing. The goal of an AV should be a car that can drive in any country/road. Not just fancy smart roads in western countries
thatswrong0 4y ago
I don't see how smart roads would solve the issues of "the unexpected".. weird pedestrian behavior, getting cut off, parked trucks, construction, etc. It seems to me that smart roads would only solve the issue of general routing, and that already seems to be dealt with as far as I can tell.
- vanilla_nut 4y ago
  It wouldn't. A real "smart road" implementation would require us to literally fence off those roads so pedestrians, deer, children, bicyclists, etc. cannot use them. And they would likely fail in any kind of inclement weather -- fog, snow, rain, etc. Essentially an admission of failure for the self-driving car industry.

pbreit 4y ago

"Tesla has bet the company on robotaxis, but their vision only tech stack doesn’t seem capable of solving it" Both clauses seem wrong.

harles 4y ago

Monocular vision only seems pretty clearly not capable of solving the problem. Stereo/multi view systems have a shot (humans are proof), but Tesla bet against that long ago. I wonder what could’ve been in a proper multi view setup.

pilaf 4y ago
Humans are perfectly capable of driving on racing simulators using a flat screen though, where binocular vision makes no difference. And Tesla cars have more than one camera on them. The front-facing camera is actually an array of 3 cameras (the two farthest ones are at about human eyes distance), but they're also equipped with forward and rearward looking side cameras, and back cameras. I think Tesla underestimated how hard vision-only FSD is, but having a single camera (they don't) is not the reason.
- harles 4y ago
  Driving simulators are a bad example. It’s too easy to learn priors. A better analogy is humans with one eye that still drive - most are taught to induce parallax. Also, I never said they have one camera. Multi camera != multi view.

elif 4y ago

FSD beta tester here. I think they are minimum 3 years away from anything exciting beta.. but the localization, mapping, and visualization are not the reason. I don't think LIDAR would contribute substantially to improvement. The fundamental flaws are in the decision-making being based upon 10-30 second feature memory, ignoring features outright, and only depending on visible road features instead of persisted map data. For instance, near my house there's an intersection where it will try to use a turn-only lane with a red arrow when it's trying to go straight thru a light. 100% of the time. Even if I'm in the correct lane with no traffic around. That's because the turn arrow on the ground is worn off. It is not a perception problem, it is a) ignoring the obvious red turn arrow signal's significance for lane selection deliberately b) makes no attempt to persist or consult map data for 'which lanes go where' c) it completely disregards painted lines on the ground in the "no drive here" striping. Also one block from my house, FSD will stay still indefinitely waiting for trash cans (displayed as trash cans) to clear the leftmost lane so that it will turn left. None of the failures I encounter are due to lack of perception.

ncmncm 4y ago
The elephant in the room is fraud. Tesla has been collecting thousands of dollars, each, from car buyers and utterly failing to deliver what it represented, and keeping the money year after year. Would be let GM, Toyota, or Audi do this? Where is the criminal prosecution? Where are the refunds?
- elif 4y ago
  I wouldn't call it fraud. I bought it knowing it was in beta and with no timeline for production release. They haven't represented it as anything more than that. Being along for the journey is a big part of why I decided to pay.
  - asdajksah2123 4y ago
    Elon Musk has literally said in a public conference keynote that robotaxis would be running in 2019, and earning Tesla owners 30k/yr since 3 years ago, and that it would be financially foolish to buy anything but a Tesla.

moralestapia 4y ago

>which makes it sound like this is the result of frustration with the technical approach instead of burnout LOL no, he was jumping ship already.

lostmsu 4y ago

He contributed some commits to his excellent https://github.com/karpathy/minGPT https://github.com/karpathy/minGPT during that time. BTW, Andrej, if you're reading this, it is not just excellent it is beyond excellent. I do a lot of tinkering with transformers and other models lately, and base them all on minGPT. My fork is now growing into a kind of monorepo for deep learning experimentation, though lately it started looking like a repo of Theseus, and the boat is not as simple anymore :)

HWR_14 4y ago

> This announcement comes after a 4 month sabbatical where Karpathy said he wanted to take some time off to “sharpen my technical edge,” which makes it sound like this is the result of frustration with the technical approach instead of burnout. That's exactly what I would expect someone burning out to say. You feel the burnout so you need time to get over it and feel 100% (regain your technical edge). You're still burnt out after 4 months, so you don't come back. Frustration with the technical approach can also cause burnout.

FireBeyond 4y ago

> but their vision only tech stack Tesla filed with the FCC in May to get authorization for a new radar system.

osuairt 4y ago

Yes, the vision only approach isn't something that you do in robotics. There is usually a hierarchy of sensors, mainly for redundancy. Example: Bumper sensory at the wheel base, sonar / Lidar at the mid, and a camera at the top for advanced sensing. For the sake of cost cutting Tesla has done away with their radar sensors at the front of the vehicle. It would be a substantial cost overhead, but have very real repercussions when it comes to safety, while also providing a "ground truth" to what at least the front facing cameras are seeing. I don't think Lidar is a practical sensor for them to adopt, because it is quite bulky and has limited viewing angles, but I would expect them to have adopted some novel, lower cost radar solution. Apart from the lower cost of the camera, I think Elon's rationale for having a camera only FSD is not valid, has made the problem needlessly complex and unsafe. He believes since we have eyes, and we can drive a car, then it should be sufficient to drive the car, but we only use eyes because these are the sensors we were born with, it is the best we have. In my mind, Elon's approach is like looking at a horse, and saying to yourself, that you want to build a car based on a horse, where instead of wheels, you have four mechanical legs, and those mechanical legs are limited is so many ways, but they should still at least "work", but there is no reason to limit locomotion in that way. The same with the vision system on a FSD, the whole spectrum of light is available, with any number of configurations, providing data at rates and with precision far beyond what a camera system can do.

agoose77 4y ago
IIRC there was a presentation from Karpathy talking about the challenges with sensor fusion, particular in resolving divergence between e.g. the vision and the radar stack: https://www.youtube.com/watch?v=NSDTZQdo6H8&t=1949s https://www.youtube.com/watch?v=NSDTZQdo6H8&t=1949s My background is in physics, but I find myself having a growing appreciate for the vision-only stack. It's really challenging building a formal understanding of the world that is robust to outliers that are so numerous as navigating in an urban environment. With vision, you have multiple kinds of information that are highly correlated (colour, spatial distribution, depth, etc) that are self-consistent. Whereas, fusing radar with vision, where object responses to radar are highly geometry & material dependent, is a much harder task. I'm really not an expert, so this reads more as an opinion than an experienced view, but I can see the merits in doubling down on vision.