5 ms·
AI pioneer Geoff Hinton: “Deep learning is going to be able to do everything”
- peteradio 6y agoI wonder what humans will learn about themselves as we create synthetic competitors.
- moconnor 6y agoThat enabling returns on capital entirely without labour functionally breaks capitalist democracies, probably.
- jacobush 6y agoA new twist on the Terminator series. Skynet starts a war because the humans tries something which would influence the stock market. Come to think of it, not much different to current reality... we already do that to ourselves.
- robertlagrant 6y agoProbably not - absolute standards of living will get even higher, but there will always be things only some people have, and more people want. (This applies with or without synthetic life doing stuff for us.)
- Agebor 6y agoThat perhaps it was already done before...
- JHonaker 6y agoI’m tired of Hinton over promising. It’s not going to do everything and he’s promoting an irresponsible position. Has there really been a huge breakthrough since the initial wave of CNNs? I guess transformers/attention, but I don’t consider GPT-3 to solve any problem at all.
- brutus1213 6y agoI'm generally skeptical on research and work in the field. However, the progress I've seen on GANs/deep fakes, natural language understanding, multi-modal, video understanding/activity recognition .. it is unbelievable how fast the field is moving. This may not result in new widgets next CES but may be game changing to society in a decade. In academic research, GPT-3 and in general language models are applicable to a host of downstream tasks across different topics. Edit .. reworded since it seemed insulting (unintentional).
- Filligree 6y agoHaving spent the last two months writing a novel, with GPT-3's help, I take issue with the claim that it's useless. It's anything but useless. Sure, I do 80-85% of the work, but that remaining 15% is the portion that would otherwise have me stuck for days. In practice, it's a 10-50x improvement in writing speed.
- a_wild_dandan 6y agoWhat's your process for using GPT-3 to aid your writing?
- Filligree 6y agoSee sibling comment.
- the_jeremy 6y agoCan you elaborate on how you use it? Do you have it generate next paragraphs when you're stuck, or something else?
- Filligree 6y agoI've elaborated significantly here: https://forums.sufficientvelocity.com/threads/generic-pawn-templates-a-gpt-3-quest.70063/ https://forums.sufficientvelocity.com/threads/generic-pawn-t... If that doesn't answer your questions, feel free to ask some in the thread. I don't check comment replies here very often. (Note: The first ten-or-so updates were purely educational, and it's only the last few where I've been trying for high quality text. If you just want to see how well that can work, then start at the end.)
- schonbild 6y agoSure, if "everything" includes making humanly mistakes as well!
- giardini 6y agoHinton wants a Nobel prize awarded while his name is still in the ring!8-)) But that will likely happen a generation or two after Hinton is gone.
- cs702 6y agoHinton qualified that statement by adding, "but I do think there's going to have to be quite a few conceptual breakthroughs." He explicitly conditioned his belief on future conceptual breakthroughs that have not been made yet. Also, his statement is not a prediction; it starts with "I do believe." Here's the full quote, copied verbatim from the article: > I do believe deep learning is going to be able to do everything, but I do think there’s going to have to be quite a few conceptual breakthroughs. For example, in 2017 Ashish Vaswani et al. introduced transformers, which derive really good vectors representing word meanings. It was a conceptual breakthrough. It’s now used in almost all the very best natural-language processing. We’re going to need a bunch more breakthroughs like that. Please don't criticize him or the article without first reading it in full.
- bo1024 6y agoYeah, it's a bit tricky ... like saying "Turing Machines are going to be able to do everything, but we'll need quite a few conceptual breakthroughs." Hard to say what percent of progress we've made since 1936 toward artificial general intelligence (whatever that means).
- cs702 6y agoIn principle, yes, of course, because we use Turing machines for deep learning :-) but I think Hinton's point is much less broad than you imply: He believes "AGI" ultimately will be achieved with deep learning approaches, as opposed to other approaches, e.g., symbolic AI.
- bo1024 6y agoYes, but "quite a few conceptual breakthroughs" leaves a lot of room open for if it would be at all similar to deep learning today.
- cs702 6y agoAgree :-) Hinton's comment was meant as, and is, an educated guess, expressed as a belief -- much like expressing belief that P≠NP, say, or that string theory will prove to be a dead end in Physics. FWIW, I happen to agree with Hinton, but only time will tell if we're right!
- 6gvONxR4sf7o 6y ago> What do you believe to be your most contrarian view on the future of AI? > Well, my problem is I have these contrarian views and then five years later, they’re mainstream. Has every single one of his contrarian views panned out? What an arrogant quote. This just diminished him in my eyes.
- etiam 6y agoRelax. It's an expected wisecrack at a question that is frankly not of very high quality.
- phenkdo 6y agoI am very skeptical of the current approaches (supervised learning) making the quantum leaps being promised. This needs a paradigm shift of weakly-supervised learning and fine-tuning for specific tasks (a la human learning).
- ebalit 6y agoIt is indeed where a lot of the current research is heading to if you count self-supervision as part of weak-supervision. Self-supervised learning brought massive improvement in NLP and is bringing state of the art result in vision after some time where purely supervised learning showed better result.
- bonoboTP 6y agoWeakly-supervised and self-supervised methods are extremely mainstream today. There is a flood of papers on these topics. Conferences are full of them. This comment was perhaps insightful 5-6 years ago, today it's absolute orthodoxy. The thing is though, supervised just works way better. It's not that people don't want to do weakly-supervised, but that it's very hard!
- phenkdo 6y agoMy point exactly. Weakly/self-supervised learning is "very hard" because it's still an active area of research and breakthroughs are needed. I feel those a pre-requisite before we can credibly approach the promise of "doing everything"
- ku-man 6y agoSince the fiasco of the nanotechnology, I became a very skeptical person when it comes to "disrupting" new technologies.
- mellosouls 6y agoNow it’s hard to find anyone who disagrees, he says. Either that's an accurate claim or representative of a cliqueyness and echo chamber approach to AGI in some quarters. Either way it's depressing.
- deleted 6y ago[deleted]
- mcguire 6y agoDoes this mean we are at the top of another AI hype cycle? Anyone have a pool on when the next AI winter starts?
- WanderPanda 6y agoWinter will be gone forever, AI climate change it is :p
- gwern 6y agohttps://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=%22AI%20winter%22&sort=byPopularity&type=all https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... According to HN, AI winter has already started in 2017, 2018, 2019, 2020...
- mhh__ 6y agoI basically buy that DL will be effective, but I think the real innovation will have to be power efficiency e.g. how many GPUs and GBs of ram does GPT-x need?
- lootsauce 6y agoIf a cyborg were to "do everything" on deep learning would it have a meaningful model of reality or would it simply be behaving as if it did?
- ilaksh 6y ago> In particular, some recent work at Google has shown that you can do fine motor control and combine that with language, so that you can open a drawer and take out a block, and the system can tell you in natural language what it’s doing. Does anyone know which paper that was?