11 ms·

Geoffrey Hinton has joined Google

158 points by roquin 14y ago

marcosploither 14y ago
I have read George Clinton, it would have been a better headline.
krmboya 14y ago
Slightly off-topic, but I wonder what Dijkstra would feel about the state of CS research if he came to the present time. Hot topics now seem to be statistical ML stuff, and not the deterministic mathematical proofs that he was an advocate of.
deleted 14y ago
[deleted]
jahewson 14y ago
So is Google now more appealing to academics than academia?
- klipt 14y ago
  For research requiring ridiculous amounts of processing power, large corporations have the advantage over most universities.
  - waterlesscloud 14y ago
    Indeed. It's not hard to see why a neural network researcher in particular would get excited about having access to tremendous amounts of data and computer clusters.
- mayank 14y ago
  It certainly appears to be. Here's a former Harvard prof who wrote a pretty interesting blog post: http://matt-welsh.blogspot.com/2010/11/why-im-leaving-harvard.html http://matt-welsh.blogspot.com/2010/11/why-im-leaving-harvar...
- naner 14y ago
  I imagine Google pays a better. By a wide margin.
- ninjin 14y ago
  In a way, yes. Research in industry has the advantage that you don't have to "waste" your time applying for grants in order to fund your own research group. A lot of people seem to live in an illusion that professors actually spend most of their time doing research, they are the managers of research groups more than anything else and growing those research groups is hard work and takes up close to all of your time (then add teaching to this). You are unlikely to have more than a few permanent researchers under you and these researchers will most likely be trying to leave within a few year and get a professorship of their own. Add to this that it is an enormous pain to solve engineering problems at a university, you usually have to hand this to students, limiting what you can produce and maintain. The best I have ever seen for a group was a single full-time software engineer and this was at arguably the most prestigious university in the world. And don't get me started on the fact that there is a poor incentive to produce good software in academia, even though said software can be essential to make research possible. I don't know Hinton in person, but I can imagine that at his age both the possibility of something new and the promise of a strong engineering and financial backing for a large group is enormously tempting (also, do they force professors into retirement or start denying them grants in Canada?). Oh, and to those pointing out the potential salary, if my knowledge of the financial situation of professors that have been on tenure is generalisable I would be surprised if personal finances would mean much at this point.
  - Lewisham 14y ago
    This is a very good summation of why I'm heading to Google once I finish grad school this academic year.
  - drpgq 14y ago
    I won't put it up here, but those that are curious can look up Hinton's salary at the University of Toronto by looking at the Ontario government sunshine list website.
    - abrichr 14y ago
      Link for the lazy: http://www.fin.gov.on.ca/en/publications/salarydisclosure/2012/univer12g.html http://www.fin.gov.on.ca/en/publications/salarydisclosure/20...
kolektiv 14y ago
Sigh. I don't know whether I'm alone in being saddened that many of the excellent thinkers of our time, who once might have spent their lives probing the the very limits of our comprehension in their field, now end up comfortably turning their minds to the services of (in the long term) rather short sighted commercial ends. I can imagine that this sentiment is probably against the prevailing HN mood, but I've been thinking a lot lately about how certain kinds of thought and investigation are only enabled and supported by certain structures. The point of a university used to be being the highest pinnacle of thought. A place where people could have everything not related to intellectual pursuit taken care of so they could devote themselves to the pursuit of greater knowledge. Now that seems to be replaced by a corporate campus. It's a cliche, of course, but how many breakthroughs of meaning are we missing out on because the brightest and best of our generation are now no longer seeking after truth, but instead seeking after a way to make more people click on an ad? Ah well, it's late, and as ever I'm post-sober. Best of luck to him and his team - I've enjoyed his lectures and papers. I hope they may continue, but I sadly suspect not.
- moultano 14y ago
  I don't think you can accuse Google of short sighted commercial ends here. The team he is working with has already done really groundbreaking stuff that falls right in line with his work. https://plus.google.com/u/0/+ResearchatGoogle/posts/EMyhnBetd2F https://plus.google.com/u/0/+ResearchatGoogle/posts/EMyhnBet...
  - deleted 14y ago
    [deleted]
    - rednukleus 14y ago
      1. They don't sell info, they just target ads 2. Google are an AI company, who currently make most of their profit from ads. They are well aware that there are big opportunities for them to make profits outside targeted ads.
      - 14y ago
roquin 14y ago
A related news is "Google acquires Canadian neural networks startup DNNresearch, aims to improve image and voice search" http://thenextweb.com/google/2013/03/12/google-acquires-canadian-neural-networks-startup-dnnresearch-aims-to-improve-image-and-voice-search/ http://thenextweb.com/google/2013/03/12/google-acquires-cana...
- aflinik 14y ago
  not only related, that's exactly the same thing
oscargrouch 14y ago
Google is sucking all the talent and the brain of the world.. i wish at least that the efforts would get published in new papers and open source software.. Or the world will enter into a dark age of knowledge, ruled by huge corporations, and everybody else having to share the crumbs that fall of the table.. I hope that not all big minds fall of for companies.. or at least that companies start to commit with knowledge shareing to the rest of the world we should stop black box knowledge companies or we are doomed in the long term.. (like the coke secret recipe case)
- mayank 14y ago
  > i wish at least that the efforts would get published in new papers You mean like this very large list? http://research.google.com/pubs/papers.html http://research.google.com/pubs/papers.html > and open source software Or this also very large list? https://code.google.com/hosting/search?q=label:google https://code.google.com/hosting/search?q=label:google
  - oscargrouch 14y ago
    Dont get me wrong.. google is much better than its "predecessors" in that field.. eg: microsoft, apple, ibm.. but in the end is only one company, with its capital open in the stock market, with boards and everything else.. while google now is half good/half evil.. who can garantee that in the future when larry and sergey goes away.. that the company wont be ruled by a Larry ellison clone? ps: while google have a pretty much decent portfolio of open source and papers floating around.. its more about "marginal software" for google... the core of its knowledge about the business still a secret.. what would be of the world now and the cloud economy without Doug Cutting, that do it all by himself? In one sentence: is only ONE company rulled by the capital market and profits and thats pretty scary.. thats all :s
    - kvb 14y ago
      I think this is unfair to at least Microsoft Research, which as far as I know has outpublished Google by a wide margin. (Full disclosure - I'm a contractor working on F#-related stuff at MSR)
cs702 14y ago
Something really interesting must be happening with AI at Google, because in the past few months both Ray Kurzweil (the best-known proponent of the singularity) and Geoff Hinton (the crazy-talented individual who invented deep-belief networks using interconnected "restricted Boltzman machines"[1]) have joined the company. -- [1] For an overview of deep belief networks, see these videos: http://www.youtube.com/watch?v=AyzOUbkUf3M http://www.youtube.com/watch?v=AyzOUbkUf3M , http://www.youtube.com/watch?v=VdIURAu1-aU http://www.youtube.com/watch?v=VdIURAu1-aU , and http://www.youtube.com/watch?v=DleXA5ADG78 http://www.youtube.com/watch?v=DleXA5ADG78
- notatoad 14y ago
  I think it's pretty clear what's so interesting: google has the some of the largest datasets ever assembled, on a hugely diverse range of subjects. If you want to play with that data, you've got to work for google. Probably the only other place you get to play with that much data is at the NSA, and their goals are a bit less fun than google's.
  - simonster 14y ago
    There's also the fact that Google likely has more computing power than any other organization worldwide, which is what seems to be the attraction for Hinton, at least according to the last sentence of his post.
tristanz 14y ago
This is exciting for Hinton I'm sure, but also somewhat worrying for broader community. Hinton has always been very open with his research: http://www.nytimes.com/2012/11/24/science/scientists-see-advances-in-deep-learning-a-part-of-artificial-intelligence.html?pagewanted=2 http://www.nytimes.com/2012/11/24/science/scientists-see-adv... One of the most striking aspects of the research led by Dr. Hinton is that it has taken place largely without the patent restrictions and bitter infighting over intellectual property that characterize high-technology fields. “We decided early on not to make money out of this, but just to sort of spread it to infect everybody,” he said. “These companies are terribly pleased with this.”
- simonster 14y ago
  I doubt Google will stop Hinton from publishing, and I think both will still benefit greatly from that arrangement. Larry and Sergey published "The Anatomy of a Large-Scale Hypertextual Web Search Engine" before they were even incorporated, and it doesn't seem to have hurt them much.
- ZeroGravitas 14y ago
  Sounds like a good fit for Google, who are leading patent disarmament among tech giants: http://www.google.com/patents/licensing/index.html http://www.google.com/patents/licensing/index.html Though Google seem to think you can still make money despite/because of sharing, through other means than patent licencing.
dchichkov 14y ago
Poor grad students :-/ Anyway, if you are unfamiliar with his work, here's Hinton't Deep Learning Saga ;) for your enjoyment: www.youtube.com/watch?v=mlXzufEk-2E
- otoburb 14y ago
  Don't worry about them. He's still remaining attached to University of Toronto part-time specifically because of his remaining graduate students[1]. [1] https://plus.google.com/102889418997957626067/posts https://plus.google.com/102889418997957626067/posts
niggler 14y ago
In contrast to many who think that people are being shepherded to the search business, I wonder if Google is trying build a new "Bell Labs"-type research division, in which case this is good news.
- T-A 14y ago
  You mean something like http://en.wikipedia.org/wiki/Google_X_Lab http://en.wikipedia.org/wiki/Google_X_Lab ? :)
kmfrk 14y ago
Google is slowly becoming the equivalent of a Silicon Valley think tank. :P Not that it's a bad thing. Lots of smart people did interesting things at IBM, just as people at Microsoft Research are doing some great things - albeit usually without anyone seeing the work in progress.
srean 14y ago
HN'ers may have a good laugh at these taken from Yan LeCun's page. LeCun made it a tradition to have Hinton jokes in the lines of Chuck Norris ones (or more appropriately Doug McCllroy ones). A few will recall that neural networks all but died from the US after Minsky's damning book. Hinton gave backpropagation which is one of the foundational pillars of feed-forward neural net algorithms. With his new thrust on whats called "deep belief networks" he is challenging his own early seminal contribution in the field. Not often do you see researchers throw away such huge swathes of their own work and start again to solve the same problem. Unless you are Niklaus Wirth of course. Some background is necessary to get the inside jokes, but I have tried to minimize the requirement. Geoff Hinton doesn't need to make hidden units. They hide by themselves when he approaches. Geoff Hinton discovered how the brain really works. Once a year for the last 25 years. Markov random fields think Geoff Hinton is intractable. Geoff Hinton can make you regret without bounds. Geoff Hinton doesn't need support vectors. He can support high-dimensional hyperplanes with his pinky. All kernels that ever dared approaching Geoff Hinton woke up convolved. The only kernel Geoff Hinton has ever used is a kernel of truth. After an encounter with Geoff Hinton, support vectors become unhinged Geoff Hinton's generalizations are boundless. Geoff Hinton goes directly to third Bayes. Links: http://yann.lecun.com/ex/fun/index.html http://yann.lecun.com/ex/fun/index.html http://en.wikipedia.org/wiki/Backpropagation http://en.wikipedia.org/wiki/Backpropagation http://en.wikipedia.org/wiki/Perceptrons_%28book%29 http://en.wikipedia.org/wiki/Perceptrons_%28book%29 http://en.wikipedia.org/wiki/Niklaus_Wirth http://en.wikipedia.org/wiki/Niklaus_Wirth Re: Downvotes. I seem to have touched a nerve. Yes I agree humor is frowned upon here, and I largely agree with that, but when its not humor for humors own sake, I sometimes make an exception. Not everyone would know who Hinton is, but may gauge that he is someone important from these fun anecdotes. But of course you are free to like it or dislike it, no hard feelings either which way.
- DaniFong 14y ago
  I think these are hilarious; thanks for posting
- IanCal 14y ago
  I like the quotes, however: > With his new thrust on whats called "deep belief networks" he is challenging his own early seminal contribution in the field I don't know if I agree with that, he still uses backprop. Backprop has always been known to have problems when you scale to millions of connections, and his work on RBMs/DBNs is really quite old. What was novel more recently was showing that the contrastive divergence step need only be performed once, rather than 100 times, while the performance remained similar. The networks are generally still 'fine tuned' with backprop. Still, the focus on generative networks (not sure if that's the right term still, been a while) and single layer training is fairly recent even if the concepts are quite old.
bronxbomber92 14y ago
I read this as "Hilary Clinton" has joined Google, and did a big double-take! Aha..

158 points by roquin 14y ago

I have read George Clinton, it would have been a better headline.

Slightly off-topic, but I wonder what Dijkstra would feel about the state of CS research if he came to the present time. Hot topics now seem to be statistical ML stuff, and not the deterministic mathematical proofs that he was an advocate of.

deleted 14y ago

[deleted]

jahewson 14y ago

So is Google now more appealing to academics than academia?

klipt 14y ago
For research requiring ridiculous amounts of processing power, large corporations have the advantage over most universities.
- waterlesscloud 14y ago
  Indeed. It's not hard to see why a neural network researcher in particular would get excited about having access to tremendous amounts of data and computer clusters.
mayank 14y ago
It certainly appears to be. Here's a former Harvard prof who wrote a pretty interesting blog post: http://matt-welsh.blogspot.com/2010/11/why-im-leaving-harvard.html http://matt-welsh.blogspot.com/2010/11/why-im-leaving-harvar...
naner 14y ago
I imagine Google pays a better. By a wide margin.
ninjin 14y ago
In a way, yes. Research in industry has the advantage that you don't have to "waste" your time applying for grants in order to fund your own research group. A lot of people seem to live in an illusion that professors actually spend most of their time doing research, they are the managers of research groups more than anything else and growing those research groups is hard work and takes up close to all of your time (then add teaching to this). You are unlikely to have more than a few permanent researchers under you and these researchers will most likely be trying to leave within a few year and get a professorship of their own. Add to this that it is an enormous pain to solve engineering problems at a university, you usually have to hand this to students, limiting what you can produce and maintain. The best I have ever seen for a group was a single full-time software engineer and this was at arguably the most prestigious university in the world. And don't get me started on the fact that there is a poor incentive to produce good software in academia, even though said software can be essential to make research possible. I don't know Hinton in person, but I can imagine that at his age both the possibility of something new and the promise of a strong engineering and financial backing for a large group is enormously tempting (also, do they force professors into retirement or start denying them grants in Canada?). Oh, and to those pointing out the potential salary, if my knowledge of the financial situation of professors that have been on tenure is generalisable I would be surprised if personal finances would mean much at this point.
- Lewisham 14y ago
  This is a very good summation of why I'm heading to Google once I finish grad school this academic year.
- drpgq 14y ago
  I won't put it up here, but those that are curious can look up Hinton's salary at the University of Toronto by looking at the Ontario government sunshine list website.
  - abrichr 14y ago
    Link for the lazy: http://www.fin.gov.on.ca/en/publications/salarydisclosure/2012/univer12g.html http://www.fin.gov.on.ca/en/publications/salarydisclosure/20...

kolektiv 14y ago

Sigh. I don't know whether I'm alone in being saddened that many of the excellent thinkers of our time, who once might have spent their lives probing the the very limits of our comprehension in their field, now end up comfortably turning their minds to the services of (in the long term) rather short sighted commercial ends. I can imagine that this sentiment is probably against the prevailing HN mood, but I've been thinking a lot lately about how certain kinds of thought and investigation are only enabled and supported by certain structures. The point of a university used to be being the highest pinnacle of thought. A place where people could have everything not related to intellectual pursuit taken care of so they could devote themselves to the pursuit of greater knowledge. Now that seems to be replaced by a corporate campus. It's a cliche, of course, but how many breakthroughs of meaning are we missing out on because the brightest and best of our generation are now no longer seeking after truth, but instead seeking after a way to make more people click on an ad? Ah well, it's late, and as ever I'm post-sober. Best of luck to him and his team - I've enjoyed his lectures and papers. I hope they may continue, but I sadly suspect not.

moultano 14y ago
I don't think you can accuse Google of short sighted commercial ends here. The team he is working with has already done really groundbreaking stuff that falls right in line with his work. https://plus.google.com/u/0/+ResearchatGoogle/posts/EMyhnBetd2F https://plus.google.com/u/0/+ResearchatGoogle/posts/EMyhnBet...
- deleted 14y ago
  [deleted]
  - rednukleus 14y ago
    1. They don't sell info, they just target ads 2. Google are an AI company, who currently make most of their profit from ads. They are well aware that there are big opportunities for them to make profits outside targeted ads.
    - 14y ago

roquin 14y ago

A related news is "Google acquires Canadian neural networks startup DNNresearch, aims to improve image and voice search" http://thenextweb.com/google/2013/03/12/google-acquires-canadian-neural-networks-startup-dnnresearch-aims-to-improve-image-and-voice-search/ http://thenextweb.com/google/2013/03/12/google-acquires-cana...

aflinik 14y ago
not only related, that's exactly the same thing

oscargrouch 14y ago

Google is sucking all the talent and the brain of the world.. i wish at least that the efforts would get published in new papers and open source software.. Or the world will enter into a dark age of knowledge, ruled by huge corporations, and everybody else having to share the crumbs that fall of the table.. I hope that not all big minds fall of for companies.. or at least that companies start to commit with knowledge shareing to the rest of the world we should stop black box knowledge companies or we are doomed in the long term.. (like the coke secret recipe case)

mayank 14y ago
> i wish at least that the efforts would get published in new papers You mean like this very large list? http://research.google.com/pubs/papers.html http://research.google.com/pubs/papers.html > and open source software Or this also very large list? https://code.google.com/hosting/search?q=label:google https://code.google.com/hosting/search?q=label:google
- oscargrouch 14y ago
  Dont get me wrong.. google is much better than its "predecessors" in that field.. eg: microsoft, apple, ibm.. but in the end is only one company, with its capital open in the stock market, with boards and everything else.. while google now is half good/half evil.. who can garantee that in the future when larry and sergey goes away.. that the company wont be ruled by a Larry ellison clone? ps: while google have a pretty much decent portfolio of open source and papers floating around.. its more about "marginal software" for google... the core of its knowledge about the business still a secret.. what would be of the world now and the cloud economy without Doug Cutting, that do it all by himself? In one sentence: is only ONE company rulled by the capital market and profits and thats pretty scary.. thats all :s
  - kvb 14y ago
    I think this is unfair to at least Microsoft Research, which as far as I know has outpublished Google by a wide margin. (Full disclosure - I'm a contractor working on F#-related stuff at MSR)

cs702 14y ago

Something really interesting must be happening with AI at Google, because in the past few months both Ray Kurzweil (the best-known proponent of the singularity) and Geoff Hinton (the crazy-talented individual who invented deep-belief networks using interconnected "restricted Boltzman machines"[1]) have joined the company. -- [1] For an overview of deep belief networks, see these videos: http://www.youtube.com/watch?v=AyzOUbkUf3M http://www.youtube.com/watch?v=AyzOUbkUf3M , http://www.youtube.com/watch?v=VdIURAu1-aU http://www.youtube.com/watch?v=VdIURAu1-aU , and http://www.youtube.com/watch?v=DleXA5ADG78 http://www.youtube.com/watch?v=DleXA5ADG78

notatoad 14y ago
I think it's pretty clear what's so interesting: google has the some of the largest datasets ever assembled, on a hugely diverse range of subjects. If you want to play with that data, you've got to work for google. Probably the only other place you get to play with that much data is at the NSA, and their goals are a bit less fun than google's.
- simonster 14y ago
  There's also the fact that Google likely has more computing power than any other organization worldwide, which is what seems to be the attraction for Hinton, at least according to the last sentence of his post.

tristanz 14y ago

This is exciting for Hinton I'm sure, but also somewhat worrying for broader community. Hinton has always been very open with his research: http://www.nytimes.com/2012/11/24/science/scientists-see-advances-in-deep-learning-a-part-of-artificial-intelligence.html?pagewanted=2 http://www.nytimes.com/2012/11/24/science/scientists-see-adv... One of the most striking aspects of the research led by Dr. Hinton is that it has taken place largely without the patent restrictions and bitter infighting over intellectual property that characterize high-technology fields. “We decided early on not to make money out of this, but just to sort of spread it to infect everybody,” he said. “These companies are terribly pleased with this.”

simonster 14y ago
I doubt Google will stop Hinton from publishing, and I think both will still benefit greatly from that arrangement. Larry and Sergey published "The Anatomy of a Large-Scale Hypertextual Web Search Engine" before they were even incorporated, and it doesn't seem to have hurt them much.
ZeroGravitas 14y ago
Sounds like a good fit for Google, who are leading patent disarmament among tech giants: http://www.google.com/patents/licensing/index.html http://www.google.com/patents/licensing/index.html Though Google seem to think you can still make money despite/because of sharing, through other means than patent licencing.

dchichkov 14y ago

Poor grad students :-/ Anyway, if you are unfamiliar with his work, here's Hinton't Deep Learning Saga ;) for your enjoyment: www.youtube.com/watch?v=mlXzufEk-2E

otoburb 14y ago
Don't worry about them. He's still remaining attached to University of Toronto part-time specifically because of his remaining graduate students[1]. [1] https://plus.google.com/102889418997957626067/posts https://plus.google.com/102889418997957626067/posts

niggler 14y ago

In contrast to many who think that people are being shepherded to the search business, I wonder if Google is trying build a new "Bell Labs"-type research division, in which case this is good news.

T-A 14y ago
You mean something like http://en.wikipedia.org/wiki/Google_X_Lab http://en.wikipedia.org/wiki/Google_X_Lab ? :)

kmfrk 14y ago

Google is slowly becoming the equivalent of a Silicon Valley think tank. :P Not that it's a bad thing. Lots of smart people did interesting things at IBM, just as people at Microsoft Research are doing some great things - albeit usually without anyone seeing the work in progress.

srean 14y ago

HN'ers may have a good laugh at these taken from Yan LeCun's page. LeCun made it a tradition to have Hinton jokes in the lines of Chuck Norris ones (or more appropriately Doug McCllroy ones). A few will recall that neural networks all but died from the US after Minsky's damning book. Hinton gave backpropagation which is one of the foundational pillars of feed-forward neural net algorithms. With his new thrust on whats called "deep belief networks" he is challenging his own early seminal contribution in the field. Not often do you see researchers throw away such huge swathes of their own work and start again to solve the same problem. Unless you are Niklaus Wirth of course. Some background is necessary to get the inside jokes, but I have tried to minimize the requirement. Geoff Hinton doesn't need to make hidden units. They hide by themselves when he approaches. Geoff Hinton discovered how the brain really works. Once a year for the last 25 years. Markov random fields think Geoff Hinton is intractable. Geoff Hinton can make you regret without bounds. Geoff Hinton doesn't need support vectors. He can support high-dimensional hyperplanes with his pinky. All kernels that ever dared approaching Geoff Hinton woke up convolved. The only kernel Geoff Hinton has ever used is a kernel of truth. After an encounter with Geoff Hinton, support vectors become unhinged Geoff Hinton's generalizations are boundless. Geoff Hinton goes directly to third Bayes. Links: http://yann.lecun.com/ex/fun/index.html http://yann.lecun.com/ex/fun/index.html http://en.wikipedia.org/wiki/Backpropagation http://en.wikipedia.org/wiki/Backpropagation http://en.wikipedia.org/wiki/Perceptrons_%28book%29 http://en.wikipedia.org/wiki/Perceptrons_%28book%29 http://en.wikipedia.org/wiki/Niklaus_Wirth http://en.wikipedia.org/wiki/Niklaus_Wirth Re: Downvotes. I seem to have touched a nerve. Yes I agree humor is frowned upon here, and I largely agree with that, but when its not humor for humors own sake, I sometimes make an exception. Not everyone would know who Hinton is, but may gauge that he is someone important from these fun anecdotes. But of course you are free to like it or dislike it, no hard feelings either which way.

DaniFong 14y ago
I think these are hilarious; thanks for posting
IanCal 14y ago
I like the quotes, however: > With his new thrust on whats called "deep belief networks" he is challenging his own early seminal contribution in the field I don't know if I agree with that, he still uses backprop. Backprop has always been known to have problems when you scale to millions of connections, and his work on RBMs/DBNs is really quite old. What was novel more recently was showing that the contrastive divergence step need only be performed once, rather than 100 times, while the performance remained similar. The networks are generally still 'fine tuned' with backprop. Still, the focus on generative networks (not sure if that's the right term still, been a while) and single layer training is fairly recent even if the concepts are quite old.

bronxbomber92 14y ago

I read this as "Hilary Clinton" has joined Google, and did a big double-take! Aha..