7 ms·

Neural Networks for Machine Learning

121 points by ameasure 14y ago

ameasure 14y ago
Hinton is a huge figure in the neural network literature and an important researcher in deep learning. After going through the first week of lectures, I can say he's also an excellent teacher. The syllabus, draft though it is, indicates the second half of the class will focus on deep learning, a field of machine learning that has demonstrated huge potential.
tocomment 14y ago
Did it already start? Is it too late to start? Also I took a nn class in college so do you think I would get much more out of this?
- ameasure 14y ago
  It just started on Monday, there's plenty of time to join in. There have been some huge developments in neural networks in the last few years, particularly with respect to deep learning. If you missed out on that you might want to try this class. Hinton has been involved in many of these advances. The second half of the course appears to focus on deep learning topics so you might want to start there if you already know the basics.
  - misiti3780 14y ago
    you cant start mid-way though ... right ?
    - ameasure 14y ago
      You'll have to wait until those lectures are made available, but you don't have to complete the previous work to see the lectures.
jimbokun 14y ago
Just browsing through the Coursera Computer Science listings, it looks like they are rapidly approaching the point where you could put together a CS curriculum superior to what you could get at any single school. The people they have teaching a lot of these topics are some of the best in the world in their field. The Micahel Collins NLP course looks really thorough and up to date, for example I took a similar course a few years ago, and I remember reading papers written by him. As has been said by many already, of course, the remaining nuts to crack are high quality interaction with other students, professors, and TAs; and accreditation. But the dis-intermediation of large universities may be nearer than we think.
- misiti3780 14y ago
  the only real problem with coursera is everyone is posting their solutions to github, so its gonna be impossible for them to prevent cheating. i agree with you though, that the flexibility it is offering is amazing
  - ChuckMcM 14y ago
    There is an interesting practical question here. Why cheat? If you are taking a class voluntarily over the Internet, what benefit would be gained by cheating? I presume that a large fraction of people who are doing volunteer coursework are doing it to learn, not to keep a GPA up for some other reason (sports eligibility, scholarship requirements, parental expectations, Etc.) so looking at other solutions on Github might actually enhance the experience for you if you look at other solutions. If you find a way to do it better than the other solutions that could be a goal in itself. This is one of those things I find most intriguing about 'free' classes on the Internet, the value equation is shifted around.
    - salman89 14y ago
      It depends on the purpose of your education. In an ideal world, it would be just to learn, but I think employers at some level look at grades/school as a qualification process.
misiti3780 14y ago
people are already complaining that you can only take the quizes once ... he had to send out an email today to everyone saying: "Many of you are unhappy with only being allowed to attempt a quiz once. Starting in week two, we have therefore decided to make up twice as many questions and to allow you to do each quiz twice if you want to. The second time you try it the questions will all be different. Your score will be the maximum of your two scores. For week one, the quizzes will remain as they are now. Many of you would like the names of the videos to be more informative. We will change the names to indicate the content and the duration. Some of you thought that some of the quiz questions were too vague. We will try to make future questions less vague. Some of you are unhappy that we do not have the resources to support Python for the programming assignments. We sympathize with you and would do it if we could. You are still welcome to use Python (or any other language) if you can port the octave starter code to your preferred language. We have no objection to people sharing the ported versions of the starter code (but only the starter code!). However, if you get starter code in another language from someone else, you are responsible for making sure it does not contain bugs." I thought that was pretty funny!
- notimetorelax 14y ago
  Yeap we got spoiled with earlier classes: Algorithms by Tim Roughgarden, Machine Learning by Andrew Ng, and many more. We probably need to follow a class on gratitude. Oh well, to be fair I would donate quite a lot for each course that I enjoyed.
  - jberryman 14y ago
    Actually all the entitled bitching and moaning on the ML class forum was by far the biggest turnoff of the whole experience for me. I was much happier after ignoring it and my "classmates" entirely.
lathamcity 14y ago
I'm in the middle of the machine learning coursera course, and registered for this one as well due to interest in the material. My one complaint is that the programming assignments weren't interesting at all. The results were interesting, but the setups were mostly given to us, and we just had to code an algorithm that was in our notes. For someone who understands the basics of linear algebra and programming, it was just a syntax challenge, and that got irritating after a bit so I stopped doing them. I won't get the certificate for completing the course, but I have a few extra hours of free time each week to add this second course, so I'm happy. I doubt that the actual homework that Stanford students taking this course get is so easy and repetitive, though, and I'm positive they wouldn't complain about not getting to retake quizzes after getting poor grades. Not to knock the course. I've learned a lot and the professor (Andrew Ng) does a good job.
- dvdhsu 14y ago
  > The results were interesting, but the setups were mostly given to us, and we just had to code an algorithm that was in our notes. Right; I agree. I'm not sure how they would go about making it more challenging though. They can't expect us to go out and collect data ourselves, after all. I suppose they could give us the data, then expect us to code the setup and algorithms up ourselves, but that, too, would become repetitive after a few assignments. > Not to knock the course. I've learned a lot and the professor (Andrew Ng) does a good job. Agreed once again. I knew nothing about machine learning before starting; now I know about neural networks, SVMs, and PCM. It's really cool how much I've learned already, for free, too! I've also signed up for this course, but the quizzes really aren't up to par. As an example: the first quiz question was about training a neural network with too much data, and about whether or not said network would be able to generalize to new test cases. Overfitting neural networks wasn't even mentioned in the lectures; I had to rely on material from Andrew's class to answer the question correctly. This chasm between the lectures and the quizzes is likely because Geoffrey is the one creating the video lectures, but he's not the one creating the quiz questions; he is having TAs do it [1]. Nevertheless, it looks like they're responding to feedback, so hopefully it'll get better with time. 1. https://class.coursera.org/neuralnets-2012-001/wiki/view?page=aboutus https://class.coursera.org/neuralnets-2012-001/wiki/view?pag...
rubashov 14y ago
I tried to do a couple coursera courses and found the video lectures highly inefficient; very needlessly time consuming, even watching them sped up. All I really want is a glorified text book with quiz grading and a final.
- vitno 14y ago
  this. No offense to these professors, but what are they presenting in their video lectures that I can't garner from their writing?
  - saraid216 14y ago
    Humanity. Which, believe it or not, makes a huge difference in learning subjects.
- notimetorelax 14y ago
  It depends on how much you value your time. Lectures usually are shorter than 2 hours per week. (There are some that have longer videos, but I think more than 2 hours is suboptimal.) I know that before courses I was wasting this time on hacker news or reddit, so I don't value my time that much. On the other hand I do now, and that's because I need to watch the lectures and do the home work. And really these lectures perform the same role in the learning process as the real lectures. You could graduate from university only with text books, but you might not get some insight that lecturers have. My 0.02 chf.
- minikomi 14y ago
  It sounds terribly privileged to say so, but I'm afraid I have to agree. Also, quite often the quizzes are directly based on the videos ("What did line A represent in ~ graph?"), while I find I self learn better through reading.
emcl 14y ago
The only course that is not significantly diluted is Koller's PGM. All others have been dumbed down to a degree where they provide no challenge to the courseree at all.
- notimetorelax 14y ago
  It is not such a huge problem when you take several courses at once. Sadly they run them only twice a year, each time I try to follow as many as possible. I cannot follow PGM because it requires too much of my time, I'd have to abandon 2 or 3 other courses. YMMV.
  - tomku 14y ago
    I'm looking at the same problem at the moment. PGM sounds really interesting, but I think that the time investment just isn't going to be workable for me unless I drop several of my other classes. My current plan is to watch the PGM videos and try to keep up with the programming assignments as long as I can, but if it comes down to a choice of one or the other, PGM will be the one to go. As far as "dumbing down", I've found that the Coursera classes that I've taken (Compilers, Automata Theory, Algorithms 1, SaaS and Machine Learning) have varied in difficulty quite widely. Compilers and Automata were both challenging and enjoyable, Algorithms 1 was about what I'd expect from a freshman/sophomore algorithms class and SaaS and Machine Learning were easy enough that they should be approachable to anyone with basic programming experience. I don't feel that the difficulty in the classes that I've taken had any particular correlation with teaching effectiveness. I found Andrew Ng's ML class to be simple, but still interesting and informative - you come out of it with enough of a basic understanding to implement simple ML techniques as well as a place to start if you wish to learn more. I think that while a theory-centric class would be a nice thing to have, he's done an amazing job of making a class that can appeal to a wide range of potential students and introduce them to a field that's usually very difficult to approach.
azakai 14y ago
> Neural Networks are gradually taking over from simpler Machine Learning methods And haven't SVMs and such gradually taken over from Neural Networks?
- nphrk 14y ago
  Well not quite. While SVMs gained a lot of popularity for having nice properties e.g. 1) a convex problem which means a unique solution and a lot of already existing technology can be used 2) the "kernel trick" which enables us to learn in complicated spaces without computing the transformations 3) can be trained online, which makes them great for huge datasets (here the point 2) might not apply - but there exist ways - if someone's interested I can point out some papers) There is an ongoing craze about deep belief networks developed by Hinton (who is teaching this course) who came up with an algorithm that can train them (there exist local optima and such, so it's far from ideal). Some of the reasons they're popular 1) They seem to be winning algorithm for many competitions / datasets, ranging from classification in computer vision to speech recognition and if I'm not mistaken even parsing. They are for example used in the newer Androids. 2) They can be used in an unsupervised mode to _automatically_ learn different representations (features) of the data, which can be then used in subsequent stages of the classification pipeline. This makes them very interesting because while labelled data might be hard to get by, we have a lot of unlaballed datasets thanks to the Internet. As what they can do - see the work by Andrew Ng when they automatically learned a cat detector. 3) They're "similar" to biological neural networks, so one might think they have the necessary richness for many interesting AI applications.
  - notimetorelax 14y ago
    Enlightening response, could you please post links to papers that explain online training of SVM? Also, I found this paper [1] on unsupervised feature detection, if you have some additional material, I'll really appreciate if you could post it! [1] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.44.4834&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.44....
- deleted 14y ago
  [deleted]
isakovic 14y ago
I took Professor Hinton's course on Neural Networks as an undergrad. This man is the most intelligent person I have ever met. He is one of the giants.

121 points by ameasure 14y ago

Hinton is a huge figure in the neural network literature and an important researcher in deep learning. After going through the first week of lectures, I can say he's also an excellent teacher. The syllabus, draft though it is, indicates the second half of the class will focus on deep learning, a field of machine learning that has demonstrated huge potential.

tocomment 14y ago

Did it already start? Is it too late to start? Also I took a nn class in college so do you think I would get much more out of this?

ameasure 14y ago
It just started on Monday, there's plenty of time to join in. There have been some huge developments in neural networks in the last few years, particularly with respect to deep learning. If you missed out on that you might want to try this class. Hinton has been involved in many of these advances. The second half of the course appears to focus on deep learning topics so you might want to start there if you already know the basics.
- misiti3780 14y ago
  you cant start mid-way though ... right ?
  - ameasure 14y ago
    You'll have to wait until those lectures are made available, but you don't have to complete the previous work to see the lectures.

jimbokun 14y ago

Just browsing through the Coursera Computer Science listings, it looks like they are rapidly approaching the point where you could put together a CS curriculum superior to what you could get at any single school. The people they have teaching a lot of these topics are some of the best in the world in their field. The Micahel Collins NLP course looks really thorough and up to date, for example I took a similar course a few years ago, and I remember reading papers written by him. As has been said by many already, of course, the remaining nuts to crack are high quality interaction with other students, professors, and TAs; and accreditation. But the dis-intermediation of large universities may be nearer than we think.

misiti3780 14y ago
the only real problem with coursera is everyone is posting their solutions to github, so its gonna be impossible for them to prevent cheating. i agree with you though, that the flexibility it is offering is amazing
- ChuckMcM 14y ago
  There is an interesting practical question here. Why cheat? If you are taking a class voluntarily over the Internet, what benefit would be gained by cheating? I presume that a large fraction of people who are doing volunteer coursework are doing it to learn, not to keep a GPA up for some other reason (sports eligibility, scholarship requirements, parental expectations, Etc.) so looking at other solutions on Github might actually enhance the experience for you if you look at other solutions. If you find a way to do it better than the other solutions that could be a goal in itself. This is one of those things I find most intriguing about 'free' classes on the Internet, the value equation is shifted around.
  - salman89 14y ago
    It depends on the purpose of your education. In an ideal world, it would be just to learn, but I think employers at some level look at grades/school as a qualification process.

misiti3780 14y ago

people are already complaining that you can only take the quizes once ... he had to send out an email today to everyone saying: "Many of you are unhappy with only being allowed to attempt a quiz once. Starting in week two, we have therefore decided to make up twice as many questions and to allow you to do each quiz twice if you want to. The second time you try it the questions will all be different. Your score will be the maximum of your two scores. For week one, the quizzes will remain as they are now. Many of you would like the names of the videos to be more informative. We will change the names to indicate the content and the duration. Some of you thought that some of the quiz questions were too vague. We will try to make future questions less vague. Some of you are unhappy that we do not have the resources to support Python for the programming assignments. We sympathize with you and would do it if we could. You are still welcome to use Python (or any other language) if you can port the octave starter code to your preferred language. We have no objection to people sharing the ported versions of the starter code (but only the starter code!). However, if you get starter code in another language from someone else, you are responsible for making sure it does not contain bugs." I thought that was pretty funny!

notimetorelax 14y ago
Yeap we got spoiled with earlier classes: Algorithms by Tim Roughgarden, Machine Learning by Andrew Ng, and many more. We probably need to follow a class on gratitude. Oh well, to be fair I would donate quite a lot for each course that I enjoyed.
- jberryman 14y ago
  Actually all the entitled bitching and moaning on the ML class forum was by far the biggest turnoff of the whole experience for me. I was much happier after ignoring it and my "classmates" entirely.

lathamcity 14y ago

I'm in the middle of the machine learning coursera course, and registered for this one as well due to interest in the material. My one complaint is that the programming assignments weren't interesting at all. The results were interesting, but the setups were mostly given to us, and we just had to code an algorithm that was in our notes. For someone who understands the basics of linear algebra and programming, it was just a syntax challenge, and that got irritating after a bit so I stopped doing them. I won't get the certificate for completing the course, but I have a few extra hours of free time each week to add this second course, so I'm happy. I doubt that the actual homework that Stanford students taking this course get is so easy and repetitive, though, and I'm positive they wouldn't complain about not getting to retake quizzes after getting poor grades. Not to knock the course. I've learned a lot and the professor (Andrew Ng) does a good job.

dvdhsu 14y ago
> The results were interesting, but the setups were mostly given to us, and we just had to code an algorithm that was in our notes. Right; I agree. I'm not sure how they would go about making it more challenging though. They can't expect us to go out and collect data ourselves, after all. I suppose they could give us the data, then expect us to code the setup and algorithms up ourselves, but that, too, would become repetitive after a few assignments. > Not to knock the course. I've learned a lot and the professor (Andrew Ng) does a good job. Agreed once again. I knew nothing about machine learning before starting; now I know about neural networks, SVMs, and PCM. It's really cool how much I've learned already, for free, too! I've also signed up for this course, but the quizzes really aren't up to par. As an example: the first quiz question was about training a neural network with too much data, and about whether or not said network would be able to generalize to new test cases. Overfitting neural networks wasn't even mentioned in the lectures; I had to rely on material from Andrew's class to answer the question correctly. This chasm between the lectures and the quizzes is likely because Geoffrey is the one creating the video lectures, but he's not the one creating the quiz questions; he is having TAs do it [1]. Nevertheless, it looks like they're responding to feedback, so hopefully it'll get better with time. 1. https://class.coursera.org/neuralnets-2012-001/wiki/view?page=aboutus https://class.coursera.org/neuralnets-2012-001/wiki/view?pag...

rubashov 14y ago

I tried to do a couple coursera courses and found the video lectures highly inefficient; very needlessly time consuming, even watching them sped up. All I really want is a glorified text book with quiz grading and a final.

vitno 14y ago
this. No offense to these professors, but what are they presenting in their video lectures that I can't garner from their writing?
- saraid216 14y ago
  Humanity. Which, believe it or not, makes a huge difference in learning subjects.
notimetorelax 14y ago
It depends on how much you value your time. Lectures usually are shorter than 2 hours per week. (There are some that have longer videos, but I think more than 2 hours is suboptimal.) I know that before courses I was wasting this time on hacker news or reddit, so I don't value my time that much. On the other hand I do now, and that's because I need to watch the lectures and do the home work. And really these lectures perform the same role in the learning process as the real lectures. You could graduate from university only with text books, but you might not get some insight that lecturers have. My 0.02 chf.
minikomi 14y ago
It sounds terribly privileged to say so, but I'm afraid I have to agree. Also, quite often the quizzes are directly based on the videos ("What did line A represent in ~ graph?"), while I find I self learn better through reading.

emcl 14y ago

The only course that is not significantly diluted is Koller's PGM. All others have been dumbed down to a degree where they provide no challenge to the courseree at all.

notimetorelax 14y ago
It is not such a huge problem when you take several courses at once. Sadly they run them only twice a year, each time I try to follow as many as possible. I cannot follow PGM because it requires too much of my time, I'd have to abandon 2 or 3 other courses. YMMV.
- tomku 14y ago
  I'm looking at the same problem at the moment. PGM sounds really interesting, but I think that the time investment just isn't going to be workable for me unless I drop several of my other classes. My current plan is to watch the PGM videos and try to keep up with the programming assignments as long as I can, but if it comes down to a choice of one or the other, PGM will be the one to go. As far as "dumbing down", I've found that the Coursera classes that I've taken (Compilers, Automata Theory, Algorithms 1, SaaS and Machine Learning) have varied in difficulty quite widely. Compilers and Automata were both challenging and enjoyable, Algorithms 1 was about what I'd expect from a freshman/sophomore algorithms class and SaaS and Machine Learning were easy enough that they should be approachable to anyone with basic programming experience. I don't feel that the difficulty in the classes that I've taken had any particular correlation with teaching effectiveness. I found Andrew Ng's ML class to be simple, but still interesting and informative - you come out of it with enough of a basic understanding to implement simple ML techniques as well as a place to start if you wish to learn more. I think that while a theory-centric class would be a nice thing to have, he's done an amazing job of making a class that can appeal to a wide range of potential students and introduce them to a field that's usually very difficult to approach.

azakai 14y ago

> Neural Networks are gradually taking over from simpler Machine Learning methods And haven't SVMs and such gradually taken over from Neural Networks?

nphrk 14y ago
Well not quite. While SVMs gained a lot of popularity for having nice properties e.g. 1) a convex problem which means a unique solution and a lot of already existing technology can be used 2) the "kernel trick" which enables us to learn in complicated spaces without computing the transformations 3) can be trained online, which makes them great for huge datasets (here the point 2) might not apply - but there exist ways - if someone's interested I can point out some papers) There is an ongoing craze about deep belief networks developed by Hinton (who is teaching this course) who came up with an algorithm that can train them (there exist local optima and such, so it's far from ideal). Some of the reasons they're popular 1) They seem to be winning algorithm for many competitions / datasets, ranging from classification in computer vision to speech recognition and if I'm not mistaken even parsing. They are for example used in the newer Androids. 2) They can be used in an unsupervised mode to _automatically_ learn different representations (features) of the data, which can be then used in subsequent stages of the classification pipeline. This makes them very interesting because while labelled data might be hard to get by, we have a lot of unlaballed datasets thanks to the Internet. As what they can do - see the work by Andrew Ng when they automatically learned a cat detector. 3) They're "similar" to biological neural networks, so one might think they have the necessary richness for many interesting AI applications.
- notimetorelax 14y ago
  Enlightening response, could you please post links to papers that explain online training of SVM? Also, I found this paper [1] on unsupervised feature detection, if you have some additional material, I'll really appreciate if you could post it! [1] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.44.4834&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.44....
deleted 14y ago
[deleted]

isakovic 14y ago

I took Professor Hinton's course on Neural Networks as an undergrad. This man is the most intelligent person I have ever met. He is one of the giants.