10 ms·

Job placement program for top students in Stanford's online AI class

201 points by mlacitation 15y ago

rfrey 15y ago
Sadly, I'm not expecting any such letter from Ng's ML class, because I'll bet there's 10k people tied for top marks. That's at least partly due to an extremely effective teaching system and style, so I'd say it's a worthwhile tradeoff.
- ccarpenterg 15y ago
  Yes. The online ML class method is extremely effective and efficient too. After two months I'm able to apply ML to the most common problems without any previous background on the topic.
- joshz 15y ago
  I don't know the numbers for the ML class and the format is a bit different, however, for DB class from the many people that were signed up only 9180 took the midterm[1] and (only 908 got scores >= 18/20) <- That is wrong, final had 20 questions, midterm had 18. So that's 908 people who scored 18, 2082 with 17 or over and 3428 with 16 or over. So if ML numbers are in any way similar to this maybe it wouldn't be so bad. That is if there was such a letter. I'll add that Professor Widom's style is quite engaging as well. [1] http://www.youtube.com/watch?v=DLtUKf4Y99w#t=2m09s http://www.youtube.com/watch?v=DLtUKf4Y99w#t=2m09s
  - brg 15y ago
    The db midterm had a max score of 18, not 20. But I agree with the intentions of your post.
    - joshz 15y ago
      That's right, I confused the final with the midterm.
  - nosignal 15y ago
    I wonder about this. Apparently the letter we get at the end contains our ranking. I wonder what the distribution of scores would be; I'd assume there would be a small fraction of enrolments actually doing the exercises, a big spike at 100%, and most falling in the range of 80-100%. Personally I forgot that I hadn't done a couple of the quizzes, so did them late, and didn't complete the optional part of the first week's programming assignment on time, so although I've probably got a "good mark" on paper, I'm probably nowhere near the top 10% of students.
- rglullis 15y ago
  I have only the best possible things to say about the video lectures, but the lab assignments are just too easy. For something that is supposed to be the advanced track, I was hoping that they would do something more than "take this description of the function, implement it in Octave, make it pass our tests". Of course, I know it's their first class and you can't get everything perfect right off the bat. But I'd love to see the lab assignments being one larger project, some kind of task where you need to apply the principles learned to solve different problems. It would be also more effective in measuring/grading students. On an unrelated note: I wonder what is the license for the classes/videos. I am working now in an adaptive e-learning platform. It would be cool to put their videos and questions in our system and see how effective the system is.
  - jorgeortiz85 15y ago
    I agree the programming assignments have been very easy. However, I doubt they have the bandwidth to manually grade thousands of assignments. Given the scale of the class, the grading is forces to be automated. It's hard to come up with such assignments and even harder to make them challenging, given an automated grading script.
    - mikeknoop 15y ago
      Ironic that the class is about AI, no?
    - rglullis 15y ago
      The grading doesn't need to be manual. Think of the lab assignments being something like a smaller scale of the Netflix challenge: 1) They provide some set of data and establish rules for the competition. 2) They implement their own solution to the challenge, and that is the benchmark. 3) A "passing grade" is obtained by getting any working system. 4) The actual grading is then given on a curve, compared against their benchmark. If your project is better than the benchmark, you get an A+, 95%-100% an A, 85%-95% gets you a B... etc.
      - nosignal 15y ago
lemonad 15y ago
It would be interesting to know what data they use for these kinds of assessments outside the given ones for the course, that is the best 6 homeworks out of 8 account for 30% of the grade, the midterm for 30% and the final exam for 40%.
rhygar 15y ago
A program like this is a great way to promote cheating.
tzs 15y ago
Drat. If I'd known they were going to do that, I'd have made sure I was in the top 1000. edit: why is this getting voted down? I would expect there are a fair number of people here who took the course with a casual approach, like I did. E.g., I usually watched the lectures early Sunday, then did the homework that night. Then, for those topics that I enjoyed and wanted in more depth, I read the sections in the book on them. This approach is relaxing and fun, but does increase the tendency to make silly mistakes and not catch them (which I in fact have done a few times now). I am, it turns out, enjoying the material enough that I think I would like, if I ever find myself looking for a job, to find one where I can use this stuff.
- Tichy 15y ago
  Is there any way to know how well you are doing (before the end of the course)? I thought I wasn't doing too badly, but with 100000 enrolled students there are bound to be lots who have 100% scores.
  - tzs 15y ago
    I don't think it is anywhere near 100k students. I can't find the cite, but I thought they commented somewhere that 20k or something like that people took the midterm. I presume people are guessing how they are doing by just looking at their homework and midterm scores so far, weighted according to the appropriate weights.
    - token4 15y ago
      I think I saw somewhere that 100k+ students had signed up initially. However, according to this NYTimes article (http://www.nytimes.com/2011/12/05/technology/khan-academy-blends-its-youtube-approach-with-classrooms.html http://www.nytimes.com/2011/12/05/technology/khan-academy-bl...): "But in education circles, Mr. Khan’s efforts have captured imaginations and spawned imitators. Two Stanford professors have drawn on his model to offer a free online artificial intelligence class. Thirty-four thousand people are now taking the course, and many more have signed up." Not sure if "taking the course" means advanced track and completed midterm or not.
quasistar 15y ago
Not sure I agree with the correlation that the 'most talented engineers' are those that 'scored highest on the AI class HWs and Exams,' but I certainly wouldn't refuse any of the folks who were able to solve the ApproximateAgent PacMan Search problem in under 30 seconds from joining my team ;) Its wonderful that Sebastian & Peter are reaching out like this after all they have done so far. Congrats to everyone that slugged it out, and good luck on the final this weekend!
- Tichy 15y ago
  Which one was the PacMan question again (unit/homework number)?
- bgutierrez 15y ago
  There are probably a lot of talented engineers not in the top 1,000, but there are also a lot of non-talented engineers not in the top 1,000. I guess it's a reasonable cut-off. Thankfully it's not the only way to get into a great company.
WildUtah 15y ago
Well, I haven't received any such email and my score in the class is 98% so far (with only the final exam pending this week). I expected that around 10,000 people would have a perfect score in the class given my experience with Asian participation in online programming contests. There are a few countries in that part of the world where organizing a group to check each others' work and ensuring perfect scores is common and encouraged even when the official requirement is to do your own work. And those happen to be countries filled with millions of smart people. India, China, and Russia seem to be the heart of the phenomenon. Google, Facebook, and Topcoder have systems to deal with it but ai-class.org does not. Anyway I'm neither surprised nor disappointed not to be in the top sliver. I'm not hireable anyway. It's funny that there is an identifiable top 1,000 at all instead of 10k+ perfect scores.
- learc83 15y ago
  I got the letter, I made a 100% on everything except homework 6 and 7 (missed 1 each on those).
  - YokYok 15y ago
    I made only one mistake (98% in homework 6) and haven't got the letter.
    - zmj 15y ago
      Have you done all the other homeworks? I also missed only 1 question on HW6 and I did get the letter.
      - YokYok 15y ago
        I did them all and got 100% everywhere (except in HW6). I am kind of disappointed. May be the quizzes are part of the grading?
        wizzard 15y ago
        I think this has to be the case. My friend has a lower HW and midterm average than me, but he got the letter and I did not.
clavalle 15y ago
Likely they look at everything, not just the graded portions, to come up with that kind of split. So if you got 100% on every quiz the first time, every homework and test you'd be in that group. If you flubbed the quizzes here and there, probably not. They will also probably look closely at that optional programming assignment as well. I knew there was a reason Google let Peter Norvig take the time to do this ;)
- brg 15y ago
  The optional programming assignment is a standard quiz. There is really not much to it, and can be gamed without writing a line of code.
dianaZ 15y ago
Is this for real? Can anyone verify that this letter was in fact sent out?
- learc83 15y ago
  Yes, I received one, and it's from the same stanford.edu email address that sent the letter congratulating people on perfect 100s after homework 4.
- wesnerm2 15y ago
  I got the letter. I received a 100% on the homeworks and 98% on the midterm.
redcircle 15y ago
This type of thing wouldn't help you get a job with me. It could make a difference in getting an interview with me, if you had no real experience on your resume/CV. I want a well-balanced team player, that has a good chance to flourish on my team. Do course grades correlate with real-world flourishing on my team? Of course not.
brg 15y ago
It seems that this was sent to students with 100% aggregate score up to this point (after dropping the bottom two quizes). One acquaintance had 1 wrong in aggregate and another had 0 wrong in aggregate but two imperfect quizes, and only the latter claims to have received the letter.
- learc83 15y ago
  From what I've seen I think they are taking all scores, I've heard from someone who got 6 100s and didn't get it, but he didn't attempt the last 2 quizes b/c he already had 6 100s. I made 6 100's, a 98, and a 97, and a 100 on the midterm and I got it.
robryan 15y ago
As someone doing the course i would say its a decent way to identify some candidates but it wont be a really high correlation. It depends a lot on the time you have to invest and how rigorously you go through your answers and study the material the questions are drawn from. Some programming challenges would be great to better separate whether people really understood the material despite being harder to mark. Personally I've always got a bit wrong in most of he homeworks but usually felt that while I understood the material I didn't recite enough time to rigorously check my answers and rewatch the materials to pick up on some of the subtle things you need to remember or some questions.
- learc83 15y ago
  They are also looking for people to work with Professor Thrun's company. So maybe they are looking for people who are passionate about the class.
  - aerique 15y ago
    I'm passionate about the class but I also have to juggle it with a new job and family life which made approaching the class casually the best option for me. So their metric isn't ideal for identifying passionate poeple ;-)
    - learc83 15y ago
      I'm sure it's not perfect, and there are exceptions. But I'm willing to be there will be a high correlation between high scores and passion.
- tcarney 15y ago
  They may be trying to collect resumes (and job placement performance) from the top 1000 students to see what the correlation is. If they can prove that correlation, then they could essentially run the courses as a recruiting tool for paying companies.
- Homunculiheaded 15y ago
  I took this course casually as well (wanted to fill in some gaps in my undergrad education). But my guess is that people that complete this course casual probably already have a lot on their plates. For me it was family, working, teaching and taking a grad class. I never double checked my hws, could only do a single pass through the lectures, and didn't spend any time checking for nuance in many of the Qs and my grade shows accordingly. However in the end I'm really excited that I was able to continue through it despite all this (I'm really curious to see the percentage of participants who saw the course through completion compared to those who signed up). The correlation will be interesting. Undoubtedly the top 1000 is made of many of the type of people that must be the best at whatever they try (not exclusively of course) and with a decent amount of time on their hands, after all there is no disincentive whatsoever for not completing something and no previously known reward for perfection (I guess a piece of paper can count). Given some wording in some of the Qs (actually my only complaint about this course) a lot of care must be taken to ensure that every question is right, so being in the top 1000 is no small effort.
LogicX 15y ago
Surprised no-one brought it up yet: What I'd like to know is if anyone involved with the class is getting a referral bonus for bringing candidates to companies... Follow up is how we feel about that...
- sbisker 15y ago
  I'm not sure why there's a down-vote here, because that was exactly the question I had. There has been a system of professors referring their top students to their colleagues for years, and on a small scale I don't see much issue. However, with these large online classes, these professors can stand to be gatekeepers to an incredibly massive - and valuable - source of leads. It doesn't help when there is no true transparency to the process, and a desire for the school to recoup costs - we assume the professor would just send the top 1000 over regardless of source, but what's to stop them from only referring the names of students who, say, enroll for credit and pay the university? Or those who pay some sort of a separate "consideration fee"? I can't put my finger on why, but I'm a little unnerved by this...
HilbertSpace 15y ago
Maybe, but be careful and think 'long shot': I was one of three researchers in a team at Yorktown Heights that did some of the best research in AI in the world, published a string of peer-reviewed papers, was the source of two commercial products, gave a paper at the AAAI IAAI conference in Stanford with for that year the 25 best AI applications in the world, personally won an award, etc. I've taught computer science at Georgetown University and Ohio State University. My Ph.D. is in applied math. Once in my career, just as a 'scientific programmer', in two weeks I sent a few resume copies, went on seven interviews, and got five offers. But after my Ph.D. and AI work, I sent over 1000 resumes to Google, Microsoft, GE, FedEx, and hundreds more, got only five interviews, and no offers. I got a nice letter back from Fisher Black (as in Black-Scholes) saying that he saw no applications of applied math or AI at GS. I ask you: Who will hire you in AI and why? In business, hiring is because some manager has some work to do and a budget to do it. That manager believes that they know nearly all that is needed to do the work and otherwise would not be betting his career on the work. Thus, the manager is not hiring high technical expertise he doesn't have. Instead the manager is, as on a factory floor 100 years ago, hiring labor to add 'muscle' to his work. In particular, unless the manager knows AI, he won't be hiring for AI. And there is at most only a tiny chance that the manager knows AI and even less chance that his project will depend on AI. Moreover, the manager does not want competition from below and does not want his project 'disrupted' from below so really doesn't want technical expertise above what is needed just to get his project done. Net, if you know some AI and want to use it in business, then find an application and start your own business. Then, since you know AI, you won't have to hire anyone in AI either. What I've said here for AI holds for essentially all advanced academic topics. Sorry 'bout that.
- anothermachine 15y ago
  My (small) office has our first AI Ph.D starting next month. You sound like a qualified candidate, if only you'd sent your resume our way. Sorry 'bout that.
- loboman 15y ago
  Yet there are AI applications coming from Google, Apple, Microsoft and many other companies (voice products, kinect, etc.), and service based companies that work on these advanced topics too like ITA used to be. Why do you think those companies weren't interested in you? or weren't you interested in the projects they had to offer? (I'm guessing it's the latter but I could be wrong) Edit: for example some opportunities here http://www.kdnuggets.com/jobs/ http://www.kdnuggets.com/jobs/
ih 15y ago
Just want to mention that the start-up behind ai-class.com is looking for great engineers. If you're interested in education shoot me an email at ih@knowlabs.com.
sriram_sun 15y ago
Getting new jobs is great. However, stay put in your jobs. Try to apply what you learned to little problems at work. Everyone has a huge database these days. I work for a medical device company and I write software that processes blood (embedded stuff). Every year, a million or so run logs find their way into one of our databases. There is a wealth of information there that we could figure out. Eg. How do me maximize yield, what's the best we can do with a certain type of donor. Do we perform worse on people in a certain geographic region etc. Eventually make useful predictions based on that data. Granted, that is not what I was hired to do, but I'm going to do it in my "free" time. Why? Because it is interesting. Find something to apply to that might add value to the business. If a enough people do that, AI techniques might become common place. Get on Kaggle, GitHub your next project etc. Putting what we've learned to practical use is how we can proliferate and disseminate this knowledge. This was the intent of the class in the first place anyway. tl;dr Using this course as a gateway to your next job is shortsighted IMO.
- nosignal 15y ago
  I wish I could +1 this more than once. This is exactly the point, to me. There's not really such a thing as "an AI job". There is application of AI techniques to existing problems. Those problems are already found riddled throughout every job everyone's ever worked in. You can't sit back and wait for AI to "happen". What would that even look like? The benefit of these techniques is in application to existing data; you don't have to quit and found some kind of computer vision/kinect/big data startup to take advantage of the knowledge. You should be aiming to put yourself out of a job by using AI.
casualaistudent 15y ago
Why are they sending out this letter before the final?
plinkplonk 15y ago
Prof Thrun mentioned in the last office hours that there were about 1650 people with "perfect scores" and he seemed a little thunderstruck by that and said he would admit these folks into Stanford if he could, because they are (paraphrasing, from memory)"Stanford quality". What struck me was how much importance he gave to this metric, which isn't that hard to game on an online offering. I know a few people (who shall remain nameless) who collaborate and check each others answers and so on before submission, in direct violation of the Stanford policy (and have 100s or close to it), and so probably have received this mail, whereas more "deserving" (note quotes) people who honestly work through the course material may not because they have, say, a 85% or 90% score. That said, my key takeaway from this is that professors are very impressed by perfect scores irrespective of how you got them. There must be something magical about that row of 100s. Once you set up a grading/ranking system, it is psychologically very hard not to admire people who end up at the top. I am personally a little dubious that the people with the highest scores would make the best pool of employees, especially given that this is an online course without the programming component, but what do I know? I wrote Java code for most of the algorithms in AIMA as a side project a few years ago [1], and after I read an online post by Peter Norvig saying a few of his students had tried and failed a few times (to implement the code in Java- Common Lisp code existed and the Python version was in its infancy), I sent him the code and this became the "official" java distribution for AIMA ( though I don't maintain it anymore- the immensely talented Ciaran O'Reilly of the Stanford Research Institute does) and no one ever invited me to Stanford or offered me a cool AI job[2], sob! :-P. No I am not bitter I tell you, not even the teeniest bit :-p [3] I wonder how this signalling will play into the upcoming courses? If there are tangential real world benefits to be gained by attempting a "perfect" score, then you can expect a lot more game playing wrt scores and exams. [1] More about how Peter Norvig shredded my initial code etc here http://news.ycombinator.com/item?id=2405277 http://news.ycombinator.com/item?id=2405277 [2] though eventually, after a lot more work, it did lead to my working on good ML/robotics etc projects from Bangalore, which is a hard thing to do in the Great Outsourcing Wasteland. [3] I am really not bitter. I wrote the code for the hell of it, not to get a job. AIMA was my introduction to the fascinating field of AI. It is a great, great book and it has a lot more material than is covered in the course. I once did want to go to Stanford and learn from the great profs there, but now in a "mountain comes to Mohammed" fashion, Stanford is coming to me. I don't care about the credentialling - I just want to learn. I took the AI online course and enjoyed Peter's and Sebastian's teaching immensely. Fwiw I should have a high 90's score, (I didn't add it all up) but nowhere near a perfect score.
jpeterson 15y ago
Well, finding out that I've no chance of making the top 1,000 in the class or any opportunity of a job is certainly a swift kick in the huevos, 2 days before the final... :(
agentultra 15y ago
Bugger. I had to drop out because my cat died and I fell behind on the home work. I hope they do it again.
jongraehl 15y ago
That this inevitably leaked due to someone's bragging will encourage future cheating. On the other hand, it was inevitable that as long as "grades" or equivalent are officially certified (even though the course is not for degree credit), that people would collect online courses as credentials. Wherever there's signaling value, there will be cheaters faking the signal. Disclaimer: I got the email and missed two homework questions.

201 points by mlacitation 15y ago

rfrey 15y ago

Sadly, I'm not expecting any such letter from Ng's ML class, because I'll bet there's 10k people tied for top marks. That's at least partly due to an extremely effective teaching system and style, so I'd say it's a worthwhile tradeoff.

ccarpenterg 15y ago
Yes. The online ML class method is extremely effective and efficient too. After two months I'm able to apply ML to the most common problems without any previous background on the topic.
joshz 15y ago
I don't know the numbers for the ML class and the format is a bit different, however, for DB class from the many people that were signed up only 9180 took the midterm[1] and (only 908 got scores >= 18/20) <- That is wrong, final had 20 questions, midterm had 18. So that's 908 people who scored 18, 2082 with 17 or over and 3428 with 16 or over. So if ML numbers are in any way similar to this maybe it wouldn't be so bad. That is if there was such a letter. I'll add that Professor Widom's style is quite engaging as well. [1] http://www.youtube.com/watch?v=DLtUKf4Y99w#t=2m09s http://www.youtube.com/watch?v=DLtUKf4Y99w#t=2m09s
- brg 15y ago
  The db midterm had a max score of 18, not 20. But I agree with the intentions of your post.
  - joshz 15y ago
    That's right, I confused the final with the midterm.
- nosignal 15y ago
  I wonder about this. Apparently the letter we get at the end contains our ranking. I wonder what the distribution of scores would be; I'd assume there would be a small fraction of enrolments actually doing the exercises, a big spike at 100%, and most falling in the range of 80-100%. Personally I forgot that I hadn't done a couple of the quizzes, so did them late, and didn't complete the optional part of the first week's programming assignment on time, so although I've probably got a "good mark" on paper, I'm probably nowhere near the top 10% of students.
rglullis 15y ago
I have only the best possible things to say about the video lectures, but the lab assignments are just too easy. For something that is supposed to be the advanced track, I was hoping that they would do something more than "take this description of the function, implement it in Octave, make it pass our tests". Of course, I know it's their first class and you can't get everything perfect right off the bat. But I'd love to see the lab assignments being one larger project, some kind of task where you need to apply the principles learned to solve different problems. It would be also more effective in measuring/grading students. On an unrelated note: I wonder what is the license for the classes/videos. I am working now in an adaptive e-learning platform. It would be cool to put their videos and questions in our system and see how effective the system is.
- jorgeortiz85 15y ago
  I agree the programming assignments have been very easy. However, I doubt they have the bandwidth to manually grade thousands of assignments. Given the scale of the class, the grading is forces to be automated. It's hard to come up with such assignments and even harder to make them challenging, given an automated grading script.
  - mikeknoop 15y ago
    Ironic that the class is about AI, no?
  - rglullis 15y ago
    The grading doesn't need to be manual. Think of the lab assignments being something like a smaller scale of the Netflix challenge: 1) They provide some set of data and establish rules for the competition. 2) They implement their own solution to the challenge, and that is the benchmark. 3) A "passing grade" is obtained by getting any working system. 4) The actual grading is then given on a curve, compared against their benchmark. If your project is better than the benchmark, you get an A+, 95%-100% an A, 85%-95% gets you a B... etc.
    - nosignal 15y ago

lemonad 15y ago

It would be interesting to know what data they use for these kinds of assessments outside the given ones for the course, that is the best 6 homeworks out of 8 account for 30% of the grade, the midterm for 30% and the final exam for 40%.

rhygar 15y ago

A program like this is a great way to promote cheating.

tzs 15y ago

Drat. If I'd known they were going to do that, I'd have made sure I was in the top 1000. edit: why is this getting voted down? I would expect there are a fair number of people here who took the course with a casual approach, like I did. E.g., I usually watched the lectures early Sunday, then did the homework that night. Then, for those topics that I enjoyed and wanted in more depth, I read the sections in the book on them. This approach is relaxing and fun, but does increase the tendency to make silly mistakes and not catch them (which I in fact have done a few times now). I am, it turns out, enjoying the material enough that I think I would like, if I ever find myself looking for a job, to find one where I can use this stuff.

Tichy 15y ago
Is there any way to know how well you are doing (before the end of the course)? I thought I wasn't doing too badly, but with 100000 enrolled students there are bound to be lots who have 100% scores.
- tzs 15y ago
  I don't think it is anywhere near 100k students. I can't find the cite, but I thought they commented somewhere that 20k or something like that people took the midterm. I presume people are guessing how they are doing by just looking at their homework and midterm scores so far, weighted according to the appropriate weights.
  - token4 15y ago
    I think I saw somewhere that 100k+ students had signed up initially. However, according to this NYTimes article (http://www.nytimes.com/2011/12/05/technology/khan-academy-blends-its-youtube-approach-with-classrooms.html http://www.nytimes.com/2011/12/05/technology/khan-academy-bl...): "But in education circles, Mr. Khan’s efforts have captured imaginations and spawned imitators. Two Stanford professors have drawn on his model to offer a free online artificial intelligence class. Thirty-four thousand people are now taking the course, and many more have signed up." Not sure if "taking the course" means advanced track and completed midterm or not.

quasistar 15y ago

Not sure I agree with the correlation that the 'most talented engineers' are those that 'scored highest on the AI class HWs and Exams,' but I certainly wouldn't refuse any of the folks who were able to solve the ApproximateAgent PacMan Search problem in under 30 seconds from joining my team ;) Its wonderful that Sebastian & Peter are reaching out like this after all they have done so far. Congrats to everyone that slugged it out, and good luck on the final this weekend!

Tichy 15y ago
Which one was the PacMan question again (unit/homework number)?
bgutierrez 15y ago
There are probably a lot of talented engineers not in the top 1,000, but there are also a lot of non-talented engineers not in the top 1,000. I guess it's a reasonable cut-off. Thankfully it's not the only way to get into a great company.

WildUtah 15y ago

Well, I haven't received any such email and my score in the class is 98% so far (with only the final exam pending this week). I expected that around 10,000 people would have a perfect score in the class given my experience with Asian participation in online programming contests. There are a few countries in that part of the world where organizing a group to check each others' work and ensuring perfect scores is common and encouraged even when the official requirement is to do your own work. And those happen to be countries filled with millions of smart people. India, China, and Russia seem to be the heart of the phenomenon. Google, Facebook, and Topcoder have systems to deal with it but ai-class.org does not. Anyway I'm neither surprised nor disappointed not to be in the top sliver. I'm not hireable anyway. It's funny that there is an identifiable top 1,000 at all instead of 10k+ perfect scores.

learc83 15y ago
I got the letter, I made a 100% on everything except homework 6 and 7 (missed 1 each on those).
- YokYok 15y ago
  I made only one mistake (98% in homework 6) and haven't got the letter.
  - zmj 15y ago
    Have you done all the other homeworks? I also missed only 1 question on HW6 and I did get the letter.
    - YokYok 15y ago
      I did them all and got 100% everywhere (except in HW6). I am kind of disappointed. May be the quizzes are part of the grading?
      - wizzard 15y ago
        I think this has to be the case. My friend has a lower HW and midterm average than me, but he got the letter and I did not.

clavalle 15y ago

Likely they look at everything, not just the graded portions, to come up with that kind of split. So if you got 100% on every quiz the first time, every homework and test you'd be in that group. If you flubbed the quizzes here and there, probably not. They will also probably look closely at that optional programming assignment as well. I knew there was a reason Google let Peter Norvig take the time to do this ;)

brg 15y ago
The optional programming assignment is a standard quiz. There is really not much to it, and can be gamed without writing a line of code.

dianaZ 15y ago

Is this for real? Can anyone verify that this letter was in fact sent out?

learc83 15y ago
Yes, I received one, and it's from the same stanford.edu email address that sent the letter congratulating people on perfect 100s after homework 4.
wesnerm2 15y ago
I got the letter. I received a 100% on the homeworks and 98% on the midterm.

redcircle 15y ago

This type of thing wouldn't help you get a job with me. It could make a difference in getting an interview with me, if you had no real experience on your resume/CV. I want a well-balanced team player, that has a good chance to flourish on my team. Do course grades correlate with real-world flourishing on my team? Of course not.

brg 15y ago

It seems that this was sent to students with 100% aggregate score up to this point (after dropping the bottom two quizes). One acquaintance had 1 wrong in aggregate and another had 0 wrong in aggregate but two imperfect quizes, and only the latter claims to have received the letter.

learc83 15y ago
From what I've seen I think they are taking all scores, I've heard from someone who got 6 100s and didn't get it, but he didn't attempt the last 2 quizes b/c he already had 6 100s. I made 6 100's, a 98, and a 97, and a 100 on the midterm and I got it.

robryan 15y ago

As someone doing the course i would say its a decent way to identify some candidates but it wont be a really high correlation. It depends a lot on the time you have to invest and how rigorously you go through your answers and study the material the questions are drawn from. Some programming challenges would be great to better separate whether people really understood the material despite being harder to mark. Personally I've always got a bit wrong in most of he homeworks but usually felt that while I understood the material I didn't recite enough time to rigorously check my answers and rewatch the materials to pick up on some of the subtle things you need to remember or some questions.

learc83 15y ago
They are also looking for people to work with Professor Thrun's company. So maybe they are looking for people who are passionate about the class.
- aerique 15y ago
  I'm passionate about the class but I also have to juggle it with a new job and family life which made approaching the class casually the best option for me. So their metric isn't ideal for identifying passionate poeple ;-)
  - learc83 15y ago
    I'm sure it's not perfect, and there are exceptions. But I'm willing to be there will be a high correlation between high scores and passion.
tcarney 15y ago
They may be trying to collect resumes (and job placement performance) from the top 1000 students to see what the correlation is. If they can prove that correlation, then they could essentially run the courses as a recruiting tool for paying companies.
Homunculiheaded 15y ago
I took this course casually as well (wanted to fill in some gaps in my undergrad education). But my guess is that people that complete this course casual probably already have a lot on their plates. For me it was family, working, teaching and taking a grad class. I never double checked my hws, could only do a single pass through the lectures, and didn't spend any time checking for nuance in many of the Qs and my grade shows accordingly. However in the end I'm really excited that I was able to continue through it despite all this (I'm really curious to see the percentage of participants who saw the course through completion compared to those who signed up). The correlation will be interesting. Undoubtedly the top 1000 is made of many of the type of people that must be the best at whatever they try (not exclusively of course) and with a decent amount of time on their hands, after all there is no disincentive whatsoever for not completing something and no previously known reward for perfection (I guess a piece of paper can count). Given some wording in some of the Qs (actually my only complaint about this course) a lot of care must be taken to ensure that every question is right, so being in the top 1000 is no small effort.

LogicX 15y ago

Surprised no-one brought it up yet: What I'd like to know is if anyone involved with the class is getting a referral bonus for bringing candidates to companies... Follow up is how we feel about that...

sbisker 15y ago
I'm not sure why there's a down-vote here, because that was exactly the question I had. There has been a system of professors referring their top students to their colleagues for years, and on a small scale I don't see much issue. However, with these large online classes, these professors can stand to be gatekeepers to an incredibly massive - and valuable - source of leads. It doesn't help when there is no true transparency to the process, and a desire for the school to recoup costs - we assume the professor would just send the top 1000 over regardless of source, but what's to stop them from only referring the names of students who, say, enroll for credit and pay the university? Or those who pay some sort of a separate "consideration fee"? I can't put my finger on why, but I'm a little unnerved by this...

HilbertSpace 15y ago

Maybe, but be careful and think 'long shot': I was one of three researchers in a team at Yorktown Heights that did some of the best research in AI in the world, published a string of peer-reviewed papers, was the source of two commercial products, gave a paper at the AAAI IAAI conference in Stanford with for that year the 25 best AI applications in the world, personally won an award, etc. I've taught computer science at Georgetown University and Ohio State University. My Ph.D. is in applied math. Once in my career, just as a 'scientific programmer', in two weeks I sent a few resume copies, went on seven interviews, and got five offers. But after my Ph.D. and AI work, I sent over 1000 resumes to Google, Microsoft, GE, FedEx, and hundreds more, got only five interviews, and no offers. I got a nice letter back from Fisher Black (as in Black-Scholes) saying that he saw no applications of applied math or AI at GS. I ask you: Who will hire you in AI and why? In business, hiring is because some manager has some work to do and a budget to do it. That manager believes that they know nearly all that is needed to do the work and otherwise would not be betting his career on the work. Thus, the manager is not hiring high technical expertise he doesn't have. Instead the manager is, as on a factory floor 100 years ago, hiring labor to add 'muscle' to his work. In particular, unless the manager knows AI, he won't be hiring for AI. And there is at most only a tiny chance that the manager knows AI and even less chance that his project will depend on AI. Moreover, the manager does not want competition from below and does not want his project 'disrupted' from below so really doesn't want technical expertise above what is needed just to get his project done. Net, if you know some AI and want to use it in business, then find an application and start your own business. Then, since you know AI, you won't have to hire anyone in AI either. What I've said here for AI holds for essentially all advanced academic topics. Sorry 'bout that.

anothermachine 15y ago
My (small) office has our first AI Ph.D starting next month. You sound like a qualified candidate, if only you'd sent your resume our way. Sorry 'bout that.
loboman 15y ago
Yet there are AI applications coming from Google, Apple, Microsoft and many other companies (voice products, kinect, etc.), and service based companies that work on these advanced topics too like ITA used to be. Why do you think those companies weren't interested in you? or weren't you interested in the projects they had to offer? (I'm guessing it's the latter but I could be wrong) Edit: for example some opportunities here http://www.kdnuggets.com/jobs/ http://www.kdnuggets.com/jobs/

ih 15y ago

Just want to mention that the start-up behind ai-class.com is looking for great engineers. If you're interested in education shoot me an email at ih@knowlabs.com.

sriram_sun 15y ago

Getting new jobs is great. However, stay put in your jobs. Try to apply what you learned to little problems at work. Everyone has a huge database these days. I work for a medical device company and I write software that processes blood (embedded stuff). Every year, a million or so run logs find their way into one of our databases. There is a wealth of information there that we could figure out. Eg. How do me maximize yield, what's the best we can do with a certain type of donor. Do we perform worse on people in a certain geographic region etc. Eventually make useful predictions based on that data. Granted, that is not what I was hired to do, but I'm going to do it in my "free" time. Why? Because it is interesting. Find something to apply to that might add value to the business. If a enough people do that, AI techniques might become common place. Get on Kaggle, GitHub your next project etc. Putting what we've learned to practical use is how we can proliferate and disseminate this knowledge. This was the intent of the class in the first place anyway. tl;dr Using this course as a gateway to your next job is shortsighted IMO.

nosignal 15y ago
I wish I could +1 this more than once. This is exactly the point, to me. There's not really such a thing as "an AI job". There is application of AI techniques to existing problems. Those problems are already found riddled throughout every job everyone's ever worked in. You can't sit back and wait for AI to "happen". What would that even look like? The benefit of these techniques is in application to existing data; you don't have to quit and found some kind of computer vision/kinect/big data startup to take advantage of the knowledge. You should be aiming to put yourself out of a job by using AI.

casualaistudent 15y ago

Why are they sending out this letter before the final?

plinkplonk 15y ago

Prof Thrun mentioned in the last office hours that there were about 1650 people with "perfect scores" and he seemed a little thunderstruck by that and said he would admit these folks into Stanford if he could, because they are (paraphrasing, from memory)"Stanford quality". What struck me was how much importance he gave to this metric, which isn't that hard to game on an online offering. I know a few people (who shall remain nameless) who collaborate and check each others answers and so on before submission, in direct violation of the Stanford policy (and have 100s or close to it), and so probably have received this mail, whereas more "deserving" (note quotes) people who honestly work through the course material may not because they have, say, a 85% or 90% score. That said, my key takeaway from this is that professors are very impressed by perfect scores irrespective of how you got them. There must be something magical about that row of 100s. Once you set up a grading/ranking system, it is psychologically very hard not to admire people who end up at the top. I am personally a little dubious that the people with the highest scores would make the best pool of employees, especially given that this is an online course without the programming component, but what do I know? I wrote Java code for most of the algorithms in AIMA as a side project a few years ago [1], and after I read an online post by Peter Norvig saying a few of his students had tried and failed a few times (to implement the code in Java- Common Lisp code existed and the Python version was in its infancy), I sent him the code and this became the "official" java distribution for AIMA ( though I don't maintain it anymore- the immensely talented Ciaran O'Reilly of the Stanford Research Institute does) and no one ever invited me to Stanford or offered me a cool AI job[2], sob! :-P. No I am not bitter I tell you, not even the teeniest bit :-p [3] I wonder how this signalling will play into the upcoming courses? If there are tangential real world benefits to be gained by attempting a "perfect" score, then you can expect a lot more game playing wrt scores and exams. [1] More about how Peter Norvig shredded my initial code etc here http://news.ycombinator.com/item?id=2405277 http://news.ycombinator.com/item?id=2405277 [2] though eventually, after a lot more work, it did lead to my working on good ML/robotics etc projects from Bangalore, which is a hard thing to do in the Great Outsourcing Wasteland. [3] I am really not bitter. I wrote the code for the hell of it, not to get a job. AIMA was my introduction to the fascinating field of AI. It is a great, great book and it has a lot more material than is covered in the course. I once did want to go to Stanford and learn from the great profs there, but now in a "mountain comes to Mohammed" fashion, Stanford is coming to me. I don't care about the credentialling - I just want to learn. I took the AI online course and enjoyed Peter's and Sebastian's teaching immensely. Fwiw I should have a high 90's score, (I didn't add it all up) but nowhere near a perfect score.

jpeterson 15y ago

Well, finding out that I've no chance of making the top 1,000 in the class or any opportunity of a job is certainly a swift kick in the huevos, 2 days before the final... :(

agentultra 15y ago

Bugger. I had to drop out because my cat died and I fell behind on the home work. I hope they do it again.

jongraehl 15y ago

That this inevitably leaked due to someone's bragging will encourage future cheating. On the other hand, it was inevitable that as long as "grades" or equivalent are officially certified (even though the course is not for degree credit), that people would collect online courses as credentials. Wherever there's signaling value, there will be cheaters faking the signal. Disclaimer: I got the email and missed two homework questions.