6 ms·

Thanks for this insight. Can you kindly also suggest a good book for someone to start with Bayesian Statistics? I could really use a suggestion about first and

Thanks for this insight. Can you kindly also suggest a good book for someone to start with Bayesian Statistics? I could really use a suggestion about first and second book on this. About Probabilistic Graphical Models, is there book other than Daphne Koller's book that you would suggest?

rangerranvir 6y ago
+1
uoaei 6y ago
Introduction to Statistical Learning https://faculty.marshall.usc.edu/gareth-james/ISL/ https://faculty.marshall.usc.edu/gareth-james/ISL/ Elements of Statistical Learning https://web.stanford.edu/~hastie/ElemStatLearn/ https://web.stanford.edu/~hastie/ElemStatLearn/ Machine Learning: A Probabilistic Perspective https://mitpress.mit.edu/books/machine-learning-1 https://mitpress.mit.edu/books/machine-learning-1
- rangerranvir 6y ago
  Thanks a ton for these. Added this to things I know that I don't know list. ;)
- kmundnic 6y ago
  "Machine Learning: a Probabilistic Perspective" is more an encyclopedia of algorithms I would say, and it has lots of typos. I personally would not recommend it (except for the amount of algorithms that it covers, many of which are usually not found in other books).
  - rangerranvir 6y ago
    Thanks for early warning. Will have to keep that in mind.
- antipaul 6y ago
  Are those really the best starts for "Bayesian statistics"? Especially the first 2 are rather the standard "intro to ML textbooks", with a frequentist focus (ISL may even have zero Bayesian stuff - Naive Bayes is not "Bayesian" – while ESL still has maybe 10% bayesian content if that). Instead, I would suggest the following for learning Bayesian methods, especially given the HN crowd: https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers https://github.com/CamDavidsonPilon/Probabilistic-Programmin...
  - uoaei 6y ago
    You make a good point. It's been a while since I flipped through them, they just come up in lots of discussions on this topic. I agree that the series you link to is really great for PPL and Bayesian methods. You may find that the library upon which it's based (PyMC3) is built on top of Theano, which has been abandoned and deprecated. PyMC4 is around the corner and uses TensorFlow Probability. Early, informal reports say it's 10x faster.
astrophysician 6y ago
I think PGM's are covered by a lot of "standard" ML texts -- someone else mentioned Murphy's book which is great and is humongous but is a good reference for pretty much every method under the sun. Bishop's Pattern Recognition and Machine Learning has a chapter thats free online: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/05/Bishop-PRML-sample.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...
- jclos 6y ago
  I'd just like to add that the entire PRML book is now free online, not just the sample: https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf https://www.microsoft.com/en-us/research/uploads/prod/2006/0...
phunge 6y ago
For Bayesian stats, "Statistical Rethinking" by McElreath is a masterpiece.
- nightski 6y ago
  This should be the top comment, I'm reading his newly released 2nd edition and it is outstanding. Other mentions are - - Doing Bayesian Data Analysis (dog book) - Student's Guide to Bayesian Statistics Slightly more advanced - Bayesian Data Analysis 3 (currently free! http://www.stat.columbia.edu/~gelman/book/)
tesseract2 6y ago
Thanks, everyone, for your kind suggestions. Much appreciated.
solar_perplexus 6y ago
I took a course on Applied Bayesian Statistics taught by David Draper in grad school and we covered Bayesian Data Analysis (Gelman et Al.) http://www.stat.columbia.edu/~gelman/book/ http://www.stat.columbia.edu/~gelman/book/ and Probability Theory and tbe Logic of Science by Ed Jaynes: https://www.amazon.com/dp/0521592712/ref=cm_sw_r_em_apa_i_v35QEb3FEYSGM https://www.amazon.com/dp/0521592712/ref=cm_sw_r_em_apa_i_v3... The former is a much recommended book since it's very comprehensive and builds everything from the ground up and was the basis for the entire course. The latter is a beast of it's own and we simply covered what was effectively the first chapter as part of the course.

rangerranvir 6y ago

uoaei 6y ago

Introduction to Statistical Learning https://faculty.marshall.usc.edu/gareth-james/ISL/ https://faculty.marshall.usc.edu/gareth-james/ISL/ Elements of Statistical Learning https://web.stanford.edu/~hastie/ElemStatLearn/ https://web.stanford.edu/~hastie/ElemStatLearn/ Machine Learning: A Probabilistic Perspective https://mitpress.mit.edu/books/machine-learning-1 https://mitpress.mit.edu/books/machine-learning-1

rangerranvir 6y ago
Thanks a ton for these. Added this to things I know that I don't know list. ;)
kmundnic 6y ago
"Machine Learning: a Probabilistic Perspective" is more an encyclopedia of algorithms I would say, and it has lots of typos. I personally would not recommend it (except for the amount of algorithms that it covers, many of which are usually not found in other books).
- rangerranvir 6y ago
  Thanks for early warning. Will have to keep that in mind.
antipaul 6y ago
Are those really the best starts for "Bayesian statistics"? Especially the first 2 are rather the standard "intro to ML textbooks", with a frequentist focus (ISL may even have zero Bayesian stuff - Naive Bayes is not "Bayesian" – while ESL still has maybe 10% bayesian content if that). Instead, I would suggest the following for learning Bayesian methods, especially given the HN crowd: https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers https://github.com/CamDavidsonPilon/Probabilistic-Programmin...
- uoaei 6y ago
  You make a good point. It's been a while since I flipped through them, they just come up in lots of discussions on this topic. I agree that the series you link to is really great for PPL and Bayesian methods. You may find that the library upon which it's based (PyMC3) is built on top of Theano, which has been abandoned and deprecated. PyMC4 is around the corner and uses TensorFlow Probability. Early, informal reports say it's 10x faster.

astrophysician 6y ago

I think PGM's are covered by a lot of "standard" ML texts -- someone else mentioned Murphy's book which is great and is humongous but is a good reference for pretty much every method under the sun. Bishop's Pattern Recognition and Machine Learning has a chapter thats free online: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/05/Bishop-PRML-sample.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...

jclos 6y ago
I'd just like to add that the entire PRML book is now free online, not just the sample: https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf https://www.microsoft.com/en-us/research/uploads/prod/2006/0...

phunge 6y ago

For Bayesian stats, "Statistical Rethinking" by McElreath is a masterpiece.

nightski 6y ago
This should be the top comment, I'm reading his newly released 2nd edition and it is outstanding. Other mentions are - - Doing Bayesian Data Analysis (dog book) - Student's Guide to Bayesian Statistics Slightly more advanced - Bayesian Data Analysis 3 (currently free! http://www.stat.columbia.edu/~gelman/book/)

tesseract2 6y ago

Thanks, everyone, for your kind suggestions. Much appreciated.

solar_perplexus 6y ago

I took a course on Applied Bayesian Statistics taught by David Draper in grad school and we covered Bayesian Data Analysis (Gelman et Al.) http://www.stat.columbia.edu/~gelman/book/ http://www.stat.columbia.edu/~gelman/book/ and Probability Theory and tbe Logic of Science by Ed Jaynes: https://www.amazon.com/dp/0521592712/ref=cm_sw_r_em_apa_i_v35QEb3FEYSGM https://www.amazon.com/dp/0521592712/ref=cm_sw_r_em_apa_i_v3... The former is a much recommended book since it's very comprehensive and builds everything from the ground up and was the basis for the entire course. The latter is a beast of it's own and we simply covered what was effectively the first chapter as part of the course.