5 ms·
Is there a gentler intro to this topic?
by RickHull 5mo ago
Is there a gentler intro to this topic?
- jey 5mo agoTry the textbook Elements of Information Theory by Cover and Thomas (2006)
- srean 5mo agoI wouldn't say it's gentle but it certainly is a great book. Great exercise problems. Some of the proofs are so elegantly done, especially the way calculus of variations is avoided. David Mackay's book hand holds a little more than Cover and Thomas, although it's remit is more than just information theory.
- mememememememo 5mo agoFound an excerpt online. Seems like a gem of a book.
- chermi 5mo agoI recommend every where I get a chance : mackay's book. https://www.inference.org.uk/mackay/itila/book.html https://www.inference.org.uk/mackay/itila/book.html
- Igrom 5mo agoWhich chapters have you found the most enlightening or useful? (off-topic: here's my own "recommend everywhere" book, "Attacking Faulty Reasoning" by T. Edward Damer, https://en.wikipedia.org/wiki/Attacking_Faulty_Reasoning https://en.wikipedia.org/wiki/Attacking_Faulty_Reasoning).
- chermi 5mo agoSorry just saw this. The first 50 or so pages are gold, not under a chapter proper. Then most of part IV for non-physicists for exposure to the statistical mechanician's worldview, especially chapters 27-33. I'm not an expert on sections 1-3, so I can't make very high value claims on their relative value. But, everytime I did look into topics covered in those chapters, I found clarity in explanation and the "how" of things by coming back to the book. The entire neural networks section contains tons of nuggets that proved both prescient and (mostly) timeless.
- krackers 5mo agoSee this video, beautiful explanation that doesn't already assume familiarity with entropy https://www.youtube.com/watch?v=ErfnhcEV1O8 https://www.youtube.com/watch?v=ErfnhcEV1O8