6 ms·
Interview on ”Bayesian Statistics the Fun Way”
- mself 7y agoI love the idea of making an approachable version of Ed Jaynes’s classic.
- RobertRoberts 7y agoI got in an argument with a friend (a mechanical/electrical engineer) who knew about bayesian statistics. My other friend, a PhD in statistics, whom I had many discussions about because both personal interest and work interests, had supplied me with my modicum of statics knowledge. My engineer friend called my PhD friend a "frequentist", like it was a dirty word, despite only having one, maybe two, classes in college about bayesian math/statistics/whatever (my ignorance). This quote jumped out at me in the article: "I wanted to write a book on Bayesian statistics that really anyone could pick up and use to gain real intuitions for how to think statistically and solve real problems using statistics." In the context of the statement, it sounds like he is claimin any non-bayesian statistics is useless (or less valuable/reliable at best) than other forms of statistical analysis?
- jmvoodoo 7y agoHaving known Will when I lived in Reno I'm certain your focus should be on "anyone could pick up and use" and not any statement about the usefulness of other approaches. The Will I know is fundamentally about teaching things in very easy to understand ways, and curious about all approaches to solving a problem.
- jdreaver 7y agoThat's not how I'm reading that quote at all. Saying Bayesian stats can solve real problems doesn't imply frequentist stats can't.
- brylie 7y agoIt just reads to me like he wants to make statistics accessible to a wide audience.
- arafa 7y agoAs someone who uses statistics all the time at work, I sympathize so much with this article and greatly enjoyed it. Every time I try to introduce a Bayesian prior, coworkers either look at me like I'm crazy (because they've never heard of or used Bayesian stats) or like I've suddenly gone soft and introduced a bunch of nebulous, touchy-feely context into the objective truth (if they're dedicated frequentists). Then we promptly switch back to p-values of .05, a lot of the time not even bothering with a statistical power calculation. I've had better success with introducing power, though. I suspect that's because we can fit it into the existing frequentist framework.
- hotdog999 7y ago> like I've suddenly gone soft and introduced a bunch of nebulous, touchy-feely context into the objective truth This drives me nuts. If you haven't, check out the paper "Beyond subjective and objective in statistics" by Gelman and Hennig (2017). Right at the beginning they make the point that any analysis includes external information in many ways, such as adjusting variables for imbalance, how we deal with outliers, regularization, etc. Especially if you're doing any sort of causal inference, you're usually making strong assumptions before estimating your model, even just in terms of which variables are included and how they're connected. The idea that priors are somehow ruining an "objective" model is just absurd to me. You're already making so many other decisions about your model that will affect estimates and your interpretation of them. Priors seem like another perfectly reasonable decision to have to make as well, with the benefit of getting results that I think in general are must more easily understood by a lay audience. (E.g., I don't think I've ever encountered someone not on my data science team that actually understands what a p-value is. But people are much better at understanding when I say, there's an X percent chance that there is a positive effect here.)
- dhfromkorea 7y ago> The idea that priors are somehow ruining an "objective" model is just absurd to me. I think some caution can be justified to a certain extent (not the blind "emotional" objections). When establishing priors in a low data regime, one must necessarily be careful. It's a knob whose mass can change a lot in the inference conclusion. That said, if we trust our belief about the region the available data do not inform us well of, why not utilize our domain knowledge/belief?
- gdy 7y agoCan anyone recommend a 'Bayesian statistics the hard way' book?
- j7ake 7y agoProbability Theory: The Logic of Science by Edwin Jaynes
- loup-vaillant 7y agohttp://www.med.mcgill.ca/epidemiology/hanley/bios601/GaussianModel/JaynesProbabilityTheory.pdf http://www.med.mcgill.ca/epidemiology/hanley/bios601/Gaussia... But really, the first two chapters aren't that hard.
- ploika 7y agoBayesian Data Analysis by Andrew Gelman http://www.stat.columbia.edu/~gelman/book/ http://www.stat.columbia.edu/~gelman/book/
- dhfromkorea 7y agoone vote for BDA. For programmers who learn better by implementing things, this book [1] is also good: [1]: https://www.amazon.com/Bayesian-Methods-Hackers-Probabilistic-Addison-Wesley/dp/0133902838 https://www.amazon.com/Bayesian-Methods-Hackers-Probabilisti...
- montecarl 7y agoParts of that book are available online[1] for free. If not for that book I would never have understood how to apply Bayesian stats to problems that interested me. [1] http://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/ http://camdavidsonpilon.github.io/Probabilistic-Programming-...
- datasciencetext 7y agoStatistical Rethinking: A Bayesian Course with Examples in R and Stan is also considered pretty good.
- samch93 7y agoAs someone who has a master's degree in statistics and often uses Bayesian statistics, I think we should not focus on whether one is a Bayesian or a frequentist, but rather be pragmatic and take the most practical approach to solving a statistical problem. Moreover, I think statistical education should start with frequentist concepts and then extend them to the Bayesian framework since the likelihood plays also a major role in obtaining the posterior distribution. In my opinion, this progression is much more natural than starting fully Bayesian.
- techwizrd 7y agoI agree with this approach, and this is roughly the approach my own Statistics master's degree takes as well. It can be challenging to understand the finer points of likelihoods and posteriors (and the how to choose a prior) without serious mathematics that you're unlikely to have upon entering a graduate statistics degree. Starting with applied probability and applied statistics (incl. regression, ANOVA, GLMs) allow you to solve problems and feel useful and engaged before being thrown into the mathematical rigor required of Bayesian statistics.
- gbrown 7y agoI agree, although I respect those who look for deeper justification for the methods we use. Bayesian statistics/decision theory does have axiomatic foundations after all.
- davidmanheim 7y agoSo does frequentist stats - they are just different axiomatic foundations and assumptions.
- gbrown 7y agoI'm less familiar with them - I've certainly seen many plausible frequentist arguments, but I've never been exposed to any unifying framework which would require that one make decisions based on type-1 error rate controlling hypothesis tests. That's not to say such foundations don't exist, I'm just happy being a philosophical Bayesian who sometimes does frequentist or algorithmic/ML things for practical reasons.
- bryanrasmussen 7y ago>First of all, p-values are not the way sane people answer questions I think they are pretty close to the way sane people answer some kinds of questions.
- MadWombat 7y ago"For coin tosses both schools of thought work pretty well" How many coin tosses in a row have to land heads before a frequentist decides that the coin is unfair?
- hirundo 7y agoAre there conditions in which a Bayesian reasoner is obliged to make racist decisions? Say you're a Bayesian infant who has never met anyone but Mom, who has blue skin, and has always been very nice to you. You then meet 100 other people of various colors, 97 of who are also nice to you. The other 3 people were mean to you, and they all have purple skin, and only they have purple skin. Person 101 approaches you, and they have purple skin. As a Bayesian baby you start to cry, anticipating meanness. According to the idea that discrimination by immutable characteristics is bad, and that we should judge people as individuals, this Bayesian choice to cry is a symptom of a racist mindset. Isn't it? In what circumstances should Bayesian priors _not_ be used for decision making, in deference to wider principles of justice?
- nightski 7y agoThis seems like pretty flawed reasoning. What you are describing is not a prior but a posterior - the distribution after the observed data has been taken into account. If anything the prior can help make you less racist by incorporating the knowledge that immutable characteristics are not good indicators of danger/not danger. The thing is though, even if you are told race doesn't matter through a prior, if you observe a strong correlation over many instances it's going to be hard to ignore that regardless of your prior (what you are told). While it may not be a causal relationship, it may still be a good predictor.
- sdinsn 7y agoA purpose of statistics is to make generalizations about a population. The solution to your problem is to have a larger sample size.
- gdy 7y ago"anticipating meanness" And that would be wise.
- davidmanheim 7y agoWhat does this have to do with Bayes? A frequentist assessing probabilities to make decisions about how to respond is in, if anything, a far worse position. The Bayesian would ideally use priors on groups and cross-group correlations to note that the weak evidence that purple -> mean barely shifts from their priors, and the inter-group mean shows that they are likely to be nice, unless your prior is that different colors have nearly independent probabilities of being nice or mean.
- RosanaAnaDana 7y agoI think the swedish fish approach is a particularly fun way: https://www.youtube.com/watch?v=3OJEae7Qb_o https://www.youtube.com/watch?v=3OJEae7Qb_o