6 ms·

Although many of the results in Fast and Slow didn't hold up, Kahneman was always refreshingly open and honest about that, and keen to identify the limits of kn

Although many of the results in Fast and Slow didn't hold up, Kahneman was always refreshingly open and honest about that, and keen to identify the limits of knowledge. Which surely is one of the best things you can say about a scientist.

mxs6 2y ago
I was fortunate enough to take a cognitive psych grad seminar from him in the 90s, co-taught by his wife, Anne Treisman. He always seemed given to thinking a little more deeply in the moment than most people do. One half-joking comment he made about science in the real world vs some idealized notion of it has always stuck with me. In a discussion about whether the results of some paper conflicted with some model or theory of cognition, he mused that scientific progress in psychology (and other non-hard sciences) was really about embarrassing rivals with competing models. No high-level model was ever stated precisely enough to rule out some particular finding; you could always tweak your theory a little to accommodate it. It's just that at some point, you might be too embarrassed to do so.
- rvbissell 2y ago
  Let's call those tweaks epi(stemological)-cycles.
- trash_cat 2y ago
  I did a Cog Sci bachelor and this is the conclusion I made I finished my studies. A lot of Thinking Fast and Slow are summaries of research done in the field over decades. In particular, biases and intuition as internalized knowledge/expertise. Sure you can complain about replication issues but this is the best model when it comes to minds that I know of.
- ignoramous 2y ago
  > He always seemed given to thinking a little more deeply in the moment than most people do. "As soon as you present a problem to me, I have some ready-made answer. Those ready-made answers get in the way of clear thinking, and we can’t help but have them." – Daniel Kahneman
resource_waste 2y ago
>was always refreshingly open and honest about that, His work got torn to shreds with science, what did you expect him to do? It really should be embarrassing rather than acceptable. It doesnt matter how honest you are AFTER you are caught. Between the replication crisis and his name on some bad papers, the guy seriously didn't care about correctness as much as interesting-faux-science.
- Tarq0n 2y ago
  There's lots of scientists that deflect and deny falsifications.
- elbear 2y ago
  Embarrassed because science worked as expected?
  - resource_waste 2y ago
    Doesn't science require replication? He wrote books based on un-replicated studies. Further, the confidence he extolled about his now debunked ideas make him a charlatan. This person was a bad scientist. If we esteem people who don't check their data and influence millions of people with falsities, we are going to create a society with low trust. Just look at this thread, the man lost respect among the people in the know. There are a few people clinging onto 'well just because its not true, doesnt mean I didn't find it interesting". I'm not sure what we get out of promoting anti-science scientists.
    - trash_cat 2y ago
      Can you be specific what ideas of his aren't scientific? It's true that science requires replication but he deals with models but perhaps uses bad studies to support it. It's like saying he should replicate the theory of evolution.
      - smcin 2y ago
        If you google "Which Kahneman claims were wrong", here are the first two of 4.97m hits" https://slate.com/technology/2016/12/kahneman-and-tversky-researched-the-science-of-error-and-still-made-errors.html https://slate.com/technology/2016/12/kahneman-and-tversky-re... https://replicationindex.com/category/thinking-fast-and-slow/ https://replicationindex.com/category/thinking-fast-and-slow... > "Table 1 shows the number of results that were available and the R-Index for chapters that mentioned empirical results. The chapters vary dramatically in terms of the number of studies that are presented (Table 1). The number of results ranges from 2 for chapters 14 and 16 to 55 for Chapter 5. For small sets of studies, the R-Index may not be very reliable, but it is all we have unless we do a careful analysis of each effect and replication studies. > Chapter 4 is the priming chapter that we carefully analyzed (Schimmack, Heene, & Kesavan, 2017). Table 1 shows that Chapter 4 is the worst chapter with an R-Index of 19. An R-Index below 50 implies that there is a less than 50% chance that a result will replicate. Tversky and Kahneman (1971) themselves warned against studies that provide so little evidence for a hypothesis. A 50% probability of answering multiple choice questions correctly is also used to fail students. So, we decided to give chapters with an R-Index below 50 a failing grade. Other chapters with failing grades are Chapter 3, 6, 7, 11, 14, 16. Chapter 24 has the highest highest score (80, which is an A- in the Canadian grading scheme), but there are only 8 results.
YeBanKo 2y ago
Do you have resources on what specific results did not hold up?
tmalsburg2 2y ago
Wouldn’t the best thing have been to retract the book, or at least to published a revised edition?
kashyapc 2y ago
Careful there. By saying "many of the results", you're hand-waving most of the book. The book covers over several decades of his collaborative research with Amos Tversky. Most of the "underpowered studies" are in the priming-related chapter, called "The Assoviative Machine". The rest of the book is still worth a careful read. (I had to make this same correction here several years ago. I didn't look up the comment to link it here.)

mxs6 2y ago

I was fortunate enough to take a cognitive psych grad seminar from him in the 90s, co-taught by his wife, Anne Treisman. He always seemed given to thinking a little more deeply in the moment than most people do. One half-joking comment he made about science in the real world vs some idealized notion of it has always stuck with me. In a discussion about whether the results of some paper conflicted with some model or theory of cognition, he mused that scientific progress in psychology (and other non-hard sciences) was really about embarrassing rivals with competing models. No high-level model was ever stated precisely enough to rule out some particular finding; you could always tweak your theory a little to accommodate it. It's just that at some point, you might be too embarrassed to do so.

rvbissell 2y ago
Let's call those tweaks epi(stemological)-cycles.
trash_cat 2y ago
I did a Cog Sci bachelor and this is the conclusion I made I finished my studies. A lot of Thinking Fast and Slow are summaries of research done in the field over decades. In particular, biases and intuition as internalized knowledge/expertise. Sure you can complain about replication issues but this is the best model when it comes to minds that I know of.
ignoramous 2y ago
> He always seemed given to thinking a little more deeply in the moment than most people do. "As soon as you present a problem to me, I have some ready-made answer. Those ready-made answers get in the way of clear thinking, and we can’t help but have them." – Daniel Kahneman

resource_waste 2y ago

>was always refreshingly open and honest about that, His work got torn to shreds with science, what did you expect him to do? It really should be embarrassing rather than acceptable. It doesnt matter how honest you are AFTER you are caught. Between the replication crisis and his name on some bad papers, the guy seriously didn't care about correctness as much as interesting-faux-science.

Tarq0n 2y ago
There's lots of scientists that deflect and deny falsifications.
elbear 2y ago
Embarrassed because science worked as expected?
- resource_waste 2y ago
  Doesn't science require replication? He wrote books based on un-replicated studies. Further, the confidence he extolled about his now debunked ideas make him a charlatan. This person was a bad scientist. If we esteem people who don't check their data and influence millions of people with falsities, we are going to create a society with low trust. Just look at this thread, the man lost respect among the people in the know. There are a few people clinging onto 'well just because its not true, doesnt mean I didn't find it interesting". I'm not sure what we get out of promoting anti-science scientists.
  - trash_cat 2y ago
    Can you be specific what ideas of his aren't scientific? It's true that science requires replication but he deals with models but perhaps uses bad studies to support it. It's like saying he should replicate the theory of evolution.
    - smcin 2y ago
      If you google "Which Kahneman claims were wrong", here are the first two of 4.97m hits" https://slate.com/technology/2016/12/kahneman-and-tversky-researched-the-science-of-error-and-still-made-errors.html https://slate.com/technology/2016/12/kahneman-and-tversky-re... https://replicationindex.com/category/thinking-fast-and-slow/ https://replicationindex.com/category/thinking-fast-and-slow... > "Table 1 shows the number of results that were available and the R-Index for chapters that mentioned empirical results. The chapters vary dramatically in terms of the number of studies that are presented (Table 1). The number of results ranges from 2 for chapters 14 and 16 to 55 for Chapter 5. For small sets of studies, the R-Index may not be very reliable, but it is all we have unless we do a careful analysis of each effect and replication studies. > Chapter 4 is the priming chapter that we carefully analyzed (Schimmack, Heene, & Kesavan, 2017). Table 1 shows that Chapter 4 is the worst chapter with an R-Index of 19. An R-Index below 50 implies that there is a less than 50% chance that a result will replicate. Tversky and Kahneman (1971) themselves warned against studies that provide so little evidence for a hypothesis. A 50% probability of answering multiple choice questions correctly is also used to fail students. So, we decided to give chapters with an R-Index below 50 a failing grade. Other chapters with failing grades are Chapter 3, 6, 7, 11, 14, 16. Chapter 24 has the highest highest score (80, which is an A- in the Canadian grading scheme), but there are only 8 results.

YeBanKo 2y ago

Do you have resources on what specific results did not hold up?

tmalsburg2 2y ago

Wouldn’t the best thing have been to retract the book, or at least to published a revised edition?

kashyapc 2y ago

Careful there. By saying "many of the results", you're hand-waving most of the book. The book covers over several decades of his collaborative research with Amos Tversky. Most of the "underpowered studies" are in the priming-related chapter, called "The Assoviative Machine". The rest of the book is still worth a careful read. (I had to make this same correction here several years ago. I didn't look up the comment to link it here.)