5 ms·

Social Psychology has one of the worst reproducibility problems in all of academia. Claims like this make is easy to see why. Simple scientific inquiry into thi

Social Psychology has one of the worst reproducibility problems in all of academia. Claims like this make is easy to see why. Simple scientific inquiry into this claim leads me to the question, “Okay, but why? That women potentially harbor worse feelings of their ex partners on average than men is one of the most interesting but useless things I’ve heard. The confounding explanations for this type of research are mind boggling. Does this research hold cross-culturally, are there any other psychological variables that explain this phenomena? Is it possible that the current nature of modern relationships is a confounding variable and sex has little to do with it? These are the more interesting questions to answer than whatever question this dubious result is trying to answer.

lwhi 7y ago
I agree. It seems to be to be a case of 'sounds like it might be right, lets see how far we can run with it'. Ultimately, I can only see a few benefits of this line of enquiry. E.g. "Honey, I'm sorry - but it's not really my fault, it's just my genetic make-up .." Ultimately, while we may be descended from lizards, rationalising behaviours based on our physiological origins might not be that useful. Especially when such sweeping statements are made about a _vast_ group of people who—when looked at in terms of their other attributes—vary enormously.
- npo9 7y ago
  It might be useful for providing therapy to understand the differences in how men and women see things.
  - lwhi 7y ago
    Perhaps, but personally I think we all vary a great deal. Any truly effective therapy needs to be based on the individual. Generalisations can be lazy - and in a pragmatic sense, only really seem useful in very specific circumstances.
- FabHK 7y ago
  People need to learn that statements about differences in group averages are not "sweeping" statements. Almost all psychological measurements vary much more within groups than between groups, and reports of differences in group averages tell us basically nothing about any individual. Yet, the effects might appear in contexts involving many individuals.
WhompingWindows 7y ago
In the first paragraph you're asking for causal explanation. Then in the second paragraph you're dishing out a bunch of potential causal explanations as confounders. Some researchers would deem it over-reaching to dish out causal explanations willy nilly. Most of those questions should be relegated to follow-up studies or a BRIEF mention in the discussion section. It's perfectly okay science to simply observe a phenomenon. If they did try to provide causal explanations as you say, they'd then be open to being lambasted for overreaching based on shaky social science research.
- im3w1l 7y ago
  If you don't have the causal explanations then you cannot generalize beyond the population you sampled. What does that mean? That the correct conclusion to draw is very narrow: Men View Their Ex-Partners More Favorably Than Women Do [in Graz, Austria, in the year 2019]
  - soneca 7y ago
    That's correct, still, it is ok. That's why you don't make bold, generalized claims based on one study.
  - zymhan 7y ago
    Not every bit of research results in a simple conclusion. That doesn't mean it's irrelevant.
  - moate 7y ago
    >>Men [in their 20s] View Their Ex-Partners More Favorably Than Women[in their 20s] Do [in Graz, Austria, in the year 2019] Fixed that for you. It's even more narrow in scope.
    - vanderZwan 7y ago
      For the medical equivalent, see: https://twitter.com/justsaysinmice https://twitter.com/justsaysinmice
  - 7y ago
markstos 7y ago
Remember Christine Blasey Ford's relationship with Brett Kavanaugh? He recalls their relationship more neutrally and she recalls it more negatively.
- deleted 7y ago
  [deleted]
vanderZwan 7y ago
The worst part is that this doesn't stop people from concluding things that they can't, especially if they have an axe to grind. I can easily imagine someone reading the headline and interpreting it as either proof of women being less reasonable, or proof of men being worse ex-partners, depending on which side of the sexist coin they land.
dahart 7y ago
> The confounding explanations for this type of research are mind boggling. Yes! This is very true, your questions are good questions to ask of any study of genders & attitudes. It’s even more important to ask of studies that try to factor out cultural issues, because confounding explanations are extremely difficult to remove. > Social Psychology has one of the worst reproducibility problems in all of academia. This is true, but it’s not scientifically fair to imply this study isn’t reproducible just because others weren’t. > Simple scientific inquiry into this claim leads me to the question, “Okay, but why? Science doesn’t necessarily answer why. Physics has one of the better reproducibility rates, and we’d love to know why, but we don’t truly know why yet. We don’t know why gravity exists, we know it’s a force that’s there, and we can predict what it’ll do, we know it happens in proportion to mass and inverse squared distance, but why? Science hasn’t answered that.
- longtom 7y ago
  > we’d love to know why, but we don’t truly know why yet This isn't much of a mystery. It is simply easier to control for external factors in physics, making it easier to isolate the causal links of interest. In the social sciences, you have layers upon layers of biological, psychological and then cultural complexities. Secondly, physicists have higher IQ as greater economic incentives attract brighter people. Lastly, physics cannot be abused politically as easily.
- K0SM0S 7y ago
  Just a few remarks, notwithstanding your/my opinions. Physics has not "one of the better reproducibility rates" but 100%; per the Scientific Method any valid theory must: - be reproducible (verifiable empirically) - have predictive power - not negate existing theories This constitutes 'acceptable proof'. It yields a 'compound effect' to theoretical research: for instance, Newton's Laws were never wrong, just narrower in scope than Einstein's. No future valid theory will ever prove any of these two wrong, it can only generalize more, because we know these two to be true. True is true forever, barring a fundamental change in this universe itself. So we keep the theoretical foundations forever, they're solid. Their interpretation, however, may change as we widen the scope, and that is the 'human' part in science, the art of it, to produce valid representations linking the math with the experience of reality. That changes a lot, in time, and even from person to person. The 'why' things are is another matter entirely and not a concern for physics nor science in general; we search for the root causes to explain 'how' things work the way they do. We stop there in theory for beyond is mere interpretation, opinion, belief. Now this would exclude all social sciences and biology and countless others from the realm of 'science' and that's not fair; these fields yield very usable results despite the nigh impossible search for pure theory. I like the 19-20th century view that 'empirical sciences' gain value when we try to the best of our ability to apply the Scientific Method and mathematical tools to account for fuzziness. Deep learning is certainly a formidable upgrade to the tooling of empirical sciences, brute-forcing 'how' things work for lack of a better truth than retrospective. Just don't compare it to e.g. physics, don't ask for predictability (as you wouldn't ask a fish to fly), it's pointless and only serves to undermine the value of empirical sciences, while overemphasizing that of theory. Because you know, in theory, practice and theory are the same, but in practice they're not... ¯\_(ツ)_/¯
coldtea 7y ago
>Simple scientific inquiry into this claim leads me to the question, “Okay, but why?" Well, the "why" is relevant to whether the statement is true. So it's not like you need to answer the "why" to be able to see if something is the case or not. >That women potentially harbor worse feelings of their ex partners on average than men is one of the most interesting but useless things I’ve heard. Useless because you can't think of a specific utility? Not all discoveries have some immediate utility -- but they can pile up, and lead to useful discoveries. Baby steps and all... But even this information alone, could be useful for social workers, con-men, marriage counselors, advertisers, and other categories, even without a why attached...
datashow 7y ago
"Social Psychology has one of the worst reproducibility problems in all of academia." Not to be defending social psychology in any way, but I have to point out this assessment is seriously misled. Social psychology gets bad impression on reproducibility, because there are tons of research on the reproducibility of social psychology studies. The statement makes it sound like there are reproducibility examinations across all fields in academia. In reality, it is rare. Most fields in academia do not get this kind of examination at all. Also, reproducibility examinations mostly are focused on experimental research. However, this study is observational research, which is a totally different kind of social science study.
zwieback 7y ago
Reproducibility is terrible in this field but for this particular study they explain their statistical methods as well as provide links to the data. You could do your own analysis if you wish. The recruits were found via email in an Austrian University so I don't think the authors would make any cross-cultural claims.

lwhi 7y ago

I agree. It seems to be to be a case of 'sounds like it might be right, lets see how far we can run with it'. Ultimately, I can only see a few benefits of this line of enquiry. E.g. "Honey, I'm sorry - but it's not really my fault, it's just my genetic make-up .." Ultimately, while we may be descended from lizards, rationalising behaviours based on our physiological origins might not be that useful. Especially when such sweeping statements are made about a _vast_ group of people who—when looked at in terms of their other attributes—vary enormously.

npo9 7y ago
It might be useful for providing therapy to understand the differences in how men and women see things.
- lwhi 7y ago
  Perhaps, but personally I think we all vary a great deal. Any truly effective therapy needs to be based on the individual. Generalisations can be lazy - and in a pragmatic sense, only really seem useful in very specific circumstances.
FabHK 7y ago
People need to learn that statements about differences in group averages are not "sweeping" statements. Almost all psychological measurements vary much more within groups than between groups, and reports of differences in group averages tell us basically nothing about any individual. Yet, the effects might appear in contexts involving many individuals.

WhompingWindows 7y ago

In the first paragraph you're asking for causal explanation. Then in the second paragraph you're dishing out a bunch of potential causal explanations as confounders. Some researchers would deem it over-reaching to dish out causal explanations willy nilly. Most of those questions should be relegated to follow-up studies or a BRIEF mention in the discussion section. It's perfectly okay science to simply observe a phenomenon. If they did try to provide causal explanations as you say, they'd then be open to being lambasted for overreaching based on shaky social science research.

im3w1l 7y ago
If you don't have the causal explanations then you cannot generalize beyond the population you sampled. What does that mean? That the correct conclusion to draw is very narrow: Men View Their Ex-Partners More Favorably Than Women Do [in Graz, Austria, in the year 2019]
- soneca 7y ago
  That's correct, still, it is ok. That's why you don't make bold, generalized claims based on one study.
- zymhan 7y ago
  Not every bit of research results in a simple conclusion. That doesn't mean it's irrelevant.
- moate 7y ago
  >>Men [in their 20s] View Their Ex-Partners More Favorably Than Women[in their 20s] Do [in Graz, Austria, in the year 2019] Fixed that for you. It's even more narrow in scope.
  - vanderZwan 7y ago
    For the medical equivalent, see: https://twitter.com/justsaysinmice https://twitter.com/justsaysinmice
- 7y ago

markstos 7y ago

Remember Christine Blasey Ford's relationship with Brett Kavanaugh? He recalls their relationship more neutrally and she recalls it more negatively.

deleted 7y ago
[deleted]

vanderZwan 7y ago

The worst part is that this doesn't stop people from concluding things that they can't, especially if they have an axe to grind. I can easily imagine someone reading the headline and interpreting it as either proof of women being less reasonable, or proof of men being worse ex-partners, depending on which side of the sexist coin they land.

dahart 7y ago

> The confounding explanations for this type of research are mind boggling. Yes! This is very true, your questions are good questions to ask of any study of genders & attitudes. It’s even more important to ask of studies that try to factor out cultural issues, because confounding explanations are extremely difficult to remove. > Social Psychology has one of the worst reproducibility problems in all of academia. This is true, but it’s not scientifically fair to imply this study isn’t reproducible just because others weren’t. > Simple scientific inquiry into this claim leads me to the question, “Okay, but why? Science doesn’t necessarily answer why. Physics has one of the better reproducibility rates, and we’d love to know why, but we don’t truly know why yet. We don’t know why gravity exists, we know it’s a force that’s there, and we can predict what it’ll do, we know it happens in proportion to mass and inverse squared distance, but why? Science hasn’t answered that.

longtom 7y ago
> we’d love to know why, but we don’t truly know why yet This isn't much of a mystery. It is simply easier to control for external factors in physics, making it easier to isolate the causal links of interest. In the social sciences, you have layers upon layers of biological, psychological and then cultural complexities. Secondly, physicists have higher IQ as greater economic incentives attract brighter people. Lastly, physics cannot be abused politically as easily.
K0SM0S 7y ago
Just a few remarks, notwithstanding your/my opinions. Physics has not "one of the better reproducibility rates" but 100%; per the Scientific Method any valid theory must: - be reproducible (verifiable empirically) - have predictive power - not negate existing theories This constitutes 'acceptable proof'. It yields a 'compound effect' to theoretical research: for instance, Newton's Laws were never wrong, just narrower in scope than Einstein's. No future valid theory will ever prove any of these two wrong, it can only generalize more, because we know these two to be true. True is true forever, barring a fundamental change in this universe itself. So we keep the theoretical foundations forever, they're solid. Their interpretation, however, may change as we widen the scope, and that is the 'human' part in science, the art of it, to produce valid representations linking the math with the experience of reality. That changes a lot, in time, and even from person to person. The 'why' things are is another matter entirely and not a concern for physics nor science in general; we search for the root causes to explain 'how' things work the way they do. We stop there in theory for beyond is mere interpretation, opinion, belief. Now this would exclude all social sciences and biology and countless others from the realm of 'science' and that's not fair; these fields yield very usable results despite the nigh impossible search for pure theory. I like the 19-20th century view that 'empirical sciences' gain value when we try to the best of our ability to apply the Scientific Method and mathematical tools to account for fuzziness. Deep learning is certainly a formidable upgrade to the tooling of empirical sciences, brute-forcing 'how' things work for lack of a better truth than retrospective. Just don't compare it to e.g. physics, don't ask for predictability (as you wouldn't ask a fish to fly), it's pointless and only serves to undermine the value of empirical sciences, while overemphasizing that of theory. Because you know, in theory, practice and theory are the same, but in practice they're not... ¯\_(ツ)_/¯

coldtea 7y ago

>Simple scientific inquiry into this claim leads me to the question, “Okay, but why?" Well, the "why" is relevant to whether the statement is true. So it's not like you need to answer the "why" to be able to see if something is the case or not. >That women potentially harbor worse feelings of their ex partners on average than men is one of the most interesting but useless things I’ve heard. Useless because you can't think of a specific utility? Not all discoveries have some immediate utility -- but they can pile up, and lead to useful discoveries. Baby steps and all... But even this information alone, could be useful for social workers, con-men, marriage counselors, advertisers, and other categories, even without a why attached...

datashow 7y ago

"Social Psychology has one of the worst reproducibility problems in all of academia." Not to be defending social psychology in any way, but I have to point out this assessment is seriously misled. Social psychology gets bad impression on reproducibility, because there are tons of research on the reproducibility of social psychology studies. The statement makes it sound like there are reproducibility examinations across all fields in academia. In reality, it is rare. Most fields in academia do not get this kind of examination at all. Also, reproducibility examinations mostly are focused on experimental research. However, this study is observational research, which is a totally different kind of social science study.

zwieback 7y ago

Reproducibility is terrible in this field but for this particular study they explain their statistical methods as well as provide links to the data. You could do your own analysis if you wish. The recruits were found via email in an Austrian University so I don't think the authors would make any cross-cultural claims.