9 ms·

This essay is missing the words “cause” and “causal”. There is a difference between discovering causes and fitting curves. The search for causes guides the desi

This essay is missing the words “cause” and “causal”. There is a difference between discovering causes and fitting curves. The search for causes guides the design of experiments, and with luck, the derivation of formulae that describe the causes. Norvig seems to be confusing the map (data, models) for the territory (causal reality).

gsf_emergency_6 9mo ago
A related* essay (2010) by a statistician on the goals of statistical modelling that I've been procrastinating on: https://www.stat.berkeley.edu/~aldous/157/Papers/shmueli.pdf https://www.stat.berkeley.edu/~aldous/157/Papers/shmueli.pdf To Explain Or To Predict? Nice quote We note that the practice in applied research of concluding that a model with a higher predictive validity is “truer,” is not a valid inference. This paper shows that a parsimonious but less true model can have a higher predictive validity than a truer but less parsimonious model. Hagerty+Srinivasan (1991) *like TFA it's a sorta review of Breiman
- 0928374082 9mo ago
  is it more than a commentary on overfitting to the tune of "with enough epicycles you can make the elephant wiggle its trunk"?
  - gsf_emergency_6 9mo ago
    If you are referring to Hagerty+Srinivasan: They certainly didn't think that a better fit => "truer". They used the term "truer" to describe a model that more accurately captures the underlying causal structure or "true" relationship between variables in a population. As for the paper I linked, I still haven't read it closely enough to confirm that D-Machine's comment below is a good dismissal. I'm inclined to think it's more like "interpolating vs extrapolating"
tripletao 9mo ago
This essay frequently uses the word "insight", and its primary topic is whether an empirically fitted statistical model can provide that (with Norvig arguing for yes, in my opinion convincingly). How does that differ from your concept of a "cause"?
- musicale 9mo ago
  > I agree that it can be difficult to make sense of a model containing billions of parameters. Certainly a human can't understand such a model by inspecting the values of each parameter individually. But one can gain insight by examing (sic) the properties of the model—where it succeeds and fails, how well it learns as a function of data, etc. Unfortunately, studying the behavior of a system doesn't necessarily provide insight into why it behaves that way; it may not even provide a good predictive model.
  - tripletao 9mo ago
    Norvig's textbook surely appears on the bookshelf of researchers including those building current top LLMs. So it's odd to say that such an approach "may not even provide a good predictive model". As of today, it is unquestionably the best known predictive model for natural language, by huge margin. I don't think that's for lack of trying, with billions of dollars or more at stake. Whether that model provides "insight" (or a "cause"; I still don't know if that's supposed to mean something different) is a deeper question, and e.g. the topic of countless papers trying to make sense of LLM activations. I don't think the answer is obvious, but I found Norvig's discussion to be thoughtful. I'm surprised to see it viewed so negatively here, dismissed with no engagement with his specific arguments and examples.
    - atomicnature 9mo ago
      You can look into Judea Pearl's definitions of causality for more information. Pearl defines a ladder of causation: 1. Seeing (association) 2. Doing (intervention) 3. Imagining (counterfactuals) In his view - most ML algos are at level 1 - they look at data and draw associations, and "agents" have started some steps in level 2 - doing. The smartest of humans operate mostly in level (3) of abstractions - where they see things, gain experience, and later build up a "strong causal model" of the world and become capable of answering "what if" questions.
D-Machine 9mo ago
I had this exact reaction, no discussion of "causal modeling" makes the whole thing seem horribly out of touch with the real issues here. You can have explanatory and predictive models that are causal models, or explanatory and predictive models that are non-causal, and that this the actual issue, not "explanation" vs. "prediction", which is not a tight enough distinction.

gsf_emergency_6 9mo ago

A related* essay (2010) by a statistician on the goals of statistical modelling that I've been procrastinating on: https://www.stat.berkeley.edu/~aldous/157/Papers/shmueli.pdf https://www.stat.berkeley.edu/~aldous/157/Papers/shmueli.pdf To Explain Or To Predict? Nice quote We note that the practice in applied research of concluding that a model with a higher predictive validity is “truer,” is not a valid inference. This paper shows that a parsimonious but less true model can have a higher predictive validity than a truer but less parsimonious model. Hagerty+Srinivasan (1991) *like TFA it's a sorta review of Breiman

0928374082 9mo ago
is it more than a commentary on overfitting to the tune of "with enough epicycles you can make the elephant wiggle its trunk"?
- gsf_emergency_6 9mo ago
  If you are referring to Hagerty+Srinivasan: They certainly didn't think that a better fit => "truer". They used the term "truer" to describe a model that more accurately captures the underlying causal structure or "true" relationship between variables in a population. As for the paper I linked, I still haven't read it closely enough to confirm that D-Machine's comment below is a good dismissal. I'm inclined to think it's more like "interpolating vs extrapolating"

tripletao 9mo ago

This essay frequently uses the word "insight", and its primary topic is whether an empirically fitted statistical model can provide that (with Norvig arguing for yes, in my opinion convincingly). How does that differ from your concept of a "cause"?

musicale 9mo ago
> I agree that it can be difficult to make sense of a model containing billions of parameters. Certainly a human can't understand such a model by inspecting the values of each parameter individually. But one can gain insight by examing (sic) the properties of the model—where it succeeds and fails, how well it learns as a function of data, etc. Unfortunately, studying the behavior of a system doesn't necessarily provide insight into why it behaves that way; it may not even provide a good predictive model.
- tripletao 9mo ago
  Norvig's textbook surely appears on the bookshelf of researchers including those building current top LLMs. So it's odd to say that such an approach "may not even provide a good predictive model". As of today, it is unquestionably the best known predictive model for natural language, by huge margin. I don't think that's for lack of trying, with billions of dollars or more at stake. Whether that model provides "insight" (or a "cause"; I still don't know if that's supposed to mean something different) is a deeper question, and e.g. the topic of countless papers trying to make sense of LLM activations. I don't think the answer is obvious, but I found Norvig's discussion to be thoughtful. I'm surprised to see it viewed so negatively here, dismissed with no engagement with his specific arguments and examples.
  - atomicnature 9mo ago
    You can look into Judea Pearl's definitions of causality for more information. Pearl defines a ladder of causation: 1. Seeing (association) 2. Doing (intervention) 3. Imagining (counterfactuals) In his view - most ML algos are at level 1 - they look at data and draw associations, and "agents" have started some steps in level 2 - doing. The smartest of humans operate mostly in level (3) of abstractions - where they see things, gain experience, and later build up a "strong causal model" of the world and become capable of answering "what if" questions.

D-Machine 9mo ago

I had this exact reaction, no discussion of "causal modeling" makes the whole thing seem horribly out of touch with the real issues here. You can have explanatory and predictive models that are causal models, or explanatory and predictive models that are non-causal, and that this the actual issue, not "explanation" vs. "prediction", which is not a tight enough distinction.