7 ms·

Sounds like you agree with nfoz... you don't always need mathematical expertise ('scientific validity') to play with data, but when you do, you need a statistic

Sounds like you agree with nfoz... you don't always need mathematical expertise ('scientific validity') to play with data, but when you do, you need a statistician. Really, it comes down whether you need to confirm your assumptions and quantify your uncertainty, etc. Often, in applied situations, you don't. In science, you do.

monkeyspaw 13y ago
Yup. I don't plan on publishing anything I do in a peer reviewed journal. If I did, I would have to be way more cautious with a lot of my assumptions. By freeing myself of that level of effort, I can test a lot more stuff and build things a lot faster. In what I do, that's more important than having something be "scientifically accurate". Similarly, when I am coding a prototype, I don't do unit tests. Don't need them, and it takes me longer to develop.
- kyzyl 13y ago
  Pardon me if I come off as rude, but you've made a couple of statements that strike me as rather silly. Maybe I've misunderstood you. The function of a peer reviewed publication is to formalize and verify that some result is correct, reproducible, and relevant (and to advance tenure). One would hope that whether you publish or not is mostly orthogonal to how you conduct your work. If you're doing data analysis in any professional capacity, you should be using mathematically sound techniques, and you should understand those techniques. There's no other way about it. I'm not sure what your line of work is, but I can't think of any technical field where one's analysis techniques being "scientifically sound" is anything but paramount. Why else would the work have any value? Perhaps it helps to think about publishing your work to be more like deploying your code, rather than testing it. That isn't to say that every step of the way you have to conduct yourself with the utmost rigor. It doesn't mean that you must prove every theorem each time you use it, nor that you have to be laying out strict tests and hypotheses every time you load a data set. But to do data analysis in a scientifically sound manner does mean that you have to understand the background of the techniques you're using, and how they apply to your data. It does mean that you have to periodically bring your analysis back to basics and make sure your ducks are still in a row; that you haven't fallen off the assumption wagon somewhere along the line. It does mean that you should know why something is working, or why something is broken, not just that it is. "Statisticians" don't sit around all day laying out hypothesis tests in latex. They do the same thing you do; they fire up R, load the data and start playing with it. It's all about the context you have in your brain when you are playing with it. This, I think, is where the formal training becomes quite important. Because as with most things, it's the unknown unknowns that will destroy you, and having the broad formal foundation gives you the tools to protect yourself from walking into a minefield problems you didn't even know existed yet.
  - monkeyspaw 13y ago
    I guess I just see the level of "scientific soundness" as a spectrum. For some work (peer reviewed journals, work you hope other people to be able to reproduce, etc.) you need a higher standard. I find that higher standard requires a significant amount more time, effort, and ultimately prevents me from accomplishing what I need to. It depends on the application and how you're using the result. Most of the time, for me, it just doesn't matter. It may be worth noting that much of what i'm talking about is for personal understanding (e.g., baseball statistics) or is not mission critical and will never see the light of day. Sometimes good enough is just that. I do appreciate the thoughtful response you wrote. But often my decision is this: do I spend 2 weeks trying to understand a technique, or do I spend 1 day using it, understanding that my analysis has limitations. When you only have 1 day, my choice becomes "do what seems like it will work" or "do nothing." I agree that more complex techniques (neural nets, etc.) may need more understanding. But because they do, they are also inaccessible to me. Maybe I can sum it up like this: it's not always necessary to do an analyses in a completely scientifically sound manner, if I can answer the questions to my satisfaction. Especially relevant when the other option is to not get anything done, and get stuck in a textbook trying to understand complex theory. It's bit me once in a while, but that doesn't matter in what I'm doing. (And its usually pretty easy to recover from.) Also worth a mention that I would not call myself a data scientist, nor do I operate in that role. It's all tradeoffs, all the way down. I just happen to choose mine differently than someone who might call themselves a data scientist or statistician.

monkeyspaw 13y ago

Yup. I don't plan on publishing anything I do in a peer reviewed journal. If I did, I would have to be way more cautious with a lot of my assumptions. By freeing myself of that level of effort, I can test a lot more stuff and build things a lot faster. In what I do, that's more important than having something be "scientifically accurate". Similarly, when I am coding a prototype, I don't do unit tests. Don't need them, and it takes me longer to develop.

kyzyl 13y ago
Pardon me if I come off as rude, but you've made a couple of statements that strike me as rather silly. Maybe I've misunderstood you. The function of a peer reviewed publication is to formalize and verify that some result is correct, reproducible, and relevant (and to advance tenure). One would hope that whether you publish or not is mostly orthogonal to how you conduct your work. If you're doing data analysis in any professional capacity, you should be using mathematically sound techniques, and you should understand those techniques. There's no other way about it. I'm not sure what your line of work is, but I can't think of any technical field where one's analysis techniques being "scientifically sound" is anything but paramount. Why else would the work have any value? Perhaps it helps to think about publishing your work to be more like deploying your code, rather than testing it. That isn't to say that every step of the way you have to conduct yourself with the utmost rigor. It doesn't mean that you must prove every theorem each time you use it, nor that you have to be laying out strict tests and hypotheses every time you load a data set. But to do data analysis in a scientifically sound manner does mean that you have to understand the background of the techniques you're using, and how they apply to your data. It does mean that you have to periodically bring your analysis back to basics and make sure your ducks are still in a row; that you haven't fallen off the assumption wagon somewhere along the line. It does mean that you should know why something is working, or why something is broken, not just that it is. "Statisticians" don't sit around all day laying out hypothesis tests in latex. They do the same thing you do; they fire up R, load the data and start playing with it. It's all about the context you have in your brain when you are playing with it. This, I think, is where the formal training becomes quite important. Because as with most things, it's the unknown unknowns that will destroy you, and having the broad formal foundation gives you the tools to protect yourself from walking into a minefield problems you didn't even know existed yet.
- monkeyspaw 13y ago
  I guess I just see the level of "scientific soundness" as a spectrum. For some work (peer reviewed journals, work you hope other people to be able to reproduce, etc.) you need a higher standard. I find that higher standard requires a significant amount more time, effort, and ultimately prevents me from accomplishing what I need to. It depends on the application and how you're using the result. Most of the time, for me, it just doesn't matter. It may be worth noting that much of what i'm talking about is for personal understanding (e.g., baseball statistics) or is not mission critical and will never see the light of day. Sometimes good enough is just that. I do appreciate the thoughtful response you wrote. But often my decision is this: do I spend 2 weeks trying to understand a technique, or do I spend 1 day using it, understanding that my analysis has limitations. When you only have 1 day, my choice becomes "do what seems like it will work" or "do nothing." I agree that more complex techniques (neural nets, etc.) may need more understanding. But because they do, they are also inaccessible to me. Maybe I can sum it up like this: it's not always necessary to do an analyses in a completely scientifically sound manner, if I can answer the questions to my satisfaction. Especially relevant when the other option is to not get anything done, and get stuck in a textbook trying to understand complex theory. It's bit me once in a while, but that doesn't matter in what I'm doing. (And its usually pretty easy to recover from.) Also worth a mention that I would not call myself a data scientist, nor do I operate in that role. It's all tradeoffs, all the way down. I just happen to choose mine differently than someone who might call themselves a data scientist or statistician.