6 ms·

Another Book on Data Science – Learn R and Python in Parallel

214 points by zelda_1 7y ago

clarityPhone 7y ago
I skimmed through the book, and think it does a very poor job at showcasing how R and Python are juxtaposed in industry. To be fair, the book advertises showing R and Python code side-by-side. And that’s what it does. But it does it unlike how the languages are most often used in industry. As a quick example, I saw no tidyverse code, which is essentially the only thing keeping R in the game. Learning R from this book won’t prepare you for writing R in most R shops. I don’t see the utility in knowing how to do the same thing in both python and R if you’re a beginner. This is even more true if you’re not taking advantage of the strengths/weaknesses of either language. Instead, just learn one of the languages well, and then learn the other well. Shallow dives in both will make you weak in both. Unfortunately, 90% of data science content seems to be geared at beginners.
- ackbar03 7y ago
  That's pretty much the only group of people who will use this though. Those that are serious about it or have some background won't really look at another book on data science and probably do the necessary research themselves
- thomasfedb 7y ago
  Is tidyverse really the only option? I'm a big fan of data.table + magrittr as a very powerful data munging combo.
  - notafraudster 7y ago
    magrittr is part of the tidyverse, but I agree that data.table is a comparably powerful and sometimes faster option versus dplyr.
    - Bootvis 7y ago
      magrittr existed before the tidyverse and can be used on standalone perfectly fine. In all benchmarks I've seen data.table is faster than dplyr on all tasks. Curious to see other results.
      - thomasfedb 7y ago
        At the scale of what I'm doing the benchmarks don't sway me, but I do like the syntax of data.table - it feels a bit like relational algebra.
        RosanaAnaDana 7y ago
        So then I would assume you must be working with tables of less than 1000 rows, because thats pretty much the only case where it doesn't matter. At anything more than 1k rows, the differences are substantial.
        thomasfedb 7y ago
        Hundreds of rows is about usual for me. I do analysis on clinical studies with human participants. Nothing too tricky, most of my munging runs in effectively zero time.
    - creddit 7y ago
      Almost always faster, actually. https://github.com/Rdatatable/data.table/wiki/Benchmarks-:-Grouping https://github.com/Rdatatable/data.table/wiki/Benchmarks-:-G...
      - RosanaAnaDana 7y ago
        I was going to make this point, but yeah. The only thing I think people have a bit of a time with is how you do operations in data.table. If you are coming from plyr/dplyr, the transition can be difficult. However, I've found that the more I do, the more I prefer it, inspite of the fact that the main reason I use dt over tidy is the phenomenal performance gain.
  - RosanaAnaDana 7y ago
    This is what I've got my crew running. Tidyverse is basicly worthless once you hit a certain scale of data. If you've got datatables representing. List functionality within datatable is blazingly fast. Faster than anything else I've seen in python or R.
- wjn0 7y ago
  I agree for the most part, but R does have a few things beyond the tidyverse: built-in dataframe support, lots of domain-specific packages, more consistent interfaces for basic statistics and machine learning models, etc. Python is definitely better for matrices (because of NumPy) and anything involving custom gradient descent methods (because of TensorFlow). I think 90% of data science content is for beginners because anything more advanced isn't best described as data science. As soon as you get beyond the initial stages of data analysis (cleaning and processing data), you're doing something best described as some other word (statistics, machine learning, etc.) - although, granted, there isn't much content in these areas if you don't know _exactly_ what you're looking for.
  - cwyers 7y ago
    Even if you ignore the tidyverse, the example code for "roll your own linear regressions by hand" uses the R6 object system, which is... not even one of the two popular object systems for R (which are S3 and S4). No beginner needs to learn how to write classes in R.
    - zelda_1 7y ago
      `no beginner needs to learn how to write classes in R`. a) using classes properly is great for all level R users; b) a major reason that classes are not widely used (for beginners) is that S3/S4 are not easy to follow. R6 provides a natural and clear way to understand and write classes (especially for beginners).
      - cwyers 7y ago
        Using classes at all is unnecessary for most R users. R is really, to the extent that paradigms matter to the average R user at all (which is: not much) a functional-first language. The idiomatic way to deal with the things you would use classes for is to use functions and closures. There are people who need objects in R, which is why R has object systems available, but it is of no help to a beginner to know them -- it doesn't help them to interact with the code they are going to see, and they don't have the background to understand why you would use classes instead of functions.
- bryanrasmussen 7y ago
  Theodore Sturgeon update: 90% of all programming books are geared at beginners.
- dlphn___xyz 7y ago
  any suggestions for intermediate-advanced level articles outside of distill?
- KKPMW 7y ago
  > tidyverse code, which is essentially the only thing keeping R in the game From my experience this is not the case. In biomedicine and bioinformatics few people actually use tidyverse because the data is much better represented as a matrix, and not in the "tidy" form. Outside of that corporations (well at least 2 I contracted with) used `data.table` explicitly. Join 3 ad-click dataframes matching by userID, sessionID and closest possible time-point - that's one line in `data.table`. Tidyverse is well suited for learning and for managing (relatively) simple datasets. But becomes cumbersome for more complex data. It can be used for those data too of course, just that it will be adding ad-hoc solutions and maybe get in a way more than help.
  - zelda_1 7y ago
    I have a similar feeling. And that is why I spent one whole chapter in data.table (and pandas). Hope more R users would like to learn and use data.table.
  - anthony_doan 7y ago
    I agree with your experiences. I've only use base R for my medical data (subsetting dataframe and such). Very rarely do I need tidy and also I find the pipe operator makes debugging harder. If and when I need it I'll use it that's that. I think R have much more packages in medical, especially statistical packages, where many fields within medical cares about inferences not just prediction/forecasting. So I disagree with the "essentially the only thing keeping R in the game". The breath of packages in R is one of the many things that keep R in the game. The tribalism and highly bias comments makes it very toxic and harder to have an honest discord. They are just tools, use what makes you happy and get the job done.
adamnemecek 7y ago
Julia is hands down much better than either of these languages. Don't waste your time.
- jhbadger 7y ago
  It's a matter of ecosystem of packages. R has a huge number of packages for many fields. Python has fewer, but might work for particular use cases. I was excited for Julia, and played around with it since 0.2, but it really hasn't generated very many packages of note in my particular field (bioinformatics).
  - adamnemecek 7y ago
    julia has really good interop with both of python and R as well as cpp, matlab, mathematica and others. Also it's not just about the numbers.
    - randomvectors 7y ago
      So you're telling us not to waste our time with R or Python... but to constantly interop with R and Python?
      - adamnemecek 7y ago
        I mean if there are packages that you need then be my guest. But like if you are starting a new project, julia is a more productive language.
        s_Hogg 7y ago
        > julia is a more productive language Source for this, please?
        adamnemecek 7y ago
        https://news.ycombinator.com/item?id=20240155 https://news.ycombinator.com/item?id=20240155
randomvectors 7y ago
This just doesn't seem to have a place. 1. It's aimed at beginners. 2. If you're a beginner, you're best off picking one language and sticking with it for a while. 3. There are so many other beginner resources that are much better.
- thewhitetulip 7y ago
  Can you please list a few? I'm currently following ISLR book & course
  - randomvectors 7y ago
    I can list a bunch. But the answer depends on a lot of things. 1. Which language? 2. Do you have any background in programming and if yes how much and in which language? Beginners can range from "I don't know what a loop is" to "I've been a front end developer for a few years and want to try something new". These people will obviously need to approach things very differently. 3. What's your math background? Do you know what a derivative is and how matrix multiplication works? Do you want to go in depth or do you need just a general understanding of the algorithms? Some people will need to start at precalculus if they really want a solid foundation. The same people also most likely won't have the patience or interest to stick with the theory for long enough. 4. How will you use what you're learning? Is there a specific goal related to this and does it have a time horizon? ... Anyway, I'm rambling. Data science is just programming and statistics. Your general learning lanes are 1. Theory - calculus, linear algebra, statistics, ML algorithms. 2. Programming. 2a. Good software development principles - writing maintainable code, version control, testing, design patterns etc. 2b. Tooling - learning the language-specific ecosystem of libraries. This is what most beginner resources (including the OP) focus on and is also the one that constantly expires and has the least transferability to other skillsets and fields. Obviously, it's still necessary - using the right tools and knowing them well goes a long way. You mentioned ISLR - it's a good beginner-ish theory book that helps you understand how the algorithms work without going too deep into the math.
  - randomvectors 7y ago
    Depends on what you're interested in, your goals and your starting point. You have two main learning lanes: 1. Theory. Math, calculus, linear algebra, probability, statistics, ML algorithms. ISLR is a very good beginner-ish resource that helps you understand the algorithms but doesn't go too deep into the math. As you go deeper, you may realise that you have gaps in your math knowledge and you need to cover a lot more probability, calculus and linear algebra. 2. Programming. 2.1. Good software engineering practices - writing maintainable code, design patters, version control, unit testing etc. 2.2. Tooling - knowing the language-specific ecosystem of libraries (the OP is an attempt to teach you this in two languages at the same time). This is what most beginner resources focus on; your knowledge here has the least transferability and tends to go out of date quickly. Still, using the right tools and knowing them well goes a long way.
Dkastro92 7y ago
I think it's a good first impression of either languague
omnimkar69 7y ago
I also skimmed through the book and it does very poor work in case of python I think they have to change this concept
omnimkar69 7y ago
I also skimmed through the book and it does very poor work in the case of python I think that they should change R bcoz 90% are the beginners
thatcat 7y ago
I'd be interested in a version that included SAS. Is SAS ever used outside of academia?
- s_Hogg 7y ago
  Yes, though at least where I am it tends to be a legacy system these days. So that means governments and large corporations might reasonably feature some use of it somewhere within their walls.
- randomvectors 7y ago
  SAS is still used by companies who don't want to use open source tools and would rather pay a lot of money for an established product name and have a support line that they can call if anything goes wrong. Understandably, it's slowly dying out.
  - tomrod 7y ago
    Support, and also indemnification. SAS lives on borrowed time.
- hobofromabroad 7y ago
  SAS ist still quite a big thing in the insurance sector, at least in Europe. But the more Data Science focused roles (vs. Pure actuarial roles) are going more and more with R or Python.
- anthony_doan 7y ago
  Yes. I've seen pharma, nonprofit cancer organization, kaiser permanente hospital, State government (epidemiology), etc...
euler_angles 7y ago
What are the good intermediate level data science books?
dlphn___xyz 7y ago
i only read the chapter on optimization/linear programming - its far too brief and misses key concepts for it to be useful. there are no real applications of the concepts covered in the section.
- dlphn___xyz 7y ago
  but it looks nice
- zelda_1 7y ago
  Yes, it is very brief. I tried to give some useful reference books/papers/links for each subject appeared in the book. Hope that is useful if the readers are willing to dive deeper. But the concept of linear programming itself is not complex, the users are generally not required to understand simplex/interior point algorithm. If they are able to translate the actual problem into the code then it is basically done. It is not like constrained optimization for which customized treatment based on mathematical theories is more important than writing the code.
anonu 7y ago
What does the HN community think about Python for statistical computing? Core Python is fine. But pandas is an atrocious mess of object orientedness and other weird stuff.

214 points by zelda_1 7y ago

I skimmed through the book, and think it does a very poor job at showcasing how R and Python are juxtaposed in industry. To be fair, the book advertises showing R and Python code side-by-side. And that’s what it does. But it does it unlike how the languages are most often used in industry. As a quick example, I saw no tidyverse code, which is essentially the only thing keeping R in the game. Learning R from this book won’t prepare you for writing R in most R shops. I don’t see the utility in knowing how to do the same thing in both python and R if you’re a beginner. This is even more true if you’re not taking advantage of the strengths/weaknesses of either language. Instead, just learn one of the languages well, and then learn the other well. Shallow dives in both will make you weak in both. Unfortunately, 90% of data science content seems to be geared at beginners.

ackbar03 7y ago
That's pretty much the only group of people who will use this though. Those that are serious about it or have some background won't really look at another book on data science and probably do the necessary research themselves
thomasfedb 7y ago
Is tidyverse really the only option? I'm a big fan of data.table + magrittr as a very powerful data munging combo.
- notafraudster 7y ago
  magrittr is part of the tidyverse, but I agree that data.table is a comparably powerful and sometimes faster option versus dplyr.
  - Bootvis 7y ago
    magrittr existed before the tidyverse and can be used on standalone perfectly fine. In all benchmarks I've seen data.table is faster than dplyr on all tasks. Curious to see other results.
    - thomasfedb 7y ago
      At the scale of what I'm doing the benchmarks don't sway me, but I do like the syntax of data.table - it feels a bit like relational algebra.
      - RosanaAnaDana 7y ago
        So then I would assume you must be working with tables of less than 1000 rows, because thats pretty much the only case where it doesn't matter. At anything more than 1k rows, the differences are substantial.
        thomasfedb 7y ago
        Hundreds of rows is about usual for me. I do analysis on clinical studies with human participants. Nothing too tricky, most of my munging runs in effectively zero time.
  - creddit 7y ago
    Almost always faster, actually. https://github.com/Rdatatable/data.table/wiki/Benchmarks-:-Grouping https://github.com/Rdatatable/data.table/wiki/Benchmarks-:-G...
    - RosanaAnaDana 7y ago
      I was going to make this point, but yeah. The only thing I think people have a bit of a time with is how you do operations in data.table. If you are coming from plyr/dplyr, the transition can be difficult. However, I've found that the more I do, the more I prefer it, inspite of the fact that the main reason I use dt over tidy is the phenomenal performance gain.
- RosanaAnaDana 7y ago
  This is what I've got my crew running. Tidyverse is basicly worthless once you hit a certain scale of data. If you've got datatables representing. List functionality within datatable is blazingly fast. Faster than anything else I've seen in python or R.
wjn0 7y ago
I agree for the most part, but R does have a few things beyond the tidyverse: built-in dataframe support, lots of domain-specific packages, more consistent interfaces for basic statistics and machine learning models, etc. Python is definitely better for matrices (because of NumPy) and anything involving custom gradient descent methods (because of TensorFlow). I think 90% of data science content is for beginners because anything more advanced isn't best described as data science. As soon as you get beyond the initial stages of data analysis (cleaning and processing data), you're doing something best described as some other word (statistics, machine learning, etc.) - although, granted, there isn't much content in these areas if you don't know _exactly_ what you're looking for.
- cwyers 7y ago
  Even if you ignore the tidyverse, the example code for "roll your own linear regressions by hand" uses the R6 object system, which is... not even one of the two popular object systems for R (which are S3 and S4). No beginner needs to learn how to write classes in R.
  - zelda_1 7y ago
    `no beginner needs to learn how to write classes in R`. a) using classes properly is great for all level R users; b) a major reason that classes are not widely used (for beginners) is that S3/S4 are not easy to follow. R6 provides a natural and clear way to understand and write classes (especially for beginners).
    - cwyers 7y ago
      Using classes at all is unnecessary for most R users. R is really, to the extent that paradigms matter to the average R user at all (which is: not much) a functional-first language. The idiomatic way to deal with the things you would use classes for is to use functions and closures. There are people who need objects in R, which is why R has object systems available, but it is of no help to a beginner to know them -- it doesn't help them to interact with the code they are going to see, and they don't have the background to understand why you would use classes instead of functions.
bryanrasmussen 7y ago
Theodore Sturgeon update: 90% of all programming books are geared at beginners.
dlphn___xyz 7y ago
any suggestions for intermediate-advanced level articles outside of distill?
KKPMW 7y ago
> tidyverse code, which is essentially the only thing keeping R in the game From my experience this is not the case. In biomedicine and bioinformatics few people actually use tidyverse because the data is much better represented as a matrix, and not in the "tidy" form. Outside of that corporations (well at least 2 I contracted with) used `data.table` explicitly. Join 3 ad-click dataframes matching by userID, sessionID and closest possible time-point - that's one line in `data.table`. Tidyverse is well suited for learning and for managing (relatively) simple datasets. But becomes cumbersome for more complex data. It can be used for those data too of course, just that it will be adding ad-hoc solutions and maybe get in a way more than help.
- zelda_1 7y ago
  I have a similar feeling. And that is why I spent one whole chapter in data.table (and pandas). Hope more R users would like to learn and use data.table.
- anthony_doan 7y ago
  I agree with your experiences. I've only use base R for my medical data (subsetting dataframe and such). Very rarely do I need tidy and also I find the pipe operator makes debugging harder. If and when I need it I'll use it that's that. I think R have much more packages in medical, especially statistical packages, where many fields within medical cares about inferences not just prediction/forecasting. So I disagree with the "essentially the only thing keeping R in the game". The breath of packages in R is one of the many things that keep R in the game. The tribalism and highly bias comments makes it very toxic and harder to have an honest discord. They are just tools, use what makes you happy and get the job done.

adamnemecek 7y ago

Julia is hands down much better than either of these languages. Don't waste your time.

jhbadger 7y ago
It's a matter of ecosystem of packages. R has a huge number of packages for many fields. Python has fewer, but might work for particular use cases. I was excited for Julia, and played around with it since 0.2, but it really hasn't generated very many packages of note in my particular field (bioinformatics).
- adamnemecek 7y ago
  julia has really good interop with both of python and R as well as cpp, matlab, mathematica and others. Also it's not just about the numbers.
  - randomvectors 7y ago
    So you're telling us not to waste our time with R or Python... but to constantly interop with R and Python?
    - adamnemecek 7y ago
      I mean if there are packages that you need then be my guest. But like if you are starting a new project, julia is a more productive language.
      - s_Hogg 7y ago
        > julia is a more productive language Source for this, please?
        adamnemecek 7y ago
        https://news.ycombinator.com/item?id=20240155 https://news.ycombinator.com/item?id=20240155

randomvectors 7y ago

This just doesn't seem to have a place. 1. It's aimed at beginners. 2. If you're a beginner, you're best off picking one language and sticking with it for a while. 3. There are so many other beginner resources that are much better.

thewhitetulip 7y ago
Can you please list a few? I'm currently following ISLR book & course
- randomvectors 7y ago
  I can list a bunch. But the answer depends on a lot of things. 1. Which language? 2. Do you have any background in programming and if yes how much and in which language? Beginners can range from "I don't know what a loop is" to "I've been a front end developer for a few years and want to try something new". These people will obviously need to approach things very differently. 3. What's your math background? Do you know what a derivative is and how matrix multiplication works? Do you want to go in depth or do you need just a general understanding of the algorithms? Some people will need to start at precalculus if they really want a solid foundation. The same people also most likely won't have the patience or interest to stick with the theory for long enough. 4. How will you use what you're learning? Is there a specific goal related to this and does it have a time horizon? ... Anyway, I'm rambling. Data science is just programming and statistics. Your general learning lanes are 1. Theory - calculus, linear algebra, statistics, ML algorithms. 2. Programming. 2a. Good software development principles - writing maintainable code, version control, testing, design patterns etc. 2b. Tooling - learning the language-specific ecosystem of libraries. This is what most beginner resources (including the OP) focus on and is also the one that constantly expires and has the least transferability to other skillsets and fields. Obviously, it's still necessary - using the right tools and knowing them well goes a long way. You mentioned ISLR - it's a good beginner-ish theory book that helps you understand how the algorithms work without going too deep into the math.
- randomvectors 7y ago
  Depends on what you're interested in, your goals and your starting point. You have two main learning lanes: 1. Theory. Math, calculus, linear algebra, probability, statistics, ML algorithms. ISLR is a very good beginner-ish resource that helps you understand the algorithms but doesn't go too deep into the math. As you go deeper, you may realise that you have gaps in your math knowledge and you need to cover a lot more probability, calculus and linear algebra. 2. Programming. 2.1. Good software engineering practices - writing maintainable code, design patters, version control, unit testing etc. 2.2. Tooling - knowing the language-specific ecosystem of libraries (the OP is an attempt to teach you this in two languages at the same time). This is what most beginner resources focus on; your knowledge here has the least transferability and tends to go out of date quickly. Still, using the right tools and knowing them well goes a long way.

Dkastro92 7y ago

I think it's a good first impression of either languague

omnimkar69 7y ago

I also skimmed through the book and it does very poor work in case of python I think they have to change this concept

omnimkar69 7y ago

I also skimmed through the book and it does very poor work in the case of python I think that they should change R bcoz 90% are the beginners

thatcat 7y ago

I'd be interested in a version that included SAS. Is SAS ever used outside of academia?

s_Hogg 7y ago
Yes, though at least where I am it tends to be a legacy system these days. So that means governments and large corporations might reasonably feature some use of it somewhere within their walls.
randomvectors 7y ago
SAS is still used by companies who don't want to use open source tools and would rather pay a lot of money for an established product name and have a support line that they can call if anything goes wrong. Understandably, it's slowly dying out.
- tomrod 7y ago
  Support, and also indemnification. SAS lives on borrowed time.
hobofromabroad 7y ago
SAS ist still quite a big thing in the insurance sector, at least in Europe. But the more Data Science focused roles (vs. Pure actuarial roles) are going more and more with R or Python.
anthony_doan 7y ago
Yes. I've seen pharma, nonprofit cancer organization, kaiser permanente hospital, State government (epidemiology), etc...

euler_angles 7y ago

What are the good intermediate level data science books?

dlphn___xyz 7y ago

i only read the chapter on optimization/linear programming - its far too brief and misses key concepts for it to be useful. there are no real applications of the concepts covered in the section.

dlphn___xyz 7y ago
but it looks nice
zelda_1 7y ago
Yes, it is very brief. I tried to give some useful reference books/papers/links for each subject appeared in the book. Hope that is useful if the readers are willing to dive deeper. But the concept of linear programming itself is not complex, the users are generally not required to understand simplex/interior point algorithm. If they are able to translate the actual problem into the code then it is basically done. It is not like constrained optimization for which customized treatment based on mathematical theories is more important than writing the code.

anonu 7y ago

What does the HN community think about Python for statistical computing? Core Python is fine. But pandas is an atrocious mess of object orientedness and other weird stuff.