5 ms·

I find R to be a great language for exploring a data set and doing some prototyping. There are a lot of wonderful statistical tools available through the core p

I find R to be a great language for exploring a data set and doing some prototyping. There are a lot of wonderful statistical tools available through the core packages and even more through the various community extensions. It does have some significant issues that I've found limit the usefulness outside of prototyping - pass by value only means code tends to end up as monolithic functions - very slow in loops so lot contorting to move things to matrix operations - they just last year got a version out that starts support for vectors and matrices with > 2^31 -1 elements which limits larger data applications. I find the plotting with ggplot and statistical functionality to be second to none though.

rm999 12y ago
> pass by value only means code tends to end up as monolithic functions I've actually found R works very well as a functional language with very lean functions. It's perhaps worth noting that R doesn't copy a dataframe in a function call if you don't modify it, which is a very common use-case for me. (I'm not sure if this extends to other datatypes) > very slow in loops so lot contorting to move things to matrix operations This is a fair criticism, I think more modern languages like Julia will win out here. That said, R has huge library support, I've often found there are compiled versions for a lot of what I want to do. > they just last year got a version out that starts support for vectors and matrices with > 2^31 -1 elements which limits larger data applications Again, a fair criticism. I've never considered R a "big data" tool, my workflow is usually a funnel where each step involves reducing data size by 1-3 orders of magnitude. For example, I may have 1 PB of transactional data, aggregate it in Hadoop to 20 TB of daily aggregated data, run a query that filters and aggregates it further, and then run my analysis in R on final data. In the end I may end up with 20 GB of data, which R can very easily handle.
- chuckcode 12y ago
  Python has better and better support for R with Rpy2 and R like data frames with Pandas, which is helping me take advantage of the incredibly useful analysis libraries in R. Also note that loops are slow enough that it is really worth learning the *apply() functions in R to avoid iterating over collections. For a relatively in depth explanation check out Hadley Wickham's book http://adv-r.had.co.nz/Functionals.html http://adv-r.had.co.nz/Functionals.html
  - mbq 12y ago
    *apply functions are loops underneath -- they only look better and save you time possibly wasted on growing some dynamically sized output structure. The way of solving slow loop in R is to find package which implements it in C/Fortran (or write your own in case there is none).
    - platz 12y ago
      I believe there is also the Rcpp package which lets you write inline compiled c++
    - craigching 12y ago
      > * apply functions are loops underneath Yes, but aren't they native loops underneath? I've seen it said both ways, that * apply is faster than R loops and that *apply isn't faster than R loops. Would be nice if someone could definitively answer the question and back it up with some stats! :) EDIT: Thanks chuckcode, sibling post to this, I stand corrected :)
    - chuckcode 12y ago
      It'a actually a little complicated but if you're interested in the details check out this stack overflow thread [1]. High level summary is that lapply() and functions built on top of it do some work in native C and so are generally faster but not all of the *apply() functions are faster. [1] http://stackoverflow.com/questions/2275896/is-rs-apply-family-more-than-syntactic-sugar http://stackoverflow.com/questions/2275896/is-rs-apply-famil...
      - mbq 12y ago
        The problem here is not the for-loop itself but the time used by the R runtime on executing the mapped function multiplied by the number of iterations (this is BTW the main source of advantage for dynamic and GCed but JITed languages like JS or Julia).
  - wch 12y ago
    It's a common misconception that for loops in R are slow. They're actually fast (around 20 million iterations per second on my computer). What can be slow is modifying data structures in particular ways using a loop. See http://rpubs.com/wch/46581 http://rpubs.com/wch/46581
mbq 12y ago
Java still has arrays limited to 2^31-few and no-one cares. And matrix operations make your code closer to the actual math, so less error-prone.
stared 12y ago
For ggplot - there is (sort of) a clone of it for Python, https://github.com/mwaskom/seaborn https://github.com/mwaskom/seaborn. And it has, arguably, better graphics. As a Python user, who resorts to R in case of need, the power of R is not in the language, but statistical community & packages.

rm999 12y ago

> pass by value only means code tends to end up as monolithic functions I've actually found R works very well as a functional language with very lean functions. It's perhaps worth noting that R doesn't copy a dataframe in a function call if you don't modify it, which is a very common use-case for me. (I'm not sure if this extends to other datatypes) > very slow in loops so lot contorting to move things to matrix operations This is a fair criticism, I think more modern languages like Julia will win out here. That said, R has huge library support, I've often found there are compiled versions for a lot of what I want to do. > they just last year got a version out that starts support for vectors and matrices with > 2^31 -1 elements which limits larger data applications Again, a fair criticism. I've never considered R a "big data" tool, my workflow is usually a funnel where each step involves reducing data size by 1-3 orders of magnitude. For example, I may have 1 PB of transactional data, aggregate it in Hadoop to 20 TB of daily aggregated data, run a query that filters and aggregates it further, and then run my analysis in R on final data. In the end I may end up with 20 GB of data, which R can very easily handle.

chuckcode 12y ago
Python has better and better support for R with Rpy2 and R like data frames with Pandas, which is helping me take advantage of the incredibly useful analysis libraries in R. Also note that loops are slow enough that it is really worth learning the *apply() functions in R to avoid iterating over collections. For a relatively in depth explanation check out Hadley Wickham's book http://adv-r.had.co.nz/Functionals.html http://adv-r.had.co.nz/Functionals.html
- mbq 12y ago
  *apply functions are loops underneath -- they only look better and save you time possibly wasted on growing some dynamically sized output structure. The way of solving slow loop in R is to find package which implements it in C/Fortran (or write your own in case there is none).
  - platz 12y ago
    I believe there is also the Rcpp package which lets you write inline compiled c++
  - craigching 12y ago
    > * apply functions are loops underneath Yes, but aren't they native loops underneath? I've seen it said both ways, that * apply is faster than R loops and that *apply isn't faster than R loops. Would be nice if someone could definitively answer the question and back it up with some stats! :) EDIT: Thanks chuckcode, sibling post to this, I stand corrected :)
  - chuckcode 12y ago
    It'a actually a little complicated but if you're interested in the details check out this stack overflow thread [1]. High level summary is that lapply() and functions built on top of it do some work in native C and so are generally faster but not all of the *apply() functions are faster. [1] http://stackoverflow.com/questions/2275896/is-rs-apply-family-more-than-syntactic-sugar http://stackoverflow.com/questions/2275896/is-rs-apply-famil...
    - mbq 12y ago
      The problem here is not the for-loop itself but the time used by the R runtime on executing the mapped function multiplied by the number of iterations (this is BTW the main source of advantage for dynamic and GCed but JITed languages like JS or Julia).
- wch 12y ago
  It's a common misconception that for loops in R are slow. They're actually fast (around 20 million iterations per second on my computer). What can be slow is modifying data structures in particular ways using a loop. See http://rpubs.com/wch/46581 http://rpubs.com/wch/46581

mbq 12y ago

Java still has arrays limited to 2^31-few and no-one cares. And matrix operations make your code closer to the actual math, so less error-prone.

stared 12y ago

For ggplot - there is (sort of) a clone of it for Python, https://github.com/mwaskom/seaborn https://github.com/mwaskom/seaborn. And it has, arguably, better graphics. As a Python user, who resorts to R in case of need, the power of R is not in the language, but statistical community & packages.