5 ms·

The programming language analysis is pretty interesting, but you have to ask, what's the point of a brand-new Java implementation? R isn't just a programming l

The programming language analysis is pretty interesting, but you have to ask, what's the point of a brand-new Java implementation? R isn't just a programming language, but it's a software framework/ecosystem. They mentioned this in the slides, but it's problematic because R crucially relies on C and Fortran interaction (which I thought the JVM can't do efficiently, since it doesn't like giving C/Fortran raw memory access to its internals). Decades of work has gone into highly optimized Fortran linear algebra libraries, for example -- which R and all the other high-level numerical languages (NumPy/SciPy, Matlab, Julia) use. And many of the CRAN packages (the availability of which are a major reason anyone uses R in the first place) are partly or mostly C/Fortran code. There are many other R implementation efforts going on right now -- Radford Neal lists a few (as well as his own) here: http://radfordneal.wordpress.com/2013/07/24/deferred-evaluation-in-renjin-riposte-and-pqr/ http://radfordneal.wordpress.com/2013/07/24/deferred-evaluat... The presentation focuses on the R programming language, which they nicely show has all sorts of misfeatures that impede rapid execution. If you're going to not try to have compatibility with R and CRAN, you might as well start from scratch with design and performance in mind, as in Julia: http://julialang.org/ http://julialang.org/

bedatadriven 13y ago
I think compatibility with GNU R and CRAN is a pretty reasonable goal - even if it's not the focus of this particular Oracle research project. We've been working hard and now systematically to get Renjin (also R on the JVM) to run CRAN packages: http://packages.renjin.org http://packages.renjin.org. Renjin also compiles C and Fortran code to JVM bytecode, though there is still some work to do there as well. Regarding the hand-tuned matrix math libraries, there's nothing to stop you from using them with Renjin - you can drop in MKL or Atlas as desired, or fall back to pure-Java versions in a pinch.
pjmlp 13y ago
Julia is quite cool, actually. As for Java, Oracle together with AMD, are in the process of making the GPU trasparent to Java developers as part of the Sumatra project. So this is one are where R could benefit of running on Oracle's JVM. It remains to be seen if other Java vendors would adopt such feature.
pron 13y ago
> ... C and Fortran interaction (which I thought the JVM can't do efficiently, since it doesn't like giving C/Fortran raw memory access to its internals). As of 2002 (JDK 1.4) Java has excellent integration with native memory (you can freely pass pointers from C/FORTRAN to Java and vice versa[1]). There are numerous Java math libraries that use BLAS/LAPACK already[2]. In fact, AFAIK, most Java matrix math libraries use FORTRAN code (at least as an option). [1]: Java side: http://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer.html http://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer... C side: http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/functions.html#nio_support http://docs.oracle.com/javase/7/docs/technotes/guides/jni/sp... [2]: For example, https://github.com/fommil/matrix-toolkits-java https://github.com/fommil/matrix-toolkits-java, http://mikiobraun.github.io/jblas/ http://mikiobraun.github.io/jblas/
- brendano 13y ago
  Ah, wonderful! So if I understand this correctly, this doesn't give C/Fortran access to Java-native primitive arrays; but instead, it's specific to NIO byte buffers (and then the matrix libraries have to build on top of that). But that should be fine for doing R replacements, at least in theory. (Personally, when programming Java I find it more convenient to use primitive arrays as opposed to matrix libraries, but that might be dependent on the operations I tend to do: lots of increment/decrements and only occasional linear algebra. I guess this isn't exactly relevant to the R replacement question.)
  - pron 13y ago
    Java also gives native libraries direct access to primitive arrays, but that requires pinning them in place for the duration of the call (i.e. not letting the GC move them) so it incurs some performance penalty. Direct byte buffers are very common in high-performance Java code. Reading/writing from/to those buffers can be made just as fast as plain Java arrays.
    - brendano 13y ago
      Thanks for the information. Sorry I was misinformed before...

bedatadriven 13y ago

I think compatibility with GNU R and CRAN is a pretty reasonable goal - even if it's not the focus of this particular Oracle research project. We've been working hard and now systematically to get Renjin (also R on the JVM) to run CRAN packages: http://packages.renjin.org http://packages.renjin.org. Renjin also compiles C and Fortran code to JVM bytecode, though there is still some work to do there as well. Regarding the hand-tuned matrix math libraries, there's nothing to stop you from using them with Renjin - you can drop in MKL or Atlas as desired, or fall back to pure-Java versions in a pinch.

pjmlp 13y ago

Julia is quite cool, actually. As for Java, Oracle together with AMD, are in the process of making the GPU trasparent to Java developers as part of the Sumatra project. So this is one are where R could benefit of running on Oracle's JVM. It remains to be seen if other Java vendors would adopt such feature.

pron 13y ago

> ... C and Fortran interaction (which I thought the JVM can't do efficiently, since it doesn't like giving C/Fortran raw memory access to its internals). As of 2002 (JDK 1.4) Java has excellent integration with native memory (you can freely pass pointers from C/FORTRAN to Java and vice versa[1]). There are numerous Java math libraries that use BLAS/LAPACK already[2]. In fact, AFAIK, most Java matrix math libraries use FORTRAN code (at least as an option). [1]: Java side: http://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer.html http://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer... C side: http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/functions.html#nio_support http://docs.oracle.com/javase/7/docs/technotes/guides/jni/sp... [2]: For example, https://github.com/fommil/matrix-toolkits-java https://github.com/fommil/matrix-toolkits-java, http://mikiobraun.github.io/jblas/ http://mikiobraun.github.io/jblas/

brendano 13y ago
Ah, wonderful! So if I understand this correctly, this doesn't give C/Fortran access to Java-native primitive arrays; but instead, it's specific to NIO byte buffers (and then the matrix libraries have to build on top of that). But that should be fine for doing R replacements, at least in theory. (Personally, when programming Java I find it more convenient to use primitive arrays as opposed to matrix libraries, but that might be dependent on the operations I tend to do: lots of increment/decrements and only occasional linear algebra. I guess this isn't exactly relevant to the R replacement question.)
- pron 13y ago
  Java also gives native libraries direct access to primitive arrays, but that requires pinning them in place for the duration of the call (i.e. not letting the GC move them) so it incurs some performance penalty. Direct byte buffers are very common in high-performance Java code. Reading/writing from/to those buffers can be made just as fast as plain Java arrays.
  - brendano 13y ago
    Thanks for the information. Sorry I was misinformed before...