10 ms·

Generalizing Support for Functional OOP in R

109 points by samch93 2y ago

rossdavidh 2y ago
To be honest, OOP never really seemed like a good fit for R. Functional programming is a much more natural fit, given that both it and R come from a mathematical point of view. R is a great language for the mathematical/statistical stuff it was invented to do, but I don't think it will ever be a general-purpose language, and it probably would become worse at its core purpose if it tried to. More work on being easily used by/incorporated into applications written in other languages, would perhaps be a more impactful thing to work on.
- bachmeier 2y ago
  > More work on being easily used by/incorporated into applications written in other languages, would perhaps be a more impactful thing to work on. That's basically a solved problem. For instance, RInside opens a C interface that can be called by any language that can call C functions, which is basically every language. It's efficient, too, because you're only passing pointers around. Here's an example in Ruby (disclaimer that I wrote it): https://github.com/eddelbuettel/rinside/blob/master/inst/examples/c_interface/passdata.rb https://github.com/eddelbuettel/rinside/blob/master/inst/exa...
- orhmeh09 2y ago
  OOP has been a critical part of real-life R for a long time, especially in complex implementations of classes of kernels, algorithms, and so on with S4, and more general-purpose with R6. Without these frameworks it would be difficult to implement them. Personally I find it more expressive for general-purpose computation than Python. The "fs" library is much better at working with files and paths than Python "os" and the multiple other modules that can be needed to work with with typical filesystem operations -/ especially if you are working with more than one file at a time. I would even say that each of the R object systems is more expressive and more flexible than the Python one. I suspect lazy evaluation is a part of this.
  - fire_lake 2y ago
    I too much prefer R to Python (it’s far more expressive, for one) however it’s clear now that Python has “won” in this space and R is a tough sell to a wider team.
    - bachmeier 2y ago
      > it’s clear now that Python has “won” in this space and R is a tough sell to a wider team R has always been a language for academics, and it continues to be popular in that domain, with no compelling reason to switch. It has seen usage in the private sector, but that has never been the driving force behind R's development or ecosystem, and I doubt it ever will be. For academics, even if a particular function is only available for Python, it's easy enough to call it from R and do everything else in R.
    - orhmeh09 2y ago
      Oh, I agree on that, and I know that most of the time I'll be asked why something isn't in Python so I mainly reserve it for when I am sure nobody will ask "why R?" :-) There is a benefit nowadays that I can rely on Python>=3.6 to be available by default anywhere I am deploying whereas R has to be installed in some way, so like Bash it's part of a toolbox I can rely on being available with at least a constrained set of features.
    - rossdavidh 2y ago
      My experience is that you can "sell" R when the statistical or modelling technique is not (or not well) implemented in python. Which still includes a lot of potentially useful statistical techniques! R should/could lean into being the tool that python programmers reach for when they need something a little less mainstream, if they made it easier for non-R programmers to do so.
  - agumonkey 2y ago
    > I suspect lazy evaluation is a part of this. I had no idea R was lazy. Makes me wanna learn it now.
    - 2y ago
- andrewla 2y ago
  OOP as used in R is very much a function of API design and not a function of routine R usage for data analysis. To many users of R they are not even aware that they are using OOP at all, especially for the S3 style of objects. When you have an object, like `model <- lm(x~y)` or `my_hist <- hist(df$foo)`, you expected to be able to `plot` it or get a `summary`; you don't call `my_hist.summary`, you call `summary(my_hist)` and `plot(model)`. Many users never look further under the hood than this. And this fits nicely into piped workflows -- `lm(x~y) |> summary()` ends up being very natural, and when you fit in the tidyverse operators many very complex workflows end up being very easy to digest. But when you do pull back the kimono it gets ugly fast. The teams involved in this are the right people who have been working to make R an amazing language mostly through enhancements to libraries, and now they're trying to push some of that functionality back into core R, which I think is fantastic.
  - th0ma5 2y ago
    Some detail about that phrase https://www.catalyst.org/2021/03/22/racism-misogyny-asian-american-women-workplace/ https://www.catalyst.org/2021/03/22/racism-misogyny-asian-am...
    - Onawa 2y ago
      I can honestly say that I had never heard that phrase used before now, but I do know I felt icky when I read it in the comment before I even clicked on your link. Definitely glad to see it is being called out, terms like this absolutely need to be removed from modern discourse.
      - deleted 2y ago
        [deleted]
    - chuckadams 2y ago
      Pretty sure the phrase GP was looking for was "pull back the curtain", which likely originated with The Wizard of Oz.
  - clatan 2y ago
- usgroup 2y ago
  Yet you’re using OOP every time you call plot, predict, summary and so on: possibly without realising.
  - CornCobs 2y ago
    I actually think this "multi-tiered" system of OOP is quite cool, when compared to languages that stick OOP in your face upfront. 1. Basic users don't even know it's there, they're just calling regular functions. 2. S3 in base is super simple to understand and easy to extend the first time you need to implement your own summary. 3. Full blown OOP with slots and methods is available when you really need it (rare for a user and not library author imo, lists and S3 are sufficient for most things). The big issue I see is the incompatibilities in the various systems making this "ramp up" not so smooth. But it looks like that's what S7 is trying to address so that's cool.
- PheonixPharts 2y ago
  Functional programming is not orthogonal to object oriented programming, they are paradigms that can be used together and the popular object system developed in Java is not close to the only way to do OOP. R, like Common Lisp, uses an OOP system based on generic functions (well, one of many OOP systems in R, but that's a different topic), where the function handles dispatching to match the object. Effectively instead of: object.method() you have: method(object) So it works perfectly with the functional paradigm while still being capable of everything an object system is. The most obvious example of this in R is the `plot` function. You as the programmer don't have to know exactly how to plot an instance of a class, you just pass it into `plot` and it will be handled correctly. If you create a new class you just have to extend the definition of plot in a standard way and it will also be handled for you. It's a shame that the many flavors of OOP remain relatively unknown to most programmers, and in many of the cases where they're tried (JavaScript's prototype system for example) people have essentially replaced them with more familiar systems.
  - epgui 2y ago
    > Effectively instead of object.method() you have method(object) This reflects a very deep misunderstanding of the distinctive characteristics of each paradigm. It's so far off that it's "not even wrong". Functional programming is much more about things like purity and referential transparency, about composing functions and/or combinators, about a particular way of managing or modelling effects, about a way of thinking, about using certain kinds of data structures and algorithms. It's not a syntactical difference.
    - fn-mote 2y ago
      >> Effectively instead of object.method() you have method(object) > This reflects a very deep misunderstanding of the distinctive characteristics of each paradigm. I think the parent made a hasty reading of the GP comment. The GP shows an awareness of multiple OO systems in R. I believe the GP is attempting to explain to a Java programmer how R could be considered object-oriented even though `plot(item)` does not "look like" what you would see in an object oriented system. Which is to say: there is an generic function dispatch based on the type of the first argument to the function. This can be _used_ to write in an OO style.
clatan 2y ago
I trust the authors immensely but i don't see what yet another class system in R solves. That's on me, but I'd like to understand more of what motivates this effort.
- usgroup 2y ago
  From the article: “S7 is a new OOP system being developed as a collaboration between representatives from R-Core, Bioconductor, tidyverse/Posit, ROpenSci, and the wider R community, with the goal of unifying S3 and S4 and promoting interoperability.” It then goes on to describe what that means in depth.
  - clatan 2y ago
    I can read, thak you, and no it doesn't. It describes 3 new generics in base R that help their new S7 system. It all seems motivated by better interop with python which is 'neat' but really doesn't seem like a critical necessity of the language. I guess it's more of a tactical thing where they're trying to make it easier for python users to eventually try R. Or for R users that work alongside python users to not abandon R.
    - t-kalinowski 2y ago
      The Python interop is in the blog post because it makes for convenient and compact examples, not because it motivated any of the features. If you're interested in what motivated S7, you may enjoy this talk Hadley gave: https://www.youtube.com/watch?v=P3FxCvSueag https://www.youtube.com/watch?v=P3FxCvSueag (R7 was the working name for the package at the time)
dmead 2y ago
This seems like a huge mess. Why is it so hard for R people to settle on a standard?
- mr_toad 2y ago
  https://xkcd.com/927/ https://xkcd.com/927/
- jcheng 2y ago
  There's a few reasons I can think of. First, R definitely needs at least two, one for functional OOP (Common Lisp style) and one for class-based OOP (Java style). The latter is _much_ less important for everyday R users but as a package author it's extremely helpful for modeling certain types of resources. (Interestingly, Python also ships with two: @singledispatch and classes; and multimethod/multidispatch also exist.) Second, because R's basic language building blocks are so flexible, it's relatively easy to build new OOP systems, resulting in more diversity. Third, I believe it's actually been close to thirty years since S4 was introduced, which was the last functional OOP system until S7. I don't think that's a terrible track record, compared to how much variety you see in equally fundamental systems in other language communities (just off the top of my head, Python: packaging standards, environment management, data frames; JavaScript: module systems, runtimes, package managers).
kgwgk 2y ago
Nice. This is a bit disappointing though: “Multiple dispatch is heavily used in S4; we don’t expect it to be heavily used in S7, but it is occasionally useful.”
- hadley 2y ago
  Why is that disappointing?
  - kgwgk 2y ago
    Maybe disappointing is not a good way to describe it but I couldn’t find a better word. I meant that it seems that multiple dispatch won’t be highlighted. I would have liked to see a better S4 - fixing some of its issues and adding things like before/after/around method - and I’m not sure this goes in that direction. It can still be an improvement in practice over the rarely-used S4 though.
CornCobs 2y ago
Something interesting I realized about their choice of name - S7. 1. It's a combination of S3 and S4 obviously. 2. It's also linked to another OOP system called R6. Interesting how it's a step forward one way (6->7) and a step 'backwards' in another way (S->R). To me it shows the philosophy of not creating something entirely new but improving the existing systems quite nicely!

109 points by samch93 2y ago

To be honest, OOP never really seemed like a good fit for R. Functional programming is a much more natural fit, given that both it and R come from a mathematical point of view. R is a great language for the mathematical/statistical stuff it was invented to do, but I don't think it will ever be a general-purpose language, and it probably would become worse at its core purpose if it tried to. More work on being easily used by/incorporated into applications written in other languages, would perhaps be a more impactful thing to work on.

bachmeier 2y ago
> More work on being easily used by/incorporated into applications written in other languages, would perhaps be a more impactful thing to work on. That's basically a solved problem. For instance, RInside opens a C interface that can be called by any language that can call C functions, which is basically every language. It's efficient, too, because you're only passing pointers around. Here's an example in Ruby (disclaimer that I wrote it): https://github.com/eddelbuettel/rinside/blob/master/inst/examples/c_interface/passdata.rb https://github.com/eddelbuettel/rinside/blob/master/inst/exa...
orhmeh09 2y ago
OOP has been a critical part of real-life R for a long time, especially in complex implementations of classes of kernels, algorithms, and so on with S4, and more general-purpose with R6. Without these frameworks it would be difficult to implement them. Personally I find it more expressive for general-purpose computation than Python. The "fs" library is much better at working with files and paths than Python "os" and the multiple other modules that can be needed to work with with typical filesystem operations -/ especially if you are working with more than one file at a time. I would even say that each of the R object systems is more expressive and more flexible than the Python one. I suspect lazy evaluation is a part of this.
- fire_lake 2y ago
  I too much prefer R to Python (it’s far more expressive, for one) however it’s clear now that Python has “won” in this space and R is a tough sell to a wider team.
  - bachmeier 2y ago
    > it’s clear now that Python has “won” in this space and R is a tough sell to a wider team R has always been a language for academics, and it continues to be popular in that domain, with no compelling reason to switch. It has seen usage in the private sector, but that has never been the driving force behind R's development or ecosystem, and I doubt it ever will be. For academics, even if a particular function is only available for Python, it's easy enough to call it from R and do everything else in R.
  - orhmeh09 2y ago
    Oh, I agree on that, and I know that most of the time I'll be asked why something isn't in Python so I mainly reserve it for when I am sure nobody will ask "why R?" :-) There is a benefit nowadays that I can rely on Python>=3.6 to be available by default anywhere I am deploying whereas R has to be installed in some way, so like Bash it's part of a toolbox I can rely on being available with at least a constrained set of features.
  - rossdavidh 2y ago
    My experience is that you can "sell" R when the statistical or modelling technique is not (or not well) implemented in python. Which still includes a lot of potentially useful statistical techniques! R should/could lean into being the tool that python programmers reach for when they need something a little less mainstream, if they made it easier for non-R programmers to do so.
- agumonkey 2y ago
  > I suspect lazy evaluation is a part of this. I had no idea R was lazy. Makes me wanna learn it now.
  - 2y ago
andrewla 2y ago
OOP as used in R is very much a function of API design and not a function of routine R usage for data analysis. To many users of R they are not even aware that they are using OOP at all, especially for the S3 style of objects. When you have an object, like `model <- lm(x~y)` or `my_hist <- hist(df$foo)`, you expected to be able to `plot` it or get a `summary`; you don't call `my_hist.summary`, you call `summary(my_hist)` and `plot(model)`. Many users never look further under the hood than this. And this fits nicely into piped workflows -- `lm(x~y) |> summary()` ends up being very natural, and when you fit in the tidyverse operators many very complex workflows end up being very easy to digest. But when you do pull back the kimono it gets ugly fast. The teams involved in this are the right people who have been working to make R an amazing language mostly through enhancements to libraries, and now they're trying to push some of that functionality back into core R, which I think is fantastic.
- th0ma5 2y ago
  Some detail about that phrase https://www.catalyst.org/2021/03/22/racism-misogyny-asian-american-women-workplace/ https://www.catalyst.org/2021/03/22/racism-misogyny-asian-am...
  - Onawa 2y ago
    I can honestly say that I had never heard that phrase used before now, but I do know I felt icky when I read it in the comment before I even clicked on your link. Definitely glad to see it is being called out, terms like this absolutely need to be removed from modern discourse.
    - deleted 2y ago
      [deleted]
  - chuckadams 2y ago
    Pretty sure the phrase GP was looking for was "pull back the curtain", which likely originated with The Wizard of Oz.
- clatan 2y ago
usgroup 2y ago
Yet you’re using OOP every time you call plot, predict, summary and so on: possibly without realising.
- CornCobs 2y ago
  I actually think this "multi-tiered" system of OOP is quite cool, when compared to languages that stick OOP in your face upfront. 1. Basic users don't even know it's there, they're just calling regular functions. 2. S3 in base is super simple to understand and easy to extend the first time you need to implement your own summary. 3. Full blown OOP with slots and methods is available when you really need it (rare for a user and not library author imo, lists and S3 are sufficient for most things). The big issue I see is the incompatibilities in the various systems making this "ramp up" not so smooth. But it looks like that's what S7 is trying to address so that's cool.
PheonixPharts 2y ago
Functional programming is not orthogonal to object oriented programming, they are paradigms that can be used together and the popular object system developed in Java is not close to the only way to do OOP. R, like Common Lisp, uses an OOP system based on generic functions (well, one of many OOP systems in R, but that's a different topic), where the function handles dispatching to match the object. Effectively instead of: object.method() you have: method(object) So it works perfectly with the functional paradigm while still being capable of everything an object system is. The most obvious example of this in R is the `plot` function. You as the programmer don't have to know exactly how to plot an instance of a class, you just pass it into `plot` and it will be handled correctly. If you create a new class you just have to extend the definition of plot in a standard way and it will also be handled for you. It's a shame that the many flavors of OOP remain relatively unknown to most programmers, and in many of the cases where they're tried (JavaScript's prototype system for example) people have essentially replaced them with more familiar systems.
- epgui 2y ago
  > Effectively instead of object.method() you have method(object) This reflects a very deep misunderstanding of the distinctive characteristics of each paradigm. It's so far off that it's "not even wrong". Functional programming is much more about things like purity and referential transparency, about composing functions and/or combinators, about a particular way of managing or modelling effects, about a way of thinking, about using certain kinds of data structures and algorithms. It's not a syntactical difference.
  - fn-mote 2y ago
    >> Effectively instead of object.method() you have method(object) > This reflects a very deep misunderstanding of the distinctive characteristics of each paradigm. I think the parent made a hasty reading of the GP comment. The GP shows an awareness of multiple OO systems in R. I believe the GP is attempting to explain to a Java programmer how R could be considered object-oriented even though `plot(item)` does not "look like" what you would see in an object oriented system. Which is to say: there is an generic function dispatch based on the type of the first argument to the function. This can be _used_ to write in an OO style.

clatan 2y ago

I trust the authors immensely but i don't see what yet another class system in R solves. That's on me, but I'd like to understand more of what motivates this effort.

usgroup 2y ago
From the article: “S7 is a new OOP system being developed as a collaboration between representatives from R-Core, Bioconductor, tidyverse/Posit, ROpenSci, and the wider R community, with the goal of unifying S3 and S4 and promoting interoperability.” It then goes on to describe what that means in depth.
- clatan 2y ago
  I can read, thak you, and no it doesn't. It describes 3 new generics in base R that help their new S7 system. It all seems motivated by better interop with python which is 'neat' but really doesn't seem like a critical necessity of the language. I guess it's more of a tactical thing where they're trying to make it easier for python users to eventually try R. Or for R users that work alongside python users to not abandon R.
  - t-kalinowski 2y ago
    The Python interop is in the blog post because it makes for convenient and compact examples, not because it motivated any of the features. If you're interested in what motivated S7, you may enjoy this talk Hadley gave: https://www.youtube.com/watch?v=P3FxCvSueag https://www.youtube.com/watch?v=P3FxCvSueag (R7 was the working name for the package at the time)

dmead 2y ago

This seems like a huge mess. Why is it so hard for R people to settle on a standard?

mr_toad 2y ago
https://xkcd.com/927/ https://xkcd.com/927/
jcheng 2y ago
There's a few reasons I can think of. First, R definitely needs at least two, one for functional OOP (Common Lisp style) and one for class-based OOP (Java style). The latter is _much_ less important for everyday R users but as a package author it's extremely helpful for modeling certain types of resources. (Interestingly, Python also ships with two: @singledispatch and classes; and multimethod/multidispatch also exist.) Second, because R's basic language building blocks are so flexible, it's relatively easy to build new OOP systems, resulting in more diversity. Third, I believe it's actually been close to thirty years since S4 was introduced, which was the last functional OOP system until S7. I don't think that's a terrible track record, compared to how much variety you see in equally fundamental systems in other language communities (just off the top of my head, Python: packaging standards, environment management, data frames; JavaScript: module systems, runtimes, package managers).

kgwgk 2y ago

Nice. This is a bit disappointing though: “Multiple dispatch is heavily used in S4; we don’t expect it to be heavily used in S7, but it is occasionally useful.”

hadley 2y ago
Why is that disappointing?
- kgwgk 2y ago
  Maybe disappointing is not a good way to describe it but I couldn’t find a better word. I meant that it seems that multiple dispatch won’t be highlighted. I would have liked to see a better S4 - fixing some of its issues and adding things like before/after/around method - and I’m not sure this goes in that direction. It can still be an improvement in practice over the rarely-used S4 though.

CornCobs 2y ago

Something interesting I realized about their choice of name - S7. 1. It's a combination of S3 and S4 obviously. 2. It's also linked to another OOP system called R6. Interesting how it's a step forward one way (6->7) and a step 'backwards' in another way (S->R). To me it shows the philosophy of not creating something entirely new but improving the existing systems quite nicely!