9 ms·
More intuitive partial function application in Python
- heavyset_go 5y agoI like how clever this is. It's a little too clever for production code for my tastes, but it might be nice to use in exploratory programming.
- cgrimm1994 5y agoI really appreciate that :) Definitely give it a spin and let me know what you think!
- BiteCode_dev 5y agoWhile it does solve the problem with functools.partial() forcing you to use keyword arguments in some cases, I don't think currying is a common enough operation to warrant a whole lib, or even a new syntax. I'd rather see partial() promoted to a universal static method attached to any callable, so it's easier to discover, and use.
- cgrimm1994 5y agoI guess it depends what kind of stuff you're working on. I do a lot of work with jax for my research (https://github.com/google/jax https://github.com/google/jax) and use functools.partial a lot.
- cgrimm1994 5y agoHey, I use functools.partial a lot in my projects and I've always felt like it was pretty limited in the types of partial function application it could accomplish. I put together a modified version of partial over the weekend that I wanted to share / get feedback on. Let me know what you all think!
- codethief 5y ago`functools.partial` has been bothering me for a long time and I've always thought it'd be nice to have something like the placeholder `·` that people use in math (i.e. `f(·, y, z)`). It didn't occur to me that a decorator might actually get us there without having to introduce new syntax. Nice project!
- cgrimm1994 5y agoI'm glad you like it! This project was basically me trying to visualize how I wish functions generally worked like in Python.
- memco 5y agoWhat specifically bothers you about functools partial? I think the decorator is nice in certain ways. However, then you have to decorate your code and guess where you’re going toy need the partials or create wrappers just to add the decorator which seems undesirable for a number of cases. I agree that partial has warts when it comes to inspection but at least they’re easy to make anywhere you want without having to apply any changes to the callee. Nice to have another option for partials though so that when you need these features there’s a way to get them.
- cgrimm1994 5y agoI posted this below, but I can put it here too: https://gist.github.com/chrisgrimm/64bca66f14528cfda6d865cc21fe77d4 https://gist.github.com/chrisgrimm/64bca66f14528cfda6d865cc2... Essentially functools.partial is somewhat limited in terms of which arguments can be partially applied.
- akdor1154 5y agoI was gonna say cool for FP guys but alien to the Python world, but then i saw the f(..., x=1) syntax and decided you're a genius. Really nice idea. In practice i worry about static checkers and IDE param hints though, how have you fared with these? Not your fault at all, but mypy/pyright have very poor support for wrappers like this, and in IDEs it's common for their arg hints to devolve to a copout (args, kwargs) as well.
- heavyset_go 5y ago> Not your fault at all, but mypy/pyright have very poor support for wrappers like this, and in IDEs it's common for their arg hints to devolve to a copout (args, kwargs) as well. You might be able to annotate it accurately with generics, ParamSpec and Concatenate[1]. [1] https://www.python.org/dev/peps/pep-0612/ https://www.python.org/dev/peps/pep-0612/
- cgrimm1994 5y agoI'll give this a look.
- cgrimm1994 5y agoHigh praise :-) I'm glad you like it! I don't know much about static checks / param hinting. TBH, I coded this up because I wanted to imagine how functions should work in Python as more of a fun exercise. Accordingly there are a lot of things I have to do to make better_partial a strong tool for the community. I'll probably have time to work on the project again later in the week.
- civilized 5y agoCould this be used without the "bp."? Half the point of coming up with a nicer notation is for the notation to actually be nicer.
- mrtranscendence 5y agoWell, you could do from better_partial import partial, _ presumably. Though since _ has some meaning in Python maybe you'd want to do from better_partial import partial, _ as x instead (or something like it).
- cgrimm1994 5y agoAs pointed out, you could import _ directly into your namespace, or map it to some other shorthand name that doesn't interfere with your naming conventions. Also note that the _ placeholder isn't the only way that better_partial helps you. You can also use ... to indicate that you want to omit all arguments except ones you explicitly pass as kwargs. For instance: from better_partial import partial, _ @partial def f(a,b,c,d,e): return (a,b,c,d,e) f(_,_,3,_,_)(1,2,4,5) == f(..., c=3)(1,2,4,5)
- tacosbane 5y agohttps://toolz.readthedocs.io/en/latest/curry.html https://toolz.readthedocs.io/en/latest/curry.html
- seanhunter 5y agoThis is a bit of a personal bugbear, but partial function application is not currying. I wrote a blog post on this a while ago because it's such a common misconception. https://www.uncarved.com/articles/not-currying/ https://www.uncarved.com/articles/not-currying/ The link you posted is just using the word wrong. Currying is not syntactic sugar for partial application, it is the opposite. (ie currying builds functions which take multiple arguments from chains of functions which each take a single argument)
- cgrimm1994 5y agoYes! I almost made this mistake. Although I'm not sure @partial is the most apt name for the decorator, as it produces a function which can be partially applied. In retrospect, @partialize or something like it might be more appropriate.
- d0mine 5y ago@curryable looks the same as @partializable: @curryable def add(a, b): return a+b add2 = add(..., 2) # curried print(add2(3)) # 5
- cgrimm1994 5y agoI think the similarity ends when you consider functions with more than 2 arguments. Let's compare a 2 argument function with a 3 argument function: @partial def f3(x, y, z): return (x, y, z) @partial def f2(x, y): return (x, y) A curried version of f2 would look like: curried_f2 = lambda x: lambda y: f2(x, y) curried_f2(1)(2) == (1, 2) Currying produces a chain of functions which each take one argument. So its true for f2 that currying and the partial decorator are similar: curried_f2(1) == f2(1, _) == f2(..., x=1) as you pointed out. But now consider the 3 argument case: curried_f3 = lambda x: lambda y: lambda z: f3(x, y, z) curried_f3(1) != f3(1, _, _) this is because f(1, _, _) takes two arguments whereas curried_f3(1) takes one.
- maximilianroos 5y agoGreat to see this! Thanks for working on it. Could you compare it to functools.partial? In the example on the Readme: import better_partial as bp @bp.partial def some_operation(x, p1, p2): return (x + p1) * p2 It claims that the bp approach is superior because: func = some_operation(bp._, 10, 20) ...is better than: func = lambda x: some_operation(x, 10, 20) But is it better than just: partial(some_operation, p1=10, p2=20) ?
- R0b0t1 5y agopartial(some_operation, p1=10, p2=20) Suffers from inconsistent syntax compared to normal function call. This is probably a negative, but it could be a positive.
- cgrimm1994 5y agoYes! I definitely need to beef up the README. I wrote up a gist for some other commenters below which I can incorporate as part of the justification. https://gist.github.com/chrisgrimm/64bca66f14528cfda6d865cc21fe77d4 https://gist.github.com/chrisgrimm/64bca66f14528cfda6d865cc2...
- MaxMoney 5y agowhats wrong with functools.partial? I don't see that anywhere in the arguments.. all you say is "I find functools.partial unintuitive".. why? its a simple wrapper.
- cgrimm1994 5y agohere's an example of one of the behaviors of functools.partial that I find unintuitive / undesirable: https://gist.github.com/chrisgrimm/64bca66f14528cfda6d865cc21fe77d4 https://gist.github.com/chrisgrimm/64bca66f14528cfda6d865cc2...
- MaxMoney 5y ago
- toomanydoubts 5y agoInteresting, but I'm wondering how does partial application for external functions works. Do you need to redefine the functions using the decorator?
- cgrimm1994 5y agoOne option would be to wrap the functions from lib import external from better_partial import partial as pp, _ pp(external)(..., arg4=10)
- raymondh 5y agoCurrent solutions: >>> list(map(partial(pow, exp=3), range(10))) [0, 1, 8, 27, 64, 125, 216, 343, 512, 729] >>> list(map(lambda base: pow(base, 3), range(10))) [0, 1, 8, 27, 64, 125, 216, 343, 512, 729] >>> [pow(base, 3) for base in range(10)] [0, 1, 8, 27, 64, 125, 216, 343, 512, 729] Proposed solution: list(map(bp.partial(pow)(bp._, 3), range(10)) ISTM the only time you come out a little ahead is if you know in advance that a function is going to be partialed (so you can decorate it), that you will need to partial arguments in other than a left to right fashion (otherwise a plain partial will suffice), that you won't use keyword arguments (the current partial works with keywords), and that you really dislike the lambda keyword (which neatly covers all cases from simple to complex without a new notation). That said, this is a clever recipe. Kudos to the author for putting it together. I don't think it really rises to the level of "better" or "more intuitive", but it is an interesting experiment.
- cgrimm1994 5y agoI'm glad you like the recipe :-) What if you have a function "f" that takes like 10 arguments and you need to pass it to a function "g" that expects as input a function takes 8 arguments. @partial def f(p1, p2, p3, p4, p5, p6, p7, p8, p9, p10): return "stuff" g(f(..., p4=1, p7=2)) # seems nicer than g(lambda p1, p2, p3, p5, p6, p8, p9, p10: f(p1, p2, p3, 1, p5, p6, 2, p8, p9, p10))
- wheelerof4te 5y agoFirst mistake, making a function with 10 parameters. We have data structures for that, if you need it. Just pass in a big struct/named tuple as one parameter. Bonus points if it is type hinted.
- d0mine 5y agoNit: List comprehension looks better for expressions (preferable over lambda+map)
- blindseer 5y agoI love the idea of using underscore to represent partial currying. Somewhat related, here's a similar PR for native support of this feature in Julia: https://github.com/JuliaLang/julia/pull/24990 https://github.com/JuliaLang/julia/pull/24990. In my opinion, I'd love to see this baked in as part of the language. Getting buy in from my team to use this in production is just not going to happen. But cool package though!
- cgrimm1994 5y agoThanks! I would love to see it baked in as well. It'd be nice if, (1) all functions defaulted to behaving like better_partial.partial decorated functions and (2) similar to Ellipsis, there was some other special sentinel that we could use in place of _ (to avoid conflicting with the convention of using _ for unused variables).
- throwaway81523 5y agoI'm sure it's nice, but adding another dependency to save some keystrokes in an edge case --- no thanks. If I want lots of partial application I'll use Haskell.
- rmbyrro 5y agoAlso thought the benefit too small for the drag of another dependency.
- deleted 5y ago[deleted]
- wirthjason 5y agoCan these be serialized? One of my main uses for partial is passing things to multiprocessing pool.map. People often reach for lambda but it cannot be pickled/serialized so it fails.
- cgrimm1994 5y agoI'll look into this soon. I honestly wasn't expecting this much traffic on what essentially was a weekend project, so I'll need to spend a bit of time making the code less insane and testing for things like pickle-ability.
- tpoacher 5y agoThis is great. I will use it with my python chaining approach: https://github.com/tpapastylianou/chain-ops-python https://github.com/tpapastylianou/chain-ops-python My only grief is that decorator, which forces you to wrap existing functions anyway (same way I had to define lambdas in my example anyway). Do you have any insight on that?
- cgrimm1994 5y agoI'm glad you like it! I agree it's not optimal to have to wrap existing functions. One option would be to decorate the functions as they're defined, although you cant do this for external functions. In my perfect world, python functions would default to behaving as they do when decorated with better_partial.partial, but it seems like a long-shot to try to get something like this added to the language itself.
- throwamon 5y agoThis is not a new idea. siuba (a dplyr-like Pandas alternative) is one project that does it, and it also has a "pipeline operator": https://github.com/machow/siuba https://github.com/machow/siuba (mtcars >> group_by(_.cyl) >> summarize(avg_hp = _.hp.mean()) ) (Edit: ok, it seems siuba doesn't have the "parameter fixing" thing.) Related: Coconut - a functional superset of Python - https://github.com/evhub/coconut https://github.com/evhub/coconut range(10) |> map$(pow$(?, 2)) |> list There were others but I can't remember which now.
- cgrimm1994 5y agoSure, I'm not claiming to have originated the idea of using _ or some other placeholder to return partially applied functions. Lots of languages have some kind of shorthand syntax for this. I just hadn't seen it done in Python before. Coconut looks cool btw! I'll give it a closer look soon.
- throwamon 5y agoSorry, I should have worded my comment differently, I didn't mean to sound accusatory or dismissive.
- cgrimm1994 5y agoNo worries!
- probablypower 5y agoThe work you've done is nice, but using a package for this issue gives off bad code smell. I'd much rather refine my functions and their relations rather than require an extra dependency that reduces readability for my coworkers.
- cgrimm1994 5y agoThanks for the feedback! This project was really me just imagining how I'd like functions to behave in Python. I didn't really expect there to be this much interest.
- cabalamat 5y ago> but under the hood my partial decorator is working all kinds of magic I'm sure it is! That's why I'd be leery of using it in production code.
- cgrimm1994 5y ago:-)