7 ms·

Code blocks in Python

49 points by mtomassoli 14y ago

kibwen 14y ago
I'd like to see some before-and-after examples of Python code that's been rewritten to take advantage of blocks using this module.
bcl 14y ago
Seems like a good way to make your code harder to read and harder to debug.
- stcredzero 14y ago
  Anonymous functions can generally be abused in ways that make code harder to read. Then again, so can recursion. It's a double edged sword.
Goladus 14y ago
Interesting, but I'm having a hard time seeing how this syntax saves me much in terms of code size and complexity compared to just wrapping the code blocks into named functions and passing those.
- Peaker 14y ago
  The problem with that is that it forces moving the code above where it is passed, which can greatly clutter things. For example, consider the function "forever" in Haskell: forever action = do action forever action Now you can use it as a new control structure, e.g: forever $ do (sock, addr) <- accept listener forkIO $ handleClient sock The $ simply means "apply" and is low-precedence, so it removes the need to put () around the entire argument to be applied. In Python, you could define: def forever(action): action() forever(action) but then, to use it, you have to give a name to your function, so: def accept_once(): (sock, addr) = listener.accept() fork(partial(handleClient, sock)) forever(accept_once) This makes "forever" much less useful as a new control structure/looping primitive. In this sense, Python makes DSLs less usable. The built-in primitives are first-class, and can have code directly within their use. Library functions are second-class, and can only have code passed by name which must then fully appear before the use. Another example is callbacks. The reason "twisted" is probably called "twisted", and that people hate callbacks so much, is that it forces writing the code in backwards order, precisely because of this problem. For example: def handle_result(result): print "Done:", result def connection_started(conn): conn.request(SomeRequest(), handle_result) def start(): start_connecting(connection_started) Compare this with Haskell, as an example: start = startConnecting $ \conn -> do request conn SomeRequest $ \result -> do putStrLn $ "Done: " ++ show result Note, in Haskell, this would actually be worked out to be (by overloading the semicolon): start = do conn <- startConnecting result <- request conn SomeRequest putStrLn $ "Done: " ++ show result But even the former nested representation is better than the backwards ("twisted") representation that makes people hate callbacks so much.
  - viraptor 14y ago
    This seems like a solution in search of a problem. There is a nice 'forever' construct. It's called 'while': while True: action() I know what you mean about generalising this to other control structures, but it looks like the whole concept starts from "I want to write 'forever' that resembles something from other languages", rather than "I want to write a loop". There are existing ways to write things like that, so is do we really need to force something from other languages into Python? What about some examples which cannot be easily handled - maybe the solution for them is something completely different than porting codeblocks.
    - stcredzero 14y ago
      You are missing the point. The forever structure is just a code example. What's missing is the ability to write your own control structures generally. (Some of which won't already exist.)
      - viraptor 14y ago
        That is exactly what I meant - this is a synthetic example. The article has synthetic examples. People want code blocks, but no one is really showing why. Own control structures are cool, ok, but what are you trying to solve with them? Where are the real, convincing before and after examples that are not easily solvable otherwise?
        Peaker 14y ago
        Here's some Haskell code I really like, and I think it would be far less nice in Python: replicateM 10 . forkIO . forever $ do .. Which is basically the same as: replicateM 10 (forkIO (forever (do ..))) replicateM 10 executes its code block argument 10 times. forkIO executes its code block argument in a new thread. forever loops forever. So this line basically creates a thread pool with 10 threads, all infinitely executing the given code block. How would you solve this in Python?
        14y ago
francoisdevlin 14y ago
This is a terrible idea, and ruby used the wrong implementation. If you want this behaviour, you should just define a higher order function/decorator, and put your "block" in its own function. This is a case where there should be only one way to do it.
- jamesgeck0 14y ago
  I agree that this doesn't fit Python well, but why was it bad in Ruby?
  - francoisdevlin 14y ago
    Because of non local returns. People abuse them when instead they should be using an exception, or preprocessing their data better.
- stcredzero 14y ago
  Blocks in other languages can help maintain encapsulation, if they can access variables from their local scope. They can also enable programmers to write their own control structures which look like natural first class control structures in the host language. I don't see this as a natural way of doing either.
  - sirclueless 14y ago
    Nested functions can access variables from local scope in Python, so this isn't a huge issue. It's a bit wonky in python (see discussion of the nonlocal keyword, Python did this wrong from the start) but it solves your encapsulation problem. def sorted_by(xs, attr): def cmp_attr(a, b): return cmp(getattr(a, attr), getattr(b, attr)) return sorted(xs, cmp_attr) It takes a little more vertical space and requires you to give a name to something that might otherwise be anonymous, but in the long term it's actually beneficial in my opinion. I've actually taken to using the same pattern in my JavaScript development, giving all of my callbacks names, because it makes maintenance so much easier later. var load_into = function (url, elem) { var handle_response = function (response) { $(elem).html(response); }; $.get(url, handle_response); };
lee 14y ago
Can someone give me an example where using a code block is better than using a simple function? And is the added complexity and loss of readability worth it?
- wslh 14y ago
  If the code block can run in a different thread (like in Grand Central Dispatch) it is very useful to do stuff in the background without blocking the GUI (that executes in a specific thread) and using the current scope variables. A progress bar is a clear example of this. Doing this without code blocks adds extra code like launching another thread, connecting the scope to the new thread.
  - lee 14y ago
    Sorry, I'm not following. Why couldn't you use a function here instead of a code block to do the exact same thing?
- d4nt 14y ago
  I'm not sure there is one. They're useful in C# as a way of defining a function within another function (which sometimes makes for more readable code). For example: public int CountRelevantItems(IEnumerable<Thing> things) { // Non-trivial filter that you don't want in a where lamdba Func<Thing, bool> isRelevant = (t) => { ... } return things.Where(t => isRelevant(t)).Count() } But in Python you can already define functions inside of functions so you have a better solution: def CountRelevantThings(things): def isRelevant(thing): ... return len(filter(isRelevant, things))
  - mhurron 14y ago
    > They're useful in C# as a way of defining a function within another function (which sometimes makes for more readable code) Why wouldn't you just create another function in the same class?
    - 14y ago
rbanffy 14y ago
Adding blocks to Python feels like adding classes to JavaScript... Do we really need them?
- qznc 14y ago
  No. You can always use "named blocks", but they are called functions.
lucian1900 14y ago
Adding function literals to Python would be nice. Something to turn "inc = lambda x: x + 1" into "inc = def(x): return x + 1". But blocks? They're a misfeature of Ruby, badly copied from Smalltalk. [edit: added "return"]
- recursive 14y ago
  It looks like you changed "lambda" to "def" and added parentheses. It seems like that feature is already there...
  - lucian1900 14y ago
    Perhaps my example was crap. They would also allow anything inside them.
    - sirclueless 14y ago
      The problem with multiline lambdas it that no one knows a good way to indent them. If they weren't butt-ugly python would probably already have them.
      - stcredzero 14y ago
        "The problem with multiline lambdas in Python is that no one knows a good way to indent them." How about new keyword(s)? "begin" and "end"? I imagine this has thoroughly been discussed ad-nauseum. Is there a good synopsis?
        recursive 14y ago
        Basically, there is no scope or block ending token anywhere else in python. And Guido thinks inline multi-line anonymous functions make programs harder to understand.
      - lucian1900 14y ago
        I'm not sure it's that big a problem. CoffeeScript handles this pretty well using indentation. CS has ambiguous cases because of optional ( ), but that wouldn't be the case with Python anyway.
llimllib 14y ago
I did something very similar 3 years ago: http://billmill.org/multi_line_lambdas.html http://billmill.org/multi_line_lambdas.html
- mtomassoli 14y ago
  Oh, I know. My module is your fault! :)
  - llimllib 14y ago
    I really like the << x syntax, it's devious :)
northisup 14y ago
This seem like a whole series of posts on 'how to twist python syntax to save a line of code or two and make it unreadable'.
fendrak 14y ago
This is all well and good, but it doesn't appear to add support for the one thing I've often wanted in Python: proper closure support for functions within functions.
- d0mine 14y ago
  Do you mean 'nonlocal' keyword?
scott_s 14y ago
The code examples are confusing. First, I think there is a typo in the first two examples; I think < should be <<. But he never actually explains what the semantics of << are until several examples in, and it took me a long time to figure out that the string value 'x, y=3' gets evaled into code. Which is strange, to mix strings as code, when the whole point is to have code be code. At the very least, the semantics of this need to be explained better. When you give a new code example, you always need to say "And this is the result." Otherwise, I can't close the loop; I have new code with new and unknown semantics, and an unknown result. I need to have a known result to figure out the new semantics. (Much like you can't solve a single equation with two unknowns; you either need to have one unknown, or two equations.)
- mtomassoli 14y ago
  Just read the docstring of the module. It's very detailed. As for mixing strings with code, I still need to adhere to Python's syntax in order to avoid syntax errors. Anyway, syntax errors in strings are caught at "rewriting time". Using words instead of operators would also cause some problems.
  - f0r 14y ago
    Should it be "<< 'x'" rather than "<< 2"?
    - mtomassoli 14y ago
      I don't believe it. Let's try to fix it again... thanks.
  - scott_s 14y ago
    You're putting the carriage before the horse. People want to evaluate what it is they're getting into before they download it. If they can't understand the code examples, they won't get as far as downloading the module and looking at the docstring. I understand why you had to use strings as code. I had two concerns about that. One, you didn't explain it - I had to figure it out on my own. And two, if a solution is supposed to make things cleaner, but ends up introducing warts like that... maybe it's not worth it. What you implemented is interesting, no doubt, but I wouldn't want to use it for that reason. However, I think you would benefit greatly from having improved examples. Even if people don't use your module, they can still build on your ideas if it's well explained.
optymizer 14y ago
"code blocks" is why they invented functions and procedures half a century ago (and even those differ only in how they return). I think people forget (or don't know) that these are all labels to jump or branch to. So your named functions, functions, lambdas, code blocks, etc, become.. well.. equivalent. Hipsters, don't confuse the young ones with your 30 ways of jumping into a block of code. Cheers!
- stcredzero 14y ago
  Not the whole story. Code blocks can be used to protect encapsulation and create DSLs which are first class citizens of their host language. So your named functions, functions, lambdas, code blocks, etc, become.. well.. equivalent. They can also become so lexically awkward as to be unusable. For example, one code a toy "fuzzy logic" system in Smalltalk with a handful of methods. The result looks like a 1st class member of the language, just like the control structures that are already there. (Same goes for the loops) "Here is the standard if-else" (condition) ifTrue: [ ... ] ifFalse: [ ... ] "Here is what my DSL can look like" (condition) ifTrue: [ ... ] ifFalse: [ ... ] ifMaybe: [ ... ] Doing this with named functions is going to scatter code between different functions. It's semantically equivalent, but harder to read. Write anything involved in such a way, and it becomes untenable. Blocks can make such DSLs an order of magnitude more readable. Hipsters, don't confuse the young ones with your 30 ways of jumping into a block of code. Cheers! A good heaping fraction of hipsters don't understand what's good about code blocks.
  - optymizer 14y ago
    "Code blocks can be used to protect encapsulation". So can functions, which are already in the language. "Doing this with named functions is going to scatter code between different functions". Please show an example. "A good heaping fraction of hipsters don't understand what's good about code blocks." That's one way of phrasing it.
bicknergseng 14y ago
I also question the utility and reason behind this. I spend a lot of time fixing memory leaks from lousy developers who use closures and anonymous functions incorrectly in JavaScript. I'm just learning Ruby as well, and see a lot of room for similar black-boxing pitfalls in procs and blocks. It seems to me like you're reducing code size at the expense of readability and debugability. That might be the ruby way, but it seems to me like a bad way to program. Like I said, I'm just learning... so I'd love counterexamples and reasons why I'm wrong.

49 points by mtomassoli 14y ago

I'd like to see some before-and-after examples of Python code that's been rewritten to take advantage of blocks using this module.

bcl 14y ago

Seems like a good way to make your code harder to read and harder to debug.

stcredzero 14y ago
Anonymous functions can generally be abused in ways that make code harder to read. Then again, so can recursion. It's a double edged sword.

Goladus 14y ago

Interesting, but I'm having a hard time seeing how this syntax saves me much in terms of code size and complexity compared to just wrapping the code blocks into named functions and passing those.

Peaker 14y ago
The problem with that is that it forces moving the code above where it is passed, which can greatly clutter things. For example, consider the function "forever" in Haskell: forever action = do action forever action Now you can use it as a new control structure, e.g: forever $ do (sock, addr) <- accept listener forkIO $ handleClient sock The $ simply means "apply" and is low-precedence, so it removes the need to put () around the entire argument to be applied. In Python, you could define: def forever(action): action() forever(action) but then, to use it, you have to give a name to your function, so: def accept_once(): (sock, addr) = listener.accept() fork(partial(handleClient, sock)) forever(accept_once) This makes "forever" much less useful as a new control structure/looping primitive. In this sense, Python makes DSLs less usable. The built-in primitives are first-class, and can have code directly within their use. Library functions are second-class, and can only have code passed by name which must then fully appear before the use. Another example is callbacks. The reason "twisted" is probably called "twisted", and that people hate callbacks so much, is that it forces writing the code in backwards order, precisely because of this problem. For example: def handle_result(result): print "Done:", result def connection_started(conn): conn.request(SomeRequest(), handle_result) def start(): start_connecting(connection_started) Compare this with Haskell, as an example: start = startConnecting $ \conn -> do request conn SomeRequest $ \result -> do putStrLn $ "Done: " ++ show result Note, in Haskell, this would actually be worked out to be (by overloading the semicolon): start = do conn <- startConnecting result <- request conn SomeRequest putStrLn $ "Done: " ++ show result But even the former nested representation is better than the backwards ("twisted") representation that makes people hate callbacks so much.
- viraptor 14y ago
  This seems like a solution in search of a problem. There is a nice 'forever' construct. It's called 'while': while True: action() I know what you mean about generalising this to other control structures, but it looks like the whole concept starts from "I want to write 'forever' that resembles something from other languages", rather than "I want to write a loop". There are existing ways to write things like that, so is do we really need to force something from other languages into Python? What about some examples which cannot be easily handled - maybe the solution for them is something completely different than porting codeblocks.
  - stcredzero 14y ago
    You are missing the point. The forever structure is just a code example. What's missing is the ability to write your own control structures generally. (Some of which won't already exist.)
    - viraptor 14y ago
      That is exactly what I meant - this is a synthetic example. The article has synthetic examples. People want code blocks, but no one is really showing why. Own control structures are cool, ok, but what are you trying to solve with them? Where are the real, convincing before and after examples that are not easily solvable otherwise?
      - Peaker 14y ago
        Here's some Haskell code I really like, and I think it would be far less nice in Python: replicateM 10 . forkIO . forever $ do .. Which is basically the same as: replicateM 10 (forkIO (forever (do ..))) replicateM 10 executes its code block argument 10 times. forkIO executes its code block argument in a new thread. forever loops forever. So this line basically creates a thread pool with 10 threads, all infinitely executing the given code block. How would you solve this in Python?
        14y ago

francoisdevlin 14y ago

This is a terrible idea, and ruby used the wrong implementation. If you want this behaviour, you should just define a higher order function/decorator, and put your "block" in its own function. This is a case where there should be only one way to do it.

jamesgeck0 14y ago
I agree that this doesn't fit Python well, but why was it bad in Ruby?
- francoisdevlin 14y ago
  Because of non local returns. People abuse them when instead they should be using an exception, or preprocessing their data better.
stcredzero 14y ago
Blocks in other languages can help maintain encapsulation, if they can access variables from their local scope. They can also enable programmers to write their own control structures which look like natural first class control structures in the host language. I don't see this as a natural way of doing either.
- sirclueless 14y ago
  Nested functions can access variables from local scope in Python, so this isn't a huge issue. It's a bit wonky in python (see discussion of the nonlocal keyword, Python did this wrong from the start) but it solves your encapsulation problem. def sorted_by(xs, attr): def cmp_attr(a, b): return cmp(getattr(a, attr), getattr(b, attr)) return sorted(xs, cmp_attr) It takes a little more vertical space and requires you to give a name to something that might otherwise be anonymous, but in the long term it's actually beneficial in my opinion. I've actually taken to using the same pattern in my JavaScript development, giving all of my callbacks names, because it makes maintenance so much easier later. var load_into = function (url, elem) { var handle_response = function (response) { $(elem).html(response); }; $.get(url, handle_response); };

lee 14y ago

Can someone give me an example where using a code block is better than using a simple function? And is the added complexity and loss of readability worth it?

wslh 14y ago
If the code block can run in a different thread (like in Grand Central Dispatch) it is very useful to do stuff in the background without blocking the GUI (that executes in a specific thread) and using the current scope variables. A progress bar is a clear example of this. Doing this without code blocks adds extra code like launching another thread, connecting the scope to the new thread.
- lee 14y ago
  Sorry, I'm not following. Why couldn't you use a function here instead of a code block to do the exact same thing?
d4nt 14y ago
I'm not sure there is one. They're useful in C# as a way of defining a function within another function (which sometimes makes for more readable code). For example: public int CountRelevantItems(IEnumerable<Thing> things) { // Non-trivial filter that you don't want in a where lamdba Func<Thing, bool> isRelevant = (t) => { ... } return things.Where(t => isRelevant(t)).Count() } But in Python you can already define functions inside of functions so you have a better solution: def CountRelevantThings(things): def isRelevant(thing): ... return len(filter(isRelevant, things))
- mhurron 14y ago
  > They're useful in C# as a way of defining a function within another function (which sometimes makes for more readable code) Why wouldn't you just create another function in the same class?
  - 14y ago

rbanffy 14y ago

Adding blocks to Python feels like adding classes to JavaScript... Do we really need them?

qznc 14y ago
No. You can always use "named blocks", but they are called functions.

lucian1900 14y ago

Adding function literals to Python would be nice. Something to turn "inc = lambda x: x + 1" into "inc = def(x): return x + 1". But blocks? They're a misfeature of Ruby, badly copied from Smalltalk. [edit: added "return"]

recursive 14y ago
It looks like you changed "lambda" to "def" and added parentheses. It seems like that feature is already there...
- lucian1900 14y ago
  Perhaps my example was crap. They would also allow anything inside them.
  - sirclueless 14y ago
    The problem with multiline lambdas it that no one knows a good way to indent them. If they weren't butt-ugly python would probably already have them.
    - stcredzero 14y ago
      "The problem with multiline lambdas in Python is that no one knows a good way to indent them." How about new keyword(s)? "begin" and "end"? I imagine this has thoroughly been discussed ad-nauseum. Is there a good synopsis?
      - recursive 14y ago
        Basically, there is no scope or block ending token anywhere else in python. And Guido thinks inline multi-line anonymous functions make programs harder to understand.
    - lucian1900 14y ago
      I'm not sure it's that big a problem. CoffeeScript handles this pretty well using indentation. CS has ambiguous cases because of optional ( ), but that wouldn't be the case with Python anyway.

llimllib 14y ago

I did something very similar 3 years ago: http://billmill.org/multi_line_lambdas.html http://billmill.org/multi_line_lambdas.html

mtomassoli 14y ago
Oh, I know. My module is your fault! :)
- llimllib 14y ago
  I really like the << x syntax, it's devious :)

northisup 14y ago

This seem like a whole series of posts on 'how to twist python syntax to save a line of code or two and make it unreadable'.

fendrak 14y ago

This is all well and good, but it doesn't appear to add support for the one thing I've often wanted in Python: proper closure support for functions within functions.

d0mine 14y ago
Do you mean 'nonlocal' keyword?

scott_s 14y ago

The code examples are confusing. First, I think there is a typo in the first two examples; I think < should be <<. But he never actually explains what the semantics of << are until several examples in, and it took me a long time to figure out that the string value 'x, y=3' gets evaled into code. Which is strange, to mix strings as code, when the whole point is to have code be code. At the very least, the semantics of this need to be explained better. When you give a new code example, you always need to say "And this is the result." Otherwise, I can't close the loop; I have new code with new and unknown semantics, and an unknown result. I need to have a known result to figure out the new semantics. (Much like you can't solve a single equation with two unknowns; you either need to have one unknown, or two equations.)

mtomassoli 14y ago
Just read the docstring of the module. It's very detailed. As for mixing strings with code, I still need to adhere to Python's syntax in order to avoid syntax errors. Anyway, syntax errors in strings are caught at "rewriting time". Using words instead of operators would also cause some problems.
- f0r 14y ago
  Should it be "<< 'x'" rather than "<< 2"?
  - mtomassoli 14y ago
    I don't believe it. Let's try to fix it again... thanks.
- scott_s 14y ago
  You're putting the carriage before the horse. People want to evaluate what it is they're getting into before they download it. If they can't understand the code examples, they won't get as far as downloading the module and looking at the docstring. I understand why you had to use strings as code. I had two concerns about that. One, you didn't explain it - I had to figure it out on my own. And two, if a solution is supposed to make things cleaner, but ends up introducing warts like that... maybe it's not worth it. What you implemented is interesting, no doubt, but I wouldn't want to use it for that reason. However, I think you would benefit greatly from having improved examples. Even if people don't use your module, they can still build on your ideas if it's well explained.

optymizer 14y ago

"code blocks" is why they invented functions and procedures half a century ago (and even those differ only in how they return). I think people forget (or don't know) that these are all labels to jump or branch to. So your named functions, functions, lambdas, code blocks, etc, become.. well.. equivalent. Hipsters, don't confuse the young ones with your 30 ways of jumping into a block of code. Cheers!

stcredzero 14y ago
Not the whole story. Code blocks can be used to protect encapsulation and create DSLs which are first class citizens of their host language. So your named functions, functions, lambdas, code blocks, etc, become.. well.. equivalent. They can also become so lexically awkward as to be unusable. For example, one code a toy "fuzzy logic" system in Smalltalk with a handful of methods. The result looks like a 1st class member of the language, just like the control structures that are already there. (Same goes for the loops) "Here is the standard if-else" (condition) ifTrue: [ ... ] ifFalse: [ ... ] "Here is what my DSL can look like" (condition) ifTrue: [ ... ] ifFalse: [ ... ] ifMaybe: [ ... ] Doing this with named functions is going to scatter code between different functions. It's semantically equivalent, but harder to read. Write anything involved in such a way, and it becomes untenable. Blocks can make such DSLs an order of magnitude more readable. Hipsters, don't confuse the young ones with your 30 ways of jumping into a block of code. Cheers! A good heaping fraction of hipsters don't understand what's good about code blocks.
- optymizer 14y ago
  "Code blocks can be used to protect encapsulation". So can functions, which are already in the language. "Doing this with named functions is going to scatter code between different functions". Please show an example. "A good heaping fraction of hipsters don't understand what's good about code blocks." That's one way of phrasing it.

bicknergseng 14y ago

I also question the utility and reason behind this. I spend a lot of time fixing memory leaks from lousy developers who use closures and anonymous functions incorrectly in JavaScript. I'm just learning Ruby as well, and see a lot of room for similar black-boxing pitfalls in procs and blocks. It seems to me like you're reducing code size at the expense of readability and debugability. That might be the ruby way, but it seems to me like a bad way to program. Like I said, I'm just learning... so I'd love counterexamples and reasons why I'm wrong.