5 ms·

> I am against it, because it allows arbitrary Python expressions inside format strings. I was for this PEP until your post made this reality apparent. I'll ta

> I am against it, because it allows arbitrary Python expressions inside format strings. I was for this PEP until your post made this reality apparent. I'll take security over convenience.

imakesnowflakes 11y ago
I will go further and say that I will even take readability over convenience. This is a step in the reverse direction. Please don't do this. I am not sure why this is even considered. We already have ways to do this clearly. Let us not add another way to do this in a less readable way that is a lot more easier to write. That is a deadly combination. Features in python are geared towards more readable code (I know about the stuff you can do with things like comprehensions, but hey I think their power justifies them enough). This will lead to people using this format due to initial convenience, but ends up regretting doing so. Please remember that code is read more often than it is written. So a requiring a little verbosity if that can enhance readability even a little bit, is good. I hope these kinds of good things about python does not get removed. I am coming from 9 years of experience with PHP. And I will say that this is not worth it. And this is actually one of the features I have come to like in Python now. And that is not considering the implications of having expression evaluation inside strings...
- mkesper 11y ago
  I actually think the proposal would be more readable because you don't have to think about what it means. Compare: "{} of very long text here...".format(value) f"{value} of very long text here..."
  - inclemnet 11y ago
    This is a simple example that does seem more readable (though it also would be if you explicitly added the value=value), but I think the concerns people have are about more complex examples, and the fact that readability is also impacted if allowing arbitrary code within strings themselves. If you don't like using the empty {} (which seems fair enough), you can already use a couple of slightly more verbose but more explicit options: "{value} of very long text here...".format(**locals) or (more verbose with many keys) "{value} of very long text here...".format(value=value) I tend to prefer the latter even with multiple keys, but also think the former is better as it makes it explicit that the string is populated with the locals rather than implicitly doing it to all strings.
    - baq 11y ago
      .format(locals()) is a hack; locals() isn't supported on some python implementations.
      - inclemnet 11y ago
        The documentation at https://docs.python.org/3/library/functions.html#locals https://docs.python.org/3/library/functions.html#locals doesn't indicate that locals is not part of the language - where is it documented that using it is a hack and that python implementations need not support it?
      - wyldfire 11y ago
        It is on PyPy and CPython. Which ones don't? And when they don't support it, how do they behave? Return an empty dict or give an error?
  - yedpodtrzitko 11y ago
    imakesnowflakes made a good point below. Also compare the readability of this: "very long text {value} another long text".\ format(value=value) vs. f"very long text {value} another long text" ^ now you have to go and visually search through the whole string to see what is being used in there
    - TazeTSchnitzel 11y ago
      Syntax highlighting will make the variables used in the string pop out. This is easier than scanning to the end of the line to find the format() call.
    - sanderjd 11y ago
      This isn't a bad point, but anecdotally, after using interpolated strings for many years, I've never come across this problem a single time.
  - imakesnowflakes 11y ago
    I thinks this is where our difference in comes from. When we are reading code, we are not usually reading the strings contained within. We are reading variables names, we are looking at where they are used, where they are assigned, stuff like that. I very rarely look inside the strings when we are actually reading code. When reading code, strings are just black boxes where nothing can happen. So we can completely ignore them. With this pep, that will change. And readability takes a big hit. Strings are no longer black boxes. And actually you can get the " {value} if very long text" format right now using the format function, but it right now requires an explicit list/dictionary of variables. This explicit list of variables really helps in the context of reading code. And forcing users to write down that explicit list of variables to use, is good.
    - Camillo 11y ago
      How do you know where the string ends? If your editor has syntax coloring and you look for the end of the string-color block, then the expressions inside the string will also have a different color and will pop right out. If you don't use syntax coloring you have to scan for " or ', and you'll learn to scan for { too.
      - imakesnowflakes 11y ago
        >How do you know where the string ends? I usually just go to the end of the line. or if that is not possible, I think I usually look at the next closest thing that obviously is not a string and scan backwards. I mean, usually I didn't have to do anything consciously (or do a scan) to spot the end of a string.
    - mkesper 11y ago
      Point taken!
    - dragonwriter 11y ago
      When I'm reading code, I'm always reading string literals contained within it. Strings that aren't integral to the code should be external resources, and I'm not even convinced it's possible to black-box them at a below-conscious level; you can selectively actively ignore them, but that takes additional effort that is only generally a net win in particularly poorly structured code.)
  - filmor 11y ago
    You can have something like this today if you really want to :): def f(s): import sys return s.format(**sys._getframe(1).f_locals) Used as a = 1 b = 2 f("{a} + {b}")
- jkern 11y ago
  Also coming from a php background, I'll 100% agree with this. I always cringe when I see strings formatted like this for pretty much the same reasons. It makes it painful to figure out what the hell a string is actually going to look like when you have a bunch of code embedded inside it
voyou 11y ago
I don't think this is actually a security concern. The only place f-strings are evaluated is where they're directly included in the source; they can't be supplied by a user (unless you're using "eval," in which case the security concern applies with or without f-strings). As the PEP says: "Because the f-strings are evaluated where the string appears in the source code, there is no additional expressiveness available with f-strings. There are also no additional security concerns: you could have also just written the same expression, not inside of an f-string."
- task_queue 11y ago
  Should have read the PEP in full. Thanks : )
ceronman 11y ago
This proposal doesn't affect security in any way. It's just syntactic sugar. Instead of writing this: 'My name is ' + format(name) + ' my age next year is ' + format(age+1) or 'My name is {name}, my age next year is {age}'.format(name=name, age=age+1) You just write f'My name is {name}, my age next year is {age=1}' It's shorter, it's more readable and more convenient. I can't wait for this PEP to be accepted. Edit: Fixed small errors in the examples.
- imakesnowflakes 11y ago
  First of all, I don't think you can do {age+1} in a string right now. But you can use string placeholders when used with format function. That should take take of this use case. So no need to add another syntactic sugar to make it more convenient. It is already as simple as possible. Let us not try to simplify it further. I will be horrified and very worried about the direction that the language has taken if this pep gets accepted.
  - ceronman 11y ago
    You're right, you can't do {age+1} in a string right now. I just edited to fix it.
  - sanderjd 11y ago
    This perspective is so interesting to me. You say the version using `.format` is "as simple as possible", but the interpolated version seems clearly simpler to me, because it eliminates the redundancy in the placeholders. What's the disconnect here?
    - imakesnowflakes 11y ago
      > but the interpolated version seems clearly simpler to me. What's the disconnect here? The disconnect is this. An explicit list of variables to use for interpolation is not redundancy. Because the placeholders does not, by themselves, refer to anything. Placeholders are just holes and only have a meaning in the context of formatting functions. And even in them, they does not refer to a local variables, even when they share a common name. So when you use .format() function, you are actually saying, "here is a string with some named holes. Fill the hole named 'A' with the value from variable 'A', the hole named 'B' with the value from variable 'B'. So hole 'A' and variable 'A' are different things even if they share the same name. Now, the simplicity argument. Every thing should be made as simple as possible. But not simpler. Why not? Because when you simplify further, you are paying a cost (often not apparent initially), some times in clarity, sometimes in correctness and so on. The pigeonhole principle. In our case, we can further simplify the process by adding an implicit mapping from named placeholders to local variables. When you do so, you are adding something implicit. The costs of which might not be apparent at this point.. So this pep is strapping on something to the whole language, to slightly simplify this one use case. Which is why I said that it is trying to simplify it further than it is possible.
CJefferson 11y ago
Note that these format strings must exist in the source code -- you can't read them from the user then execute it.
jerf 11y ago
I'd submit the security concern is more that specifying a string interpolation format without first-class thought about how to encode interpolated strings is a terrible idea in 2015, and people need to really stop doing this. See http://www.jerf.org/iri/post/2942 http://www.jerf.org/iri/post/2942 and the example library I use to demonstrate the point, https://github.com/thejerf/strinterp https://github.com/thejerf/strinterp . Of course all current methods of string interpolation in Python have that problem too. And typing that sentence really, really makes we want to link http://xkcd.com/927/ http://xkcd.com/927/ . I'm unconvinced adding a fourth choice at this very late date can fix anything.

> I am against it, because it allows arbitrary Python expressions inside format strings. I was for this PEP until your post made this reality apparent. I'll take security over convenience.

imakesnowflakes 11y ago

I will go further and say that I will even take readability over convenience. This is a step in the reverse direction. Please don't do this. I am not sure why this is even considered. We already have ways to do this clearly. Let us not add another way to do this in a less readable way that is a lot more easier to write. That is a deadly combination. Features in python are geared towards more readable code (I know about the stuff you can do with things like comprehensions, but hey I think their power justifies them enough). This will lead to people using this format due to initial convenience, but ends up regretting doing so. Please remember that code is read more often than it is written. So a requiring a little verbosity if that can enhance readability even a little bit, is good. I hope these kinds of good things about python does not get removed. I am coming from 9 years of experience with PHP. And I will say that this is not worth it. And this is actually one of the features I have come to like in Python now. And that is not considering the implications of having expression evaluation inside strings...

mkesper 11y ago
I actually think the proposal would be more readable because you don't have to think about what it means. Compare: "{} of very long text here...".format(value) f"{value} of very long text here..."
- inclemnet 11y ago
  This is a simple example that does seem more readable (though it also would be if you explicitly added the value=value), but I think the concerns people have are about more complex examples, and the fact that readability is also impacted if allowing arbitrary code within strings themselves. If you don't like using the empty {} (which seems fair enough), you can already use a couple of slightly more verbose but more explicit options: "{value} of very long text here...".format(**locals) or (more verbose with many keys) "{value} of very long text here...".format(value=value) I tend to prefer the latter even with multiple keys, but also think the former is better as it makes it explicit that the string is populated with the locals rather than implicitly doing it to all strings.
  - baq 11y ago
    .format(locals()) is a hack; locals() isn't supported on some python implementations.
    - inclemnet 11y ago
      The documentation at https://docs.python.org/3/library/functions.html#locals https://docs.python.org/3/library/functions.html#locals doesn't indicate that locals is not part of the language - where is it documented that using it is a hack and that python implementations need not support it?
    - wyldfire 11y ago
      It is on PyPy and CPython. Which ones don't? And when they don't support it, how do they behave? Return an empty dict or give an error?
- yedpodtrzitko 11y ago
  imakesnowflakes made a good point below. Also compare the readability of this: "very long text {value} another long text".\ format(value=value) vs. f"very long text {value} another long text" ^ now you have to go and visually search through the whole string to see what is being used in there
  - TazeTSchnitzel 11y ago
    Syntax highlighting will make the variables used in the string pop out. This is easier than scanning to the end of the line to find the format() call.
  - sanderjd 11y ago
    This isn't a bad point, but anecdotally, after using interpolated strings for many years, I've never come across this problem a single time.
- imakesnowflakes 11y ago
  I thinks this is where our difference in comes from. When we are reading code, we are not usually reading the strings contained within. We are reading variables names, we are looking at where they are used, where they are assigned, stuff like that. I very rarely look inside the strings when we are actually reading code. When reading code, strings are just black boxes where nothing can happen. So we can completely ignore them. With this pep, that will change. And readability takes a big hit. Strings are no longer black boxes. And actually you can get the " {value} if very long text" format right now using the format function, but it right now requires an explicit list/dictionary of variables. This explicit list of variables really helps in the context of reading code. And forcing users to write down that explicit list of variables to use, is good.
  - Camillo 11y ago
    How do you know where the string ends? If your editor has syntax coloring and you look for the end of the string-color block, then the expressions inside the string will also have a different color and will pop right out. If you don't use syntax coloring you have to scan for " or ', and you'll learn to scan for { too.
    - imakesnowflakes 11y ago
      >How do you know where the string ends? I usually just go to the end of the line. or if that is not possible, I think I usually look at the next closest thing that obviously is not a string and scan backwards. I mean, usually I didn't have to do anything consciously (or do a scan) to spot the end of a string.
  - mkesper 11y ago
    Point taken!
  - dragonwriter 11y ago
    When I'm reading code, I'm always reading string literals contained within it. Strings that aren't integral to the code should be external resources, and I'm not even convinced it's possible to black-box them at a below-conscious level; you can selectively actively ignore them, but that takes additional effort that is only generally a net win in particularly poorly structured code.)
- filmor 11y ago
  You can have something like this today if you really want to :): def f(s): import sys return s.format(**sys._getframe(1).f_locals) Used as a = 1 b = 2 f("{a} + {b}")
jkern 11y ago
Also coming from a php background, I'll 100% agree with this. I always cringe when I see strings formatted like this for pretty much the same reasons. It makes it painful to figure out what the hell a string is actually going to look like when you have a bunch of code embedded inside it

voyou 11y ago

I don't think this is actually a security concern. The only place f-strings are evaluated is where they're directly included in the source; they can't be supplied by a user (unless you're using "eval," in which case the security concern applies with or without f-strings). As the PEP says: "Because the f-strings are evaluated where the string appears in the source code, there is no additional expressiveness available with f-strings. There are also no additional security concerns: you could have also just written the same expression, not inside of an f-string."

task_queue 11y ago
Should have read the PEP in full. Thanks : )

ceronman 11y ago

This proposal doesn't affect security in any way. It's just syntactic sugar. Instead of writing this: 'My name is ' + format(name) + ' my age next year is ' + format(age+1) or 'My name is {name}, my age next year is {age}'.format(name=name, age=age+1) You just write f'My name is {name}, my age next year is {age=1}' It's shorter, it's more readable and more convenient. I can't wait for this PEP to be accepted. Edit: Fixed small errors in the examples.

imakesnowflakes 11y ago
First of all, I don't think you can do {age+1} in a string right now. But you can use string placeholders when used with format function. That should take take of this use case. So no need to add another syntactic sugar to make it more convenient. It is already as simple as possible. Let us not try to simplify it further. I will be horrified and very worried about the direction that the language has taken if this pep gets accepted.
- ceronman 11y ago
  You're right, you can't do {age+1} in a string right now. I just edited to fix it.
- sanderjd 11y ago
  This perspective is so interesting to me. You say the version using `.format` is "as simple as possible", but the interpolated version seems clearly simpler to me, because it eliminates the redundancy in the placeholders. What's the disconnect here?
  - imakesnowflakes 11y ago
    > but the interpolated version seems clearly simpler to me. What's the disconnect here? The disconnect is this. An explicit list of variables to use for interpolation is not redundancy. Because the placeholders does not, by themselves, refer to anything. Placeholders are just holes and only have a meaning in the context of formatting functions. And even in them, they does not refer to a local variables, even when they share a common name. So when you use .format() function, you are actually saying, "here is a string with some named holes. Fill the hole named 'A' with the value from variable 'A', the hole named 'B' with the value from variable 'B'. So hole 'A' and variable 'A' are different things even if they share the same name. Now, the simplicity argument. Every thing should be made as simple as possible. But not simpler. Why not? Because when you simplify further, you are paying a cost (often not apparent initially), some times in clarity, sometimes in correctness and so on. The pigeonhole principle. In our case, we can further simplify the process by adding an implicit mapping from named placeholders to local variables. When you do so, you are adding something implicit. The costs of which might not be apparent at this point.. So this pep is strapping on something to the whole language, to slightly simplify this one use case. Which is why I said that it is trying to simplify it further than it is possible.

CJefferson 11y ago

Note that these format strings must exist in the source code -- you can't read them from the user then execute it.

jerf 11y ago

I'd submit the security concern is more that specifying a string interpolation format without first-class thought about how to encode interpolated strings is a terrible idea in 2015, and people need to really stop doing this. See http://www.jerf.org/iri/post/2942 http://www.jerf.org/iri/post/2942 and the example library I use to demonstrate the point, https://github.com/thejerf/strinterp https://github.com/thejerf/strinterp . Of course all current methods of string interpolation in Python have that problem too. And typing that sentence really, really makes we want to link http://xkcd.com/927/ http://xkcd.com/927/ . I'm unconvinced adding a fourth choice at this very late date can fix anything.