5 ms·

is there any specific reason for using python 2 instead of 3 or it is just how you guys do it

Freak_NL 8y ago
Python 3 is ten (!) years old and perfectly usable, but it suffers from a well-known, but curious problem where people who occasionally use Python for some light scripting tend to go for Python 2.7. Rarely it's a choice made because of library availability, more often it is because of the perceived ubiquity of Python 2.7 installs on the target user's computer or simply because of a mindset where Python 2.7 is good enough and getting the hang of Python 3 seems like a huge obstacle (it isn't). I'd love to know why this project specifically chose Python 2.7 too. Python 2 reaches its end of life in 2020, any new Python code should really be written in Python 3. By the way, if you want to prevent getting a bunch of downvotes like your comment did, be polite! Not including any capitalization or punctuation is considered downright rude by many. Your text reads like a robot's printout.
- josefx 8y ago
  You forgot one reason not to switch from 2 to 3: Zero benefit. Nothing python 3 does would benefit my scripts to any extend over just using python 2 as I always did. Is learning 3 a huge hurdle? No, it is however in some cases completely unnecessary.
  - rjeli 8y ago
    Yeah, for writing scripts like this - python 2: 0% chance of thinking about Unicode. python 3: 5% chance i have to waste time debugging some random str decoding issue, no benefits. why bother?
    - hackbinary 8y ago
      Why bother? Because Python 2 is EOL in a little less than 2 years. The NHS chose not to upgrade to from Windows XP and 2003 and look where that got them last year: a massive crypto locker infection.
      - rwmj 8y ago
        I suppose unless someone either forks it or keeps delivering patches outside the Python project. That wasn't really an option for Windows XP, but I'm quite sure if it had been then someone would be doing it.
        sweettea 8y ago
        This is already true -- tauthon is a updated, patched Python 2 fork. https://github.com/naftaliharris/tauthon https://github.com/naftaliharris/tauthon
      - rphlx 8y ago
        FWIW Win10 was affected by that same horrific SMB RCE vuln. So the implied argument that 10 has been immune or even much less vulnerable to ransomware over the past year or two is on shaky ground, though I agree it probably will start to have some merit going forward.
    - baq 8y ago
      You must live in the US.
    - shawnz 8y ago
      This might be true if you're an english speaker, running the script on an english platform and only consuming data from english services. And also you are sure that no non-english speaker will ever take over the development of your script and you will never have to localize it to other languages. Otherwise, it's exactly the opposite. Python 3 is what you should be using if you want 0% chance of being stopped by a string encoding issue. Python 3 might occasionally require some extra steps when consuming strings compared to Python 2, but the reality is that those steps were always necessary. Python 2 just hid those details in a way that was only really safe for english-exclusive development. That doesn't mean that Python 2 is easier to use or less brittle. In fact I would say it means the opposite.
      - fpoling 8y ago
        Python2 worked fine with Russian input/output.
      - josefx 8y ago
        For most strings I don't care about language, encoding or related overhead. In my scripts they are best dealt with as ophaque bytes with a few specific byte patterns that are the same in ascii and utf8, as well as various other encodings. Last unicode issue I had was on a system german characters, because some library assumed it had to explicitly perform encoding with a bad default setting. If the library didn't try to be smart the program would have worked independently of system or language, instead it failed on any non english system by trying to convert a perfectly fine, system specific encoding to utf8.
    - cmyr 8y ago
      Put another way: python 2 has a 0% chance of correctly handling Unicode.
      - mjevans 8y ago
        Not actually the case. UTF-8, using only SPECIFIC operations that don't try to split up strings or replace things that aren't exact matches for a given text, will result in valid output as long as there was valid input. All interchanged Unicode text should be UTF-8, never use another encoding* (without a really compelling reason). No, storing it as an array of unicode characters isn't a compelling reason during interchange. ALSO, never use a BOM; that will break things. The second answer (should be anchor-linked) in this goes over MOST of the advantages of UTF-8, but it doesn't capture that some carefully input operations in otherwise completely Unicode //unaware// 'string' functions result in no change to string validity. The only potential issue is if recognizing something in different normalization representations is important. However, for nearly all quick and dirty tasks (where a short script is most likely) it usually doesn't matter. For everything else a different paradigm than the one Python3 picked would be better. (One where adding filters to a read file is OPTIONAL and they can be invoked on individual byte-strings as well.) https://stackoverflow.com/questions/6882301/what-is-the-best-utf#6884203 https://stackoverflow.com/questions/6882301/what-is-the-best...
  - jake_the_third 8y ago
    Seeing that there was a vulnerability in python just a couple days ago, I'd consider getting security updates a benefit. Most people will stop running python 2 when malware targeting it becomes rampant. So people being able to run your code is another benefit. :o)
  - shawnz 8y ago
    The benefit is that you won't be stunting adoption of Python 3, which will help get us back to a homogeneous Python landscape more quickly.
  - rdtsc 8y ago
    Great point. Python 3 for many just wasn't a great carrot. That's why they didn't switch. For a while it was even slower. Also most importantly Python 2 wasn't that bad of a stick. It was just pretty darn good to begin with. And of course that it is still supported on some OS-s people will keep using.
  - odonnellryan 8y ago
    You don't have to learn Python 3, and there are plenty of cool features (ok well being cool isn't always enough reason but it is a reason) and there are libraries that are not supported on 2.x anymore. Also you don't have so much longer until support drops entirely for 2.x, and then you really should port your projects. Why wait and port? Just write them in 3. There is nothing wrong with 3. Just starting in 3 is easy and not an issue. It is basically the same language.
  - carapace 8y ago
    This. I've been trying to make this point myself but people don't seem to want to acknowledge it, so it's gratifying to hear someone else say it too. What if Python 3 was a separate project that someone other than GvR had put forward, would adoption have the same impetus? I don't think so.
- gaius 8y ago
  more often it is because of the perceived ubiquity of Python 2.7 installs on the target user's computer It’s not just “perceived”, it is actual reality e.g. on CentOS. If you need compatibility still the best option is 2, regardless.
  - ptx 8y ago
    In RHEL 8, Python 3 is finally the default Python version: https://developers.redhat.com/blog/2018/11/14/python-in-rhel-8/ https://developers.redhat.com/blog/2018/11/14/python-in-rhel...
  - Wowfunhappy 8y ago
    ^^ Python 2.7 is also available by default on macOS, whereas you have to go out of your way to install Python 3.
    - gaius 8y ago
      ... and Debian 9 I’m no Luddite but the simple fact today if you want to reliably deploy and can’t or don’t want to ship a container, Python 2 is the reality.
      - rphlx 8y ago
        FWIW deb9 installs both 3.5 and 2.7 by default.
- fpoling 8y ago
  On Redhat and compatible systems python 2 will be supported for at least 10 years.
  - dijit 8y ago
    30th of June 2024 is not 10 years away. RHEL 7 will be the last RedHat with python2 by default and it goes end of life in 2024. Additionally; expecting them to properly support all the new features and manually creating (not backporting) fixes for issues is accrediting RedHat with having more resources than the reality allows.
    - fpoling 8y ago
      EOL for RHEL 8 will be in 10 years and python2 in RHEL8 will be a fully supported package even if not installed by default. And I suspect that it is not only Redhat that will need to support python2 past 2020. For example, I guess Ubuntu 20.4 with EOL in 2025 will still have a package. So there will be upgrade and fixes for python2 just not from python.org.
- avian 8y ago
  > I'd love to know why this project specifically chose Python 2.7 too. My guess is they just went with whatever was most convenient and worked. I doubt if this kind of questions were even on the radar. It's security research, not a production website. In research you tend to go with whatever tool does the job. Look at the overall complexity of this work. Maybe the author simply felt more confident with Python 2. You take whatever shortcuts that make things simpler, even if it means using 20 year old software.
- j1elo 8y ago
  It's not a huge obstacle, but it _is_ an obstacle. I'm not a Python programmer, but I can find my way around without much issue. Still, from where I'm sitting all that stuff about envs and the 2, 3 or 4 redundant, overriden, or deprecated mix of solutions sounds strange to me, I don't really want to buy into that just yet (I need Python like once per month, no more). For the casual programmer ("just want to change this line and run, please") it's just a block in the road that virtually all instructions you might see on any random README in Github might tell you to use "pip install..." but then you have to second guess if they mean "pip2" or "pip3", and whether they assume you'll be running with v2 or v3, maybe they tell you to run "pip install something" (which runs pip2), then in the script there is a Python 3 shebang... the installed dependency won't be found and now you have to understand all this mess if you want to continue. This is in Ubuntu, where the system's "python" is Python 2, and you need to explicitly add a "3" to all your commands if you want to run v3. I'm sure there are clever ways to solve the issue, but I bet they'll break some use case. I'd just rather none of this nonsense existed at all, then no funny workarounds would be needed. I wonder, how come other languages are able to slowly introduce some breaking changes over time (I'd swear Ruby was mentioned in some other HN thread about doing this) but not cause this kind of mess (as perceived from my humble external point of view)?
  - dev_dull 8y ago
    In other words, most likely because it’s the default python interpreter for this system.
CamperBob2 8y ago
Why is this downvoted to goatse.cx territory? It's a perfectly reasonable question with no self-evident answer.
- kencausey 8y ago
  I didn't downvote it but it seems well off-topic to me.
  - yjftsjthsd-h 8y ago
    It's a question about an implementation detail. It's not the big headline news, but it is relevant.
exikyut 8y ago
Nobody else answering this question seems to have read the article. > Like DAL, the OpenIPC library includes a command-line interface (CLI), written in Python and provided as a library for Python as part of Intel System Studio, which can be installed on the system with the help of pip. Intel System Studio doesn't ship its own python, it depends on the OS' python version. So here we have an industry-infra dependency on python, and these typically seem to still be stuck in 2.7 land. Python is being used here to actually wrap/coordinate the components that do the JTAG handshake. A neat approach, just using a whole programming language as the REPL. I like it. But this is one situation where I'm going to be very conservative with the tooling, as I don't want to break my system! :) So, that's the real reason why. Also; the repo actually has, right in view, a recent commit message of "fix for Python 3 compatibiliy". The authors are clearly just using 2.7 because ISS requires it and considering the bigger picture of tinkering with PoCs, mucking about with multiple Python versions seems like a waste of cycles and unnecessary management+verification of moving parts (I don't savor the idea of figuring out "ok how do I tell ISS where 2.7 is and how do I absolutely make sure it's avoiding 3.0?"). -- HN, please endeavor to pursue higher quality discussion. Devolving into meta is an excellent way to encourage stagnation. (I know this sounds a bit of a self-righteous thing to say but it's tricky to objectively articulate.)
- someguydave 8y ago
  Agreed. Nothing repels an actual hardware hacker harder than finding that the reaction to a significant finding consists mainly of bikesheding the python version of his concept script
draw_down 8y ago
Of course this is the biggest conversation in the thread. Ay yi yi, HN.
rasz 8y ago
3 forces you to screw around with Unicode encoding/decoding everything, painful when dealing with raw bytes.
- marssaxman 8y ago
  That's called "dealing with the reality of international text instead of burying your head in the sand," isn't it? Bytes are not text any more than bytes are a picture or bytes are a sound recording; it is only in the context of an encoding that bytes can be interpreted as something more. For that matter, given the inexplicable popularity of emojis, it isn't even a matter of international text anymore.
  - rasz 8y ago
    "dealing with raw bytes" = processing raw binary data (firmware dump, binary network protocols etc), not complete works of Shakespeare translated to 10 languages. So yes, go ahead with 3 if you are into emoji.

Python 3 is ten (!) years old and perfectly usable, but it suffers from a well-known, but curious problem where people who occasionally use Python for some light scripting tend to go for Python 2.7. Rarely it's a choice made because of library availability, more often it is because of the perceived ubiquity of Python 2.7 installs on the target user's computer or simply because of a mindset where Python 2.7 is good enough and getting the hang of Python 3 seems like a huge obstacle (it isn't). I'd love to know why this project specifically chose Python 2.7 too. Python 2 reaches its end of life in 2020, any new Python code should really be written in Python 3. By the way, if you want to prevent getting a bunch of downvotes like your comment did, be polite! Not including any capitalization or punctuation is considered downright rude by many. Your text reads like a robot's printout.

josefx 8y ago
You forgot one reason not to switch from 2 to 3: Zero benefit. Nothing python 3 does would benefit my scripts to any extend over just using python 2 as I always did. Is learning 3 a huge hurdle? No, it is however in some cases completely unnecessary.
- rjeli 8y ago
  Yeah, for writing scripts like this - python 2: 0% chance of thinking about Unicode. python 3: 5% chance i have to waste time debugging some random str decoding issue, no benefits. why bother?
  - hackbinary 8y ago
    Why bother? Because Python 2 is EOL in a little less than 2 years. The NHS chose not to upgrade to from Windows XP and 2003 and look where that got them last year: a massive crypto locker infection.
    - rwmj 8y ago
      I suppose unless someone either forks it or keeps delivering patches outside the Python project. That wasn't really an option for Windows XP, but I'm quite sure if it had been then someone would be doing it.
      - sweettea 8y ago
        This is already true -- tauthon is a updated, patched Python 2 fork. https://github.com/naftaliharris/tauthon https://github.com/naftaliharris/tauthon
    - rphlx 8y ago
      FWIW Win10 was affected by that same horrific SMB RCE vuln. So the implied argument that 10 has been immune or even much less vulnerable to ransomware over the past year or two is on shaky ground, though I agree it probably will start to have some merit going forward.
  - baq 8y ago
    You must live in the US.
  - shawnz 8y ago
    This might be true if you're an english speaker, running the script on an english platform and only consuming data from english services. And also you are sure that no non-english speaker will ever take over the development of your script and you will never have to localize it to other languages. Otherwise, it's exactly the opposite. Python 3 is what you should be using if you want 0% chance of being stopped by a string encoding issue. Python 3 might occasionally require some extra steps when consuming strings compared to Python 2, but the reality is that those steps were always necessary. Python 2 just hid those details in a way that was only really safe for english-exclusive development. That doesn't mean that Python 2 is easier to use or less brittle. In fact I would say it means the opposite.
    - fpoling 8y ago
      Python2 worked fine with Russian input/output.
    - josefx 8y ago
      For most strings I don't care about language, encoding or related overhead. In my scripts they are best dealt with as ophaque bytes with a few specific byte patterns that are the same in ascii and utf8, as well as various other encodings. Last unicode issue I had was on a system german characters, because some library assumed it had to explicitly perform encoding with a bad default setting. If the library didn't try to be smart the program would have worked independently of system or language, instead it failed on any non english system by trying to convert a perfectly fine, system specific encoding to utf8.
  - cmyr 8y ago
    Put another way: python 2 has a 0% chance of correctly handling Unicode.
    - mjevans 8y ago
      Not actually the case. UTF-8, using only SPECIFIC operations that don't try to split up strings or replace things that aren't exact matches for a given text, will result in valid output as long as there was valid input. All interchanged Unicode text should be UTF-8, never use another encoding* (without a really compelling reason). No, storing it as an array of unicode characters isn't a compelling reason during interchange. ALSO, never use a BOM; that will break things. The second answer (should be anchor-linked) in this goes over MOST of the advantages of UTF-8, but it doesn't capture that some carefully input operations in otherwise completely Unicode //unaware// 'string' functions result in no change to string validity. The only potential issue is if recognizing something in different normalization representations is important. However, for nearly all quick and dirty tasks (where a short script is most likely) it usually doesn't matter. For everything else a different paradigm than the one Python3 picked would be better. (One where adding filters to a read file is OPTIONAL and they can be invoked on individual byte-strings as well.) https://stackoverflow.com/questions/6882301/what-is-the-best-utf#6884203 https://stackoverflow.com/questions/6882301/what-is-the-best...
- jake_the_third 8y ago
  Seeing that there was a vulnerability in python just a couple days ago, I'd consider getting security updates a benefit. Most people will stop running python 2 when malware targeting it becomes rampant. So people being able to run your code is another benefit. :o)
- shawnz 8y ago
  The benefit is that you won't be stunting adoption of Python 3, which will help get us back to a homogeneous Python landscape more quickly.
- rdtsc 8y ago
  Great point. Python 3 for many just wasn't a great carrot. That's why they didn't switch. For a while it was even slower. Also most importantly Python 2 wasn't that bad of a stick. It was just pretty darn good to begin with. And of course that it is still supported on some OS-s people will keep using.
- odonnellryan 8y ago
  You don't have to learn Python 3, and there are plenty of cool features (ok well being cool isn't always enough reason but it is a reason) and there are libraries that are not supported on 2.x anymore. Also you don't have so much longer until support drops entirely for 2.x, and then you really should port your projects. Why wait and port? Just write them in 3. There is nothing wrong with 3. Just starting in 3 is easy and not an issue. It is basically the same language.
- carapace 8y ago
  This. I've been trying to make this point myself but people don't seem to want to acknowledge it, so it's gratifying to hear someone else say it too. What if Python 3 was a separate project that someone other than GvR had put forward, would adoption have the same impetus? I don't think so.
gaius 8y ago
more often it is because of the perceived ubiquity of Python 2.7 installs on the target user's computer It’s not just “perceived”, it is actual reality e.g. on CentOS. If you need compatibility still the best option is 2, regardless.
- ptx 8y ago
  In RHEL 8, Python 3 is finally the default Python version: https://developers.redhat.com/blog/2018/11/14/python-in-rhel-8/ https://developers.redhat.com/blog/2018/11/14/python-in-rhel...
- Wowfunhappy 8y ago
  ^^ Python 2.7 is also available by default on macOS, whereas you have to go out of your way to install Python 3.
  - gaius 8y ago
    ... and Debian 9 I’m no Luddite but the simple fact today if you want to reliably deploy and can’t or don’t want to ship a container, Python 2 is the reality.
    - rphlx 8y ago
      FWIW deb9 installs both 3.5 and 2.7 by default.
fpoling 8y ago
On Redhat and compatible systems python 2 will be supported for at least 10 years.
- dijit 8y ago
  30th of June 2024 is not 10 years away. RHEL 7 will be the last RedHat with python2 by default and it goes end of life in 2024. Additionally; expecting them to properly support all the new features and manually creating (not backporting) fixes for issues is accrediting RedHat with having more resources than the reality allows.
  - fpoling 8y ago
    EOL for RHEL 8 will be in 10 years and python2 in RHEL8 will be a fully supported package even if not installed by default. And I suspect that it is not only Redhat that will need to support python2 past 2020. For example, I guess Ubuntu 20.4 with EOL in 2025 will still have a package. So there will be upgrade and fixes for python2 just not from python.org.
avian 8y ago
> I'd love to know why this project specifically chose Python 2.7 too. My guess is they just went with whatever was most convenient and worked. I doubt if this kind of questions were even on the radar. It's security research, not a production website. In research you tend to go with whatever tool does the job. Look at the overall complexity of this work. Maybe the author simply felt more confident with Python 2. You take whatever shortcuts that make things simpler, even if it means using 20 year old software.
j1elo 8y ago
It's not a huge obstacle, but it _is_ an obstacle. I'm not a Python programmer, but I can find my way around without much issue. Still, from where I'm sitting all that stuff about envs and the 2, 3 or 4 redundant, overriden, or deprecated mix of solutions sounds strange to me, I don't really want to buy into that just yet (I need Python like once per month, no more). For the casual programmer ("just want to change this line and run, please") it's just a block in the road that virtually all instructions you might see on any random README in Github might tell you to use "pip install..." but then you have to second guess if they mean "pip2" or "pip3", and whether they assume you'll be running with v2 or v3, maybe they tell you to run "pip install something" (which runs pip2), then in the script there is a Python 3 shebang... the installed dependency won't be found and now you have to understand all this mess if you want to continue. This is in Ubuntu, where the system's "python" is Python 2, and you need to explicitly add a "3" to all your commands if you want to run v3. I'm sure there are clever ways to solve the issue, but I bet they'll break some use case. I'd just rather none of this nonsense existed at all, then no funny workarounds would be needed. I wonder, how come other languages are able to slowly introduce some breaking changes over time (I'd swear Ruby was mentioned in some other HN thread about doing this) but not cause this kind of mess (as perceived from my humble external point of view)?
- dev_dull 8y ago
  In other words, most likely because it’s the default python interpreter for this system.

CamperBob2 8y ago

Why is this downvoted to goatse.cx territory? It's a perfectly reasonable question with no self-evident answer.

kencausey 8y ago
I didn't downvote it but it seems well off-topic to me.
- yjftsjthsd-h 8y ago
  It's a question about an implementation detail. It's not the big headline news, but it is relevant.

exikyut 8y ago

Nobody else answering this question seems to have read the article. > Like DAL, the OpenIPC library includes a command-line interface (CLI), written in Python and provided as a library for Python as part of Intel System Studio, which can be installed on the system with the help of pip. Intel System Studio doesn't ship its own python, it depends on the OS' python version. So here we have an industry-infra dependency on python, and these typically seem to still be stuck in 2.7 land. Python is being used here to actually wrap/coordinate the components that do the JTAG handshake. A neat approach, just using a whole programming language as the REPL. I like it. But this is one situation where I'm going to be very conservative with the tooling, as I don't want to break my system! :) So, that's the real reason why. Also; the repo actually has, right in view, a recent commit message of "fix for Python 3 compatibiliy". The authors are clearly just using 2.7 because ISS requires it and considering the bigger picture of tinkering with PoCs, mucking about with multiple Python versions seems like a waste of cycles and unnecessary management+verification of moving parts (I don't savor the idea of figuring out "ok how do I tell ISS where 2.7 is and how do I absolutely make sure it's avoiding 3.0?"). -- HN, please endeavor to pursue higher quality discussion. Devolving into meta is an excellent way to encourage stagnation. (I know this sounds a bit of a self-righteous thing to say but it's tricky to objectively articulate.)

someguydave 8y ago
Agreed. Nothing repels an actual hardware hacker harder than finding that the reaction to a significant finding consists mainly of bikesheding the python version of his concept script

draw_down 8y ago

Of course this is the biggest conversation in the thread. Ay yi yi, HN.

rasz 8y ago

3 forces you to screw around with Unicode encoding/decoding everything, painful when dealing with raw bytes.

marssaxman 8y ago
That's called "dealing with the reality of international text instead of burying your head in the sand," isn't it? Bytes are not text any more than bytes are a picture or bytes are a sound recording; it is only in the context of an encoding that bytes can be interpreted as something more. For that matter, given the inexplicable popularity of emojis, it isn't even a matter of international text anymore.
- rasz 8y ago
  "dealing with raw bytes" = processing raw binary data (firmware dump, binary network protocols etc), not complete works of Shakespeare translated to 10 languages. So yes, go ahead with 3 if you are into emoji.