5 ms·

There's no magic: virtualenv edition

134 points by akaptur 14y ago

pm90 14y ago
Excellent article. Although it is quite obvious how venv worked, I had never bothered to actually look up the source. Consequently, I was always a bit hesitant on using it, because I have a customized environment for installing stuff and didn't want to break it. But venv is doing just what I'm doing, but automatically :)
jacobian 14y ago
Related: Carl Meyer's "Reverse-engineering Ian Bicking's brain" talk from PyCon 2011: http://pyvideo.org/video/389/pycon-2011--reverse-engineering-ian-bicking--39-s http://pyvideo.org/video/389/pycon-2011--reverse-engineering...
yesimahuman 14y ago
Understanding magic means you understand the concepts. It's a very advanced position to be in relative to your programming education. I'm guessing it's also why "hard" classes in school start with the concepts rather than the magic (physics is a perfect example). You can't really go far if you don't get the concepts. I've been doing some training lately for new hires, and I am inclined to start with the concepts and then teach the magic, but I wonder if it's the most effective way to learn in the short term?
- deleted 14y ago
  [deleted]
- rplacd 14y ago
  Don't fear abstraction - that's as far as we'll get in plenty of cases, especially when we're talking about the epistemology of the sciences; in any case, there's always an element of human control involved - once opinionated humans are put in charge of articulating abstractions, we tend towards elegance, orthogonality, the stuff that makes you grin. Fear leaky abstractions, though.
ImprovedSilence 14y ago
haha, I had no idea "source" and '.' were the same! It all gets a little bit clearer, one step at time....
- pyre 14y ago
  IIRC, 'source' is a bash-ism, while '.' is POSIX, so it might not work in all shells. That said $() isn't POSIX, but works on enough shells, that I'm comfortable replacing `` with it.
  - mpyne 14y ago
    Actually $() is in POSIX (or at least, it is now). You're right about 'source' though, shell scripts that want a semblance of portability must use '.'.
  - nuxi 14y ago
    $() has been POSIX for a while, here's the SUSv2 definition from 1997: http://pubs.opengroup.org/onlinepubs/7908799/xcu/chap2.html#tag_001_006_003 http://pubs.opengroup.org/onlinepubs/7908799/xcu/chap2.html#...
j4mie 14y ago
See also "Virtualenv's bin/activate is Doing It Wrong": https://gist.github.com/datagrok/2199506 https://gist.github.com/datagrok/2199506 The activate/deactivate magic is responsible for some real nastiness in deployment scripts, and total confusion for beginners who are thrown by the statefulness. Virtualenvwrapper makes the situation even worse by hiding the virtual environments away. By far the easiest way to use virtualenv is to forget about activation completely, and directly reference the binary you want to use. For example, rather than typing: source env/bin/activate python myprogram.py Just run env/bin/python myprogram.py It's an extra few characters on every command, but it's worth it to totally remove the magic.
- kami8845 14y ago
  No thank you. I will continue using my "magic" that has caused me 0 problems over the past couple of years running python scripts thousands of times. If you hate automation so much, feel free to keep typing "env/bin/python" every time you execute something, but for 99.99% of us, virtualenv works perfectly fine for everyday use.
  - ProblemFactory 14y ago
    > for 99.99% of us, virtualenv works perfectly fine for everyday use. Depends on what you are running and from where. The usual virtualenv activate (although I recommend virtualenvwrapper's workon) works fine for a human interacting with a terminal. But it is annoying for bash scripts, extremely annoying for Fabric scripts where each command is run in a separate SSH session, and sometimes completely impossible when a server needs to spawn a python subprocess. Used this exact method to solve issues with nginx+uwsgi web server deployments a few weeks ago. Using the full path to the virtualenv's python works in any context, even when you can't set environment variables or run two commands.
    - skyraider 14y ago
      Using Fabric's with prefix('source venv/bin/activate'): .. is not annoying in the slightest (to me).
clicks 14y ago
(A little bit off-topic, but I see that you're the person who wrote the Flask tutorial, just wanted to say thanks for writing it, I loved it: https://github.com/akaptur/MyFlaskTutorial https://github.com/akaptur/MyFlaskTutorial ).
- akaptur 14y ago
  Glad you liked it, but that's actually my fork of the original, which was by another Hacker Schooler: https://github.com/bev-a-tron/MyFlaskTutorial https://github.com/bev-a-tron/MyFlaskTutorial I really liked the way she distinguished arbitrary variable names from mandatory class names etc. So often, tutorials don't make that clear.
  - clicks 14y ago
    Ah. Well thanks to you both. :)

134 points by akaptur 14y ago

pm90 14y ago

Excellent article. Although it is quite obvious how venv worked, I had never bothered to actually look up the source. Consequently, I was always a bit hesitant on using it, because I have a customized environment for installing stuff and didn't want to break it. But venv is doing just what I'm doing, but automatically :)

jacobian 14y ago

Related: Carl Meyer's "Reverse-engineering Ian Bicking's brain" talk from PyCon 2011: http://pyvideo.org/video/389/pycon-2011--reverse-engineering-ian-bicking--39-s http://pyvideo.org/video/389/pycon-2011--reverse-engineering...

yesimahuman 14y ago

Understanding magic means you understand the concepts. It's a very advanced position to be in relative to your programming education. I'm guessing it's also why "hard" classes in school start with the concepts rather than the magic (physics is a perfect example). You can't really go far if you don't get the concepts. I've been doing some training lately for new hires, and I am inclined to start with the concepts and then teach the magic, but I wonder if it's the most effective way to learn in the short term?

deleted 14y ago
[deleted]
rplacd 14y ago
Don't fear abstraction - that's as far as we'll get in plenty of cases, especially when we're talking about the epistemology of the sciences; in any case, there's always an element of human control involved - once opinionated humans are put in charge of articulating abstractions, we tend towards elegance, orthogonality, the stuff that makes you grin. Fear leaky abstractions, though.

ImprovedSilence 14y ago

haha, I had no idea "source" and '.' were the same! It all gets a little bit clearer, one step at time....

pyre 14y ago
IIRC, 'source' is a bash-ism, while '.' is POSIX, so it might not work in all shells. That said $() isn't POSIX, but works on enough shells, that I'm comfortable replacing `` with it.
- mpyne 14y ago
  Actually $() is in POSIX (or at least, it is now). You're right about 'source' though, shell scripts that want a semblance of portability must use '.'.
- nuxi 14y ago
  $() has been POSIX for a while, here's the SUSv2 definition from 1997: http://pubs.opengroup.org/onlinepubs/7908799/xcu/chap2.html#tag_001_006_003 http://pubs.opengroup.org/onlinepubs/7908799/xcu/chap2.html#...

j4mie 14y ago

See also "Virtualenv's bin/activate is Doing It Wrong": https://gist.github.com/datagrok/2199506 https://gist.github.com/datagrok/2199506 The activate/deactivate magic is responsible for some real nastiness in deployment scripts, and total confusion for beginners who are thrown by the statefulness. Virtualenvwrapper makes the situation even worse by hiding the virtual environments away. By far the easiest way to use virtualenv is to forget about activation completely, and directly reference the binary you want to use. For example, rather than typing: source env/bin/activate python myprogram.py Just run env/bin/python myprogram.py It's an extra few characters on every command, but it's worth it to totally remove the magic.

kami8845 14y ago
No thank you. I will continue using my "magic" that has caused me 0 problems over the past couple of years running python scripts thousands of times. If you hate automation so much, feel free to keep typing "env/bin/python" every time you execute something, but for 99.99% of us, virtualenv works perfectly fine for everyday use.
- ProblemFactory 14y ago
  > for 99.99% of us, virtualenv works perfectly fine for everyday use. Depends on what you are running and from where. The usual virtualenv activate (although I recommend virtualenvwrapper's workon) works fine for a human interacting with a terminal. But it is annoying for bash scripts, extremely annoying for Fabric scripts where each command is run in a separate SSH session, and sometimes completely impossible when a server needs to spawn a python subprocess. Used this exact method to solve issues with nginx+uwsgi web server deployments a few weeks ago. Using the full path to the virtualenv's python works in any context, even when you can't set environment variables or run two commands.
  - skyraider 14y ago
    Using Fabric's with prefix('source venv/bin/activate'): .. is not annoying in the slightest (to me).

clicks 14y ago

(A little bit off-topic, but I see that you're the person who wrote the Flask tutorial, just wanted to say thanks for writing it, I loved it: https://github.com/akaptur/MyFlaskTutorial https://github.com/akaptur/MyFlaskTutorial ).

akaptur 14y ago
Glad you liked it, but that's actually my fork of the original, which was by another Hacker Schooler: https://github.com/bev-a-tron/MyFlaskTutorial https://github.com/bev-a-tron/MyFlaskTutorial I really liked the way she distinguished arbitrary variable names from mandatory class names etc. So often, tutorials don't make that clear.
- clicks 14y ago
  Ah. Well thanks to you both. :)