8 ms·
The author’s point about “not caring about pip vs poetry vs uv” is missing that uv directly supports this use case, including PyPI dependencies, and all you nee
by hamishwhc 9mo ago
The author’s point about “not caring about pip vs poetry vs uv” is missing that uv directly supports this use case, including PyPI dependencies, and all you need is uv and your preferred Python version installed: https://docs.astral.sh/uv/guides/scripts/#using-a-shebang-to-create-an-executable-file https://docs.astral.sh/uv/guides/scripts/#using-a-shebang-to...
- benrutter 9mo agoI thought that too, but I think the tricky bit is if you're a non-python user, this isn't yet obvious. If you've never used Clojure and start a Clojure project, you will almost definitely find advice telling you to use Leiningen. For Python, if you search online you might find someone saying to use uv, but also potentially venv, poetry or hatch. I definitely think uv is taking over, but its not yet ubiquitous. Ironically, I actually had a similar thing installing Go the other day. I'd never used Go before, and installed it using apt only to find that version was too old and I'd done it wrong. Although in that case, it was a much quicker resolution than I think anyone fighting with virtual environments would have.
- houzi 9mo agoDo you think a non-python user would piece it together if the shebang line reveals what tool to use?
- benrutter 9mo agoI think yes if that line was UV. But otherwise, of its just python, you have the issue that you need two tools, one for running scripts and one for managing dependencies and environments.
- idoubtit 9mo agoThat's my experience. I'm not a Python developer, and installing Python programs has been a mess for decades, so I'd rather stay away from the language than try another new tool. Over the years, I've used setup.py, pip, pipenv (which kept crashing though it was an official recommendation), manual venv+pip (or virtualenv? I vaguely remember there were 2 similar tools and none was part of a minimal Python install). Does uv work in all of these cases? The uv doc pointed out by the GP is vague about legacy projects, though I've just skimmed through the long page. IIRC, Python tools didn't share their data across projects, so they could build the same heavy dependencies multiple times. I've also seen projects with incomplete dependencies (installed through Conda, IIRC) which were a major pain to get working. For many years, the only simple and sane way to run some Python code was in a Docker image, which has its own drawbacks.
- whimsicalism 9mo agouv solved it, it’s safe to come back now.
- lexicality 9mo ago> Does uv work in all of these cases? Yes. The goal of uv is to defuck the python ecosystem and they're doing a very good job at it so far.
- aeurielesn 9mo agoThat's giving way too much credit to uv.
- karel-3d 9mo agouv is really that good.
- hexo 9mo agoIf so, ok, let's port this prototype to back to python and get rid of uv.
- llbeansandrice 9mo agoWhat does this comment mean? Port the dependency and virtual environment manager back to the language? Should we port npm “back” to node js?
- karel-3d 9mo agoWell, go does have the module management, including downloading new versions of itself, built-in into the `go` tool itself. It is really great. But I don't see this hapenning in python.
- 9mo ago
- zahlman 9mo ago> you might find someone saying to use uv, but also potentially venv, poetry or hatch. This is sort of like saying "You might find someone saying to drive a Ford, but also potentially internal combustion engine, Nissan or Hyundai".
- evilduck 9mo agoOnly to those already steeped in Python. To an outsider they're all equally arbitrary non-descriptive words and there's not even obvious proper noun capitalization to tell apart a component from a tool brand.
- zahlman 9mo agoIt's always rather irritating to me that people make these complaints without trying to understand any of the under-the-hood stuff, because the ultimate conclusion is that it's somehow a bad thing that, on a FOSS project, multiple people tried to solve a problem concurrently.
- NetMageSCW 9mo agoThat’s especially ironic given that inside Python part of the philosophy is “There should be one-- and preferably only one --obvious way to do it.” So why does Python’s external environment seem more like something that escape from a Perl zoo?
- kstrauser 9mo agoThe one obvious way is the underlying virtualenv abstraction. Everything else just makes that part easier or more transparent.
- zahlman 9mo agoWhat kstrauser said. But with much more detail: it seems complicated because * People refuse to learn basic concepts that are readily explained by many sources; e.g. https://chriswarrick.com/blog/2018/09/04/python-virtual-environments/ https://chriswarrick.com/blog/2018/09/04/python-virtual-envi... [0]. * People cling to memories of long-obsolete issues. When people point to XKCD 1987 they overlook that Python 2.x has been EOL for almost six years (and 3.6 for over four, but whatever)[1]; only Mac users have to worry about "homebrew" (which I understand was directly interfering with stuff back in the day) or "framework builds" of Python; easy_install is similarly a long-deprecated dinosaur that you also would never need once you have pip set up; and fewer and fewer people actually need Anaconda for anything[2][3]. * There is never just one way to do it, depending on your understanding of "do". Everyone will always imagine that the underlying functionality can be wrapped in a more user-friendly way, and they will have multiple incompatible ideas about what is the most user-friendly. But there is one obvious "way to do it", which is to set up the virtual environment and then launch the virtual environment's Python executable. Literally everything else is window dressing on top of that. The only thing that "activating" the environment does is configure environment variables so that `python` means the virtual environment's Python executable. All your various alternative tools are just presenting different ways to ensure that you run the correct Python (under the assumption that you don't want to remember a path to it, I guess) and to bundle up the virtual environment creation with some other development task. The Python community did explicitly provide for multiple people to provide such wrappers. This was not by providing the "15th competing standard". It was by providing the standard (really a set of standards designed to work together: the virtual environment support in the standard library, the PEPs describing `pyproject.toml`, and so on), which replaced a Wild West (where Setuptools was the sheriff and pip its deputy). [0]: By the way, this is by someone who doesn't like virtual environments and was one of the biggest backers of PEP 582. [1]: Of course, this is not Randall Munroe's fault. The comic dates to 2018, right in the middle of the period where the community was trying to sort things out and figure out how to not require the often problematic `setup.py` configuration for every project including pure-Python ones. [2]: The SciPy stack has been installable from wheels for almost everyone for quite some time and they were even able to get 3.12 wheels out promptly despite being hamstrung by the standard library `distutils` removal. [3]: Those who do need it, meanwhile, can generally live within that environment entirely.
- regularfry 9mo agoThere's definitely a philosophical shift that you can observe happening over the last 12-15 years or so, where at the start you have the interpreter as the centre of the world and at the end there's an ecosystem management tool that you use to give yourself an interpreter (and virtual environments, and so on) per project. I think this properly kicked off with RVM, which needed to come into existence because you had this situation where the Ruby interpreter was going through incompatible changes, the versions on popular distributions were lagging, and Rails, the main reason people were turning to Ruby, was relatively militant about which interpreter versions it would support. Also, building the interpreter such that it would successfully run Rails wasn't trivial. Not that hard, but enough that a convenience wrapper mattered. So you had a whole generation of web devs coming up in an environment where the core language wasn't the first touchpoint, and there wasn't an assumption that you could (or should) rely on what you could apt-get install on the base OS. This is broadly an extremely good thing. But the critical thing that RVM did was that it broke the circular dependency at the core of the problem: it didn't itself depend on having a working ruby interpreter. Prior to that you could observe a sort of sniffiness about tools for a language which weren't implemented in that language, but RVM solved enough of the pain that it barged straight past that. Then you had similar tools popping up in other languages - nvm and leiningen are the first that spring to mind, but I'd also throw (for instance) asdf into the mix here - where the executable that you call to set up your environment has a '#!/bin/bash' shebang line. Go has sidestepped most of this because of three things: 1) rigorous backwards compatibility; 2) the simplest possible installation onramp; 3) being timed with the above timeline so that having a pre-existing `go` binary provided by your OS is unlikely unless you install it yourself. And none of those are true of Python. The backwards compatibility breaks in this period are legendary, you almost always do have a pre-existing Python to confuse things, and installing a new python without breaking that pre-existing Python, which your OS itself depends on, is a risk. Add to that the sniffiness I mentioned (which you can still see today on `uv` threads) and you've got a situation where Python is catching up to what other languages managed a decade ago. Again.
- bee_rider 9mo agoIt is sort of funny, if we squint just the wrong way, “ecosystem management tool first, then think about interpreters” starts to look a lot like… a package manager, haha.
- NeutralCrane 9mo agouv has been around for less than two years. It’s on track to become the default choice, it’s just a matter of time.
- MarsIronPI 9mo ago> If you've never used Clojure and start a Clojure project, you will almost definitely find advice telling you to use Leiningen. I thought the current best practice for Clojure was to use the shiny new built-in tooling? deps.edn or something like that?
- codemonkey-zeta 9mo agodeps.edn is becoming the default choice, yes. I interpreted the parent comment as saying "you will see advice to use leiningen (even though newer solutions exist, simply because it _was_ the default choice when the articles were written)"
- deleted 9mo ago[deleted]
- fulafel 9mo agoClojure CLI (aka deps.edn) came out in 2018 and in the survey "how do you manage your dependencies?" question crossed 50% usage in early 2020. So for 6-8 years now.
- benrutter 9mo agoAh woops! Sorry for my outdated clojure knowledge, this sounds cool though- I'll give it a go!
- meander_water 9mo agoActually you can go one better: #!/usr/bin/env -S uv run --python 3.14 --script Then you don't even need python installed. uv will install the version of python you specified and run the command.
- rikafurude21 9mo agoalternatively, uv lets you do this: #!/usr/bin/env -S uv run --script # # /// script # requires-python = ">=3.12" # dependencies = ["foo"] # ///
- semi-extrinsic 9mo agoThe /// script block is actually specified in PEP 723 and supported by several other tools apart from uv.
- yjftsjthsd-h 9mo agoThat's good to hear; do you know what other tools support it?
- semi-extrinsic 9mo agoFrom what I can tell, Hatch, PDM, pipx and pip-run also support it.
- nemosaltat 9mo agoThe last time I commented extolling the virtues of uv on here, I got a similar reply, pointing out that PEP 723 specs this behavior, and uv isn’t the only way. So I’ll try again in this thread: I’m bullish on uv, and waiting for Cunningham.
- semi-extrinsic 9mo agoI am all in on uv as well, and advocating for its use heavily at $dayjob. But I think having as much as possible of these things encoded in standards is good for the ecosystem. Maybe in a few years time, someone will make something even better than uv. And in the meantime, having things standardised speeds up adoption in e.g. syntax highlighting in editors and such.
- tgv 9mo agoWon't those dependencies then be global? With potential conflicts as a result?
- stephenlf 9mo agoNope! uv takes care of that. uv is a work of art.
- tgv 9mo agoThen I should seriously take a look at it. I figured it was just another package manager.
- auxym 9mo agouv uses a global cache but hardlinks the dependencies for your script into a temp venv that is only for your script, so its still pretty fast.
- embedding-shape 9mo agoI've moved over mostly to uv too, using `uv pip` when needed but mostly sticking with `uv add`. But as soon as you start using `uv pip` you end up with all the drawbacks of `uv pip`, namely that whatever you pass after can affect earlier dependency resolutions too. Running `uv pip install dep-a` and then `... dep-b` isn't the same as `... dep-b` first and then `... dep-a`, or the same as `uv pip install dep-a dep-b` which coming from an environment that does proper dependency resolution and have workspaces, can be really confusing. This is more of a pip issue than uv though, and `uv pip` is still preferable in my mind, but seems Python package management will forever be a mess, not even the bandaid uv can fix things like these.
- sieep 9mo agoIve been away from python for awhile now, I was under the impression uv was somehow solving this dependency hell. Whats the benefit of using uv/pip together? Speed?
- chuckadams 9mo ago`uv pip` is still uv, it's just uv's compatibility layer for pip.
- embedding-shape 9mo agoAs far as I can tell, `pip` by itself still doesn't even do something basic as resolving the dependency tree first, then download all the packages in parallel, as an basic example. The `uv pip` shim does. And regardless if you use only uv, or pip-via-uv, or straight up pip, dependencies you install later steps over dependencies you installed earlier, and no tool so far seems to try to solve this, which leads me to conclude it's a Python problem, not a package manager problem.
- micik 9mo agoi found uv frustrating. i dont know what problem is it trying to solve. it's not a tool for managing virtualenvs, but it does them as well. i guess it's a tool for dependency management. the "uv tool" stuff. kinda weird. i gave it an honest try but i was working around it with shell functions all the time. in the end i went back to good old virtualenvwrapper.sh and setting PYTHONPATH. full control over what goes into the venv and how. i guess people like writing new tools. i can understand that.
- t43562 9mo ago....but you have to be able to get UV and on some platforms (e.g. a raspberry pi) it won't build because the version of rust is too old. So I wrote a script called "pv" in python which works a bit like uv - just enough to get my program to work. It made me laugh a bit, but it works anywhere, well enough for my usecase. All I had to do was embed a primitive AI generated TOML parser in it.
- zahlman 9mo ago> All I had to do was embed a primitive AI generated TOML parser in it. The standard recommendation for this is `tomli`, which became the basis of the standard library `tomllib` in 3.11.
- zahlman 9mo agoThere are really so many things about this point that I don't get. First off, in my mind the kinds of things that are "scripts" don't have dependencies outside the standard library, or if they do are highly specific to my own needs on my own system. (It's also notable that one of the advantages the author cites for Go in this niche is a standard library that avoids the need for dependencies in quick scripts! Is this not one of Python's major selling points since day 1?) Second, even if you have dependencies you don't have to learn differences between these tools. You can pick one and use it. Third, virtual environments are literally just a place on disk for those dependencies to be installed, that contains a config file and some stubs that are automatically set up by a one-liner provided by the standard library. You don't need to go into them and inspect anything if you don't want to. You don't need to use the activation script; you can just specify the venv's executable instead if you prefer. None of it is conceptually difficult. Fourth, sharing an environment for these quick scripts actually just works fine an awful lot of the time. I got away with it for years before proper organization became second nature, and I would usually still be fine with it (except that having an isolated environment for the current project is the easiest way to be sure that I've correctly listed its dependencies). In my experience it's just not a thing for your quick throwaway scripts to be dependent on incompatible Numpy versions or whatever. ... And really, to avoid ever having to think about the dependencies you provide dynamically, you're going to switch to a compiled language? If it were such a good idea, nobody would have thought of making languages like Python in the first place. And uh... > As long as the receiving end has the latest version of go, the script will run on any OS for tens of years in the future. Anyone who's ever tried to get python working on different systems knows what a steep annoying curve it is. The pseudo-shebang trick here isn't going to work on Windows any more than a conventional one is. And no, when I switched from Windows to Linux, getting my Python stuff to work was not a "steep annoying curve" at all. It came more or less automatically with acclimating to Linux in general. (I guess referring to ".pyproject" instead of the actually-meaningful `pyproject.toml` is just part of the trolling.)
- kstrauser 9mo ago> Third, virtual environments are literally just a place on disk for those dependencies I had a recent conversation with a colleague. I said how nice it is using uv now. They said they were glad because they hated messing with virtualenvs so much that preferred TypeScript now. I asked them what node_modules is, they paused for a moment, and replied “point taken”. Uv still uses venvs because it’s the official way Python stores all the project packages in one place. Node/npm, Go/go, and Rust/cargo all do similar things, but I only really here people grousing about Python’s version, which as you say, you can totally ignore and never ever look at.
- JodieBenitez 9mo agoyou don't even need you prefered python version, uv will download it.
- the__alchemist 9mo agoI solved this in 2019 with PyFlow, but no one used it, so I lost interest. It's an OSS tool written in rust that automatically and transparently manages python versions and venvs. You just setup a `pyproject.toml`, run `pyflow main.py` etc, and it just works. Installs and locks dependencies like Cargo, installs and runs the correct Python version for the project etc. At the time, Poetry and Pipenv were the popular tools, but I found they were not sufficient; they did a good job abstracting dependencies, but not venvs and Python version.
- greensh 9mo agosounds awesome. Just out of interest, why do you think pyflow didn't catch on, but UV did?
- the__alchemist 9mo agoMy best guess: I'm bad at marketing, and gave up too soon. The feedback I received was generally "Why would I use this when Pip, Pipenv and Poetry work fine?". To me they didn't; they were a hassle due to not handling venvs and Py versions, but I didn't find many people to also have had the same problem.
- greensh 9mo agothanks for sharing. Marketing seems frustrating to me for an open source project. I had similar issues with python in the past and i wish I knew about this project back then.
- the_mitsuhiko 9mo agoPolish and that uv gets you entire python interpreters automatically without having to compile or manually install them. That in retrospective was what made rye temporarily attractive and popular.