6 ms·

> This pattern, for example, can’t be lazy: try: import numpy; except ModuleNotFoundError: ... Wait, that seems bad. These kinds of modules (especially numba)

> This pattern, for example, can’t be lazy: try: import numpy; except ModuleNotFoundError: ... Wait, that seems bad. These kinds of modules (especially numba) are often exactly the ones that you want to delay importing. Why not provide a way to check the existence at import time? How are you supposed to handle nonexistence in that case?

zbentley 15d ago
...crash? Seriously: crash if your dependencies aren't available. There are better ways to do optional dependencies. ImportError ain't it.
- dataflow 15d ago
  What is the right way to handle an optional dependency?
  - rtpg 15d ago
    I think this import flow tends to be the canonical one. I might recommend doing something like importing from something like `numpy.version` (or some other "random" very small utils package) so that the work done on file load is still fairly small.
    - dataflow 14d ago
      It's not clear such a thing even exists in general though. It's certainly not guaranteed by packages. What would that be for numba?
      - rtpg 13d ago
        Yep it's not clear, and a bit of a case-by-case thing. Fortunately most codebases only need to do this in a handful of spots so you can really "just" look at a package and find a file that probably is safe. For example, numba._version only imports standard library stuff so is probably good enough here[0] It would be nice to have general querying capabilities here, of course. I just think that in practice there's at least a halfway-decent workaround in almost any real situation [0] https://github.com/numba/numba/blob/main/numba/_version.py https://github.com/numba/numba/blob/main/numba/_version.py
    - zbentley 14d ago
      Common? Yes. Canonical/ideal? No. Optional dependency specifiers in your package metadata is a much better way to go. You can use those even if you don't publish your package as an artifact; `pip install -e .[optional-thing]` should work, or can be made to, easily.
      - rtpg 14d ago
        That's only part of the puzzle. Your program needs to branch on whether the dependency is installed or not! "Oh you don't have optional dependency? Then do X" needs to happen at runtime right?
        zbentley 11d ago
        I explained further here: https://news.ycombinator.com/item?id=49604462 https://news.ycombinator.com/item?id=49604462 In my preference order: an explicit CLI option/envvar which requires the optional dep, a flag file on the filesystem, or an ImportError attempt if you absolutely must. Use a stub/trampoline file in the conditionals so that static analyzers and laziness analysis on that file itself still work.
  - zbentley 14d ago
    https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#dependencies-optional-dependencies https://packaging.python.org/en/latest/guides/writing-pyproj...
    - dataflow 14d ago
      I don't understand, sorry. How does declaring an optional dependency in a toml file solve the problem of accessing the symbol in your code? Also, you need something that works for simple one-file scripts, not just a whole package or project.
      - zbentley 11d ago
        Declare the optional dependency and then import it unconditionally if the optional is installed. You can conditionally import a file you control based on asking whether an optional specifier was installed, either by probing the filesystem, a CLI argument, an environment variable, or whatever folks come up with for https://github.com/pypa/packaging-problems/issues/215 https://github.com/pypa/packaging-problems/issues/215. The "if: ... import" laziness-defeating clause only applies to your stub module, and the imports in that module are unconditional and thus will work with analyzers/laziness/etc. If you need something for one-off scripts, use https://docs.astral.sh/uv/guides/scripts/ https://docs.astral.sh/uv/guides/scripts/.
js2 15d ago
> The error here will move to the first usage of something from numpy. There is a semi-lazy alternative: import importlib.util if importlib.util.find_spec("numpy") is None: ... # whatever you wanted to do if numpy is missing lazy import numpy

zbentley 15d ago

...crash? Seriously: crash if your dependencies aren't available. There are better ways to do optional dependencies. ImportError ain't it.

dataflow 15d ago
What is the right way to handle an optional dependency?
- rtpg 15d ago
  I think this import flow tends to be the canonical one. I might recommend doing something like importing from something like `numpy.version` (or some other "random" very small utils package) so that the work done on file load is still fairly small.
  - dataflow 14d ago
    It's not clear such a thing even exists in general though. It's certainly not guaranteed by packages. What would that be for numba?
    - rtpg 13d ago
      Yep it's not clear, and a bit of a case-by-case thing. Fortunately most codebases only need to do this in a handful of spots so you can really "just" look at a package and find a file that probably is safe. For example, numba._version only imports standard library stuff so is probably good enough here[0] It would be nice to have general querying capabilities here, of course. I just think that in practice there's at least a halfway-decent workaround in almost any real situation [0] https://github.com/numba/numba/blob/main/numba/_version.py https://github.com/numba/numba/blob/main/numba/_version.py
  - zbentley 14d ago
    Common? Yes. Canonical/ideal? No. Optional dependency specifiers in your package metadata is a much better way to go. You can use those even if you don't publish your package as an artifact; `pip install -e .[optional-thing]` should work, or can be made to, easily.
    - rtpg 14d ago
      That's only part of the puzzle. Your program needs to branch on whether the dependency is installed or not! "Oh you don't have optional dependency? Then do X" needs to happen at runtime right?
      - zbentley 11d ago
        I explained further here: https://news.ycombinator.com/item?id=49604462 https://news.ycombinator.com/item?id=49604462 In my preference order: an explicit CLI option/envvar which requires the optional dep, a flag file on the filesystem, or an ImportError attempt if you absolutely must. Use a stub/trampoline file in the conditionals so that static analyzers and laziness analysis on that file itself still work.
- zbentley 14d ago
  https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#dependencies-optional-dependencies https://packaging.python.org/en/latest/guides/writing-pyproj...
  - dataflow 14d ago
    I don't understand, sorry. How does declaring an optional dependency in a toml file solve the problem of accessing the symbol in your code? Also, you need something that works for simple one-file scripts, not just a whole package or project.
    - zbentley 11d ago
      Declare the optional dependency and then import it unconditionally if the optional is installed. You can conditionally import a file you control based on asking whether an optional specifier was installed, either by probing the filesystem, a CLI argument, an environment variable, or whatever folks come up with for https://github.com/pypa/packaging-problems/issues/215 https://github.com/pypa/packaging-problems/issues/215. The "if: ... import" laziness-defeating clause only applies to your stub module, and the imports in that module are unconditional and thus will work with analyzers/laziness/etc. If you need something for one-off scripts, use https://docs.astral.sh/uv/guides/scripts/ https://docs.astral.sh/uv/guides/scripts/.

js2 15d ago

> The error here will move to the first usage of something from numpy. There is a semi-lazy alternative: import importlib.util if importlib.util.find_spec("numpy") is None: ... # whatever you wanted to do if numpy is missing lazy import numpy