7 ms·

Removing the GIL sounds like it will make typical Python programs slower and will introduce a lot of complexity? What is the real world benefit we will get in

• simonw 2y ago
  My hunch is that in just a few years time single core computers will be almost extinct. Removing the GIL now feels to me like good strategic preparation for the near future.
  • im3w1l 2y ago
    Single core computers yes. Single core containers though..
    • gomerspiles 2y ago
      Single core containers are also a terrible idea. Life got much less deadlocked as soon as there were 2+ processors everywhere. (Huh, people like hard OS design problems for marginal behavior? OSes had trouble adopting SMP but we also got to jettison a lot of deadlock discussions as soon as there was CPU 2. It only takes a few people not prioritizing 1 CPU testing at any layer to make your 1 CPU container much worse than a 2 VCPU container limited to a 1 CPU average.)
    • seabrookmx 2y ago
      It's actually quite difficult to get a "single core" container (ie: a container with access to only one logical processor). When you set "request: 1" in Kubernetes or another container manager, you're saying "give me 1 CPU worth of CPU time" but if the underlying Linux host has 16 logical cores your container will still see them. Your container is free to use 1/16th of each of them, 100% of one of them, or anything in-between. You might think this doesn't matter in the end but it can if you have a lot of workloads on that node and those cores are busy. Your single threaded throughout can become quite compromised as a result.
      • Dylan16807 2y ago
        > if you have a lot of workloads on that node and those cores are busy. Your single threaded throughout can become quite compromised as a result. While yes this can cause a slowdown, wouldn't it still happen if each container thought it had a single core?
        • seabrookmx 2y ago
          Only if you have more containers than cores.
          • Dylan16807 2y ago
            That depends on what your scheduler does. Having one virtual core doesn't necessarily mean you always get the same physical core. Also you said "a lot of workloads" so yes probably more containers than cores.
            • seabrookmx 2y ago
              Most of my pods have a CPU request >= 1 so more containers than cores is rare. But obviously that really depends on your workload(s). I don't think the scheduler picking a different core matters much unless your workload is super cache sensitive. My point is more about access to single threaded performance. If you have a single threaded workload (ex: an ffmpeg audio encode) and you want it to be able to access as many cycles from a single core as possible, it isn't always as simple as request: 1
      • 3np 2y ago
        It's easy, though? On Docker, --cpuset-cpus=0 will pin the container to the first core. K8s: https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/ https://kubernetes.io/docs/tasks/administer-cluster/cpu-mana... CPU affinity and pinning is something I think you should be able to achieve without too much hassle.
        • edude03 2y ago
  • naming_the_user 2y ago
    It depends what you mean by extinct. I can't think of any actual computer outside of embedded that has been single core for at least a decade. The Core Duo and Athlon X2 were released almost 20 years ago now and within a few years basically everything was multicore. (When did we get old?) If you mean that single core workloads will be extinct, well, that's a harder sell.
    • simonw 2y ago
      Yeah, I just checked and even a RaspberryPi has four cores these days. So I guess they went extinct a long time ago!
      • formerly_proven 2y ago
        Even many microcontrollers have multiple cores nowadays. It’s not the norm just yet, though.
      • poincaredisk 2y ago
        Yes, but: * Most of the programs I write are not (trivially) parallelizable, and a the bottleneck is still a single core performance * There is more than one process at any time, especially on servers. Other cores are also busy and have their own work to do.
        • deadbunny 2y ago
          Yes, but: 1. Other people with different needs exist. 2. That's why we have schedulers.
  • deleted 2y ago
    [deleted]
  • masklinn 2y ago
    > My hunch is that in just a few years time single core computers will be almost extinct. Single core computers are already functionally extinct, but single-threaded programs are not.
    • pjmlp 2y ago
      Depends on the OS, on Windows or Android, even single processes have multiple threads under the hood.
• cma 2y ago
  There will be consumer chips with 64 cores before long
• tonygrue 2y ago
  There is an argument that if you need in process multithreading you should use a different language. But a lot of people need to use python because everything else they’re doing is in python. There are quite a few common cases where in process multi threading is useful. The main ones are where you have large inputs or large outputs to the work units. In process is nice because you can move the input or output state to the work units instead of having to copy it. One very common case is almost all gui applications. Where you want to be able to do all work on background threads and just move data back and forth from the coordinating ui thread. JavaScript’s lack of support here, outside of a native language compiled into emscripten, is one reason web apps are so hard to make jankless. The copies of data across web workers or python processes are quite expensive as far as things go. Once a week or so, I run into a high compute python scenario where the existing forms of multiprocessing fail me. Large shared inputs and or don’t want the multiprocess overhead; but GIL slows everything down.
  • vlovich123 2y ago
    > Where you want to be able to do all work on background threads and just move data back and forth from the coordinating ui thread. JavaScript’s lack of support here, outside of a native language compiled into emscripten, is one reason web apps are so hard to make jankless I thought transferring array buffers through web workers didn’t involve any copies of you actually transferred ownership: worker.postMessage(view.buffer, [view.buffer]); I can understand that web workers might be more annoying to orchestrate than native threads and the like but I’m not sure that it lacks the primitives to make it possible. More likely it’s really hard to have a pauseless GC for JS (Python predominantly relies on reference counting and uses gc just to catch cycles).
    • Etheryte 2y ago
      This is true, but when do you really work with array buffers in Javascript? The default choice for whatever it is that you're doing is almost always something else, save for a few edge cases, and then you're stuck trying to bend your business logic to a different data type.
• Zyten 2y ago
  What you’re describing is basically using MPI in some way, shape or form. This works, but also can introduce a lot of complexity. If your program doesn’t need to communicate, then it’s easy. But that’s not the case for all programs. Especially once we’re talking about simulations and other applications running on HPC systems. Sometimes it’s also easier to split work using multiple threads. Other programming languages let you do that and actually use multiple threads efficiently. In Python, the benefit was just too limited due to the GIL.
• lifthrasiir 2y ago
  > Removing the GIL sounds like it will make typical Python programs slower and will introduce a lot of complexity? This was the original reason for CPython to retain GIL for very long time, and probably true for most of that time. That's why the eventual GIL removal had to be paired with other important performance improvements like JIT, which was only implemented after some feasible paths were found and explicitly funded by a big sponsor.
  • klranh 2y ago
    That is the official story. None of it has materialized so far.
    • lifthrasiir 2y ago
      Python development is done in public so you can just benchmark against the development version to see its improvement. In fact, daily benchmarks are already posted to [1]; it indicates around 20% improvement (corresponding to 1.25x in the table) since 3.10. The only thing you can't easily verify is that whether GIL was indeed historically necessary in the past. [1] https://github.com/faster-cpython/benchmarking-public https://github.com/faster-cpython/benchmarking-public
• pansa2 2y ago
  > What is the real world benefit we will get in return? If you have many CPU cores and an embarrassingly parallel algorithm, multi-threaded Python can now approach the performance of a single-threaded compiled language.
  • Certhas 2y ago
    The question really is if one couldn't make multiprocess better instead of multithreaded. I did a ton of MPI work with python ten years ago already. What's more I am now seeing in Julia that multithreading doesn't scale to larger core counts (like 128) due to the garbage collector. I had to revert to multithreaded again.
    • Sakos 2y ago
      I assume you meant you had to revert to multiprocess?
      • Certhas 2y ago
        Yes exactly. Thanks.
  • 0x000xca0xfe 2y ago
    You could already easily parallelize with the multiprocessing module. The real difference is the lower communication overhead between threads vs. processes thanks to a shared address space.
    • mharig 2y ago
      [dead]
    • bmitc 2y ago
      Easily is an overstatement. Multiprocessing is fraught with quirks.
      • 2y ago
• bdd8f1df777b 2y ago
  The biggest use case (that I am aware of) of GIL-less Python is for parallel feeding data into ML model training. * PyTorch currently uses `multiprocessing` for that, but it is fraught with bugs and with less than ideal performance, which is sorely needed for ML training (it can starve the GPU). * Tensorflow just discards Python for data loading. Its data loaders are actually in C++ so it has no performance problems. But it is so inflexible that it is always painful for me to load data in TF. Given how hot ML is, and how Python is currently the major language for ML, it makes sense for them to optimize for this.
• carapace 2y ago
  > What is the real world benefit we will get in return? None. I've been using Python "in anger" for twenty years and the GIL has been a problem zero times. It seems to me that removing the GIL will only make for more difficulty in debugging.
• inoop 2y ago
  As always, it depends a lot on what you're doing, and a lot of people are using Python for AI. One of the drawbacks of multi-processing versus multi-threading is that you cannot share memory (easily, cheaply) between processes. During model training, and even during inference, this becomes a problem. For example, imagine a high volume, low latency, synchronous computer vision inference service. If you're handling each request in a different process, then you're going to have to jump through a bunch of hoops to make this performant. For example, you'll need to use shared memory to move data around, because images are large, and sockets are slow. Another issue is that each process will need a different copy of the model in GPU memory, which is a problem in a world where GPU memory is at a premium. You could of course have a single process for the GPU processing part of your model, and then automatically batch inputs into this process, etc. etc. (and people do) but all this is just to work around the lack of proper threading support in Python. By the way, if anyone is struggling with these challenges today, I recommend taking a peek at nvidia's Triton inference server (https://github.com/triton-inference-server/server https://github.com/triton-inference-server/server), which handles a lot of these details for you. It supports things like zero-copy sharing of tensors between parts of your model running in different processes/threads and does auto-batching between requests as well. Especially auto-batching gave us big throughput increase with a minor latency penalty!
  • saagarjha 2y ago
    Machine learning people not call their thing Triton challenge (IMPOSSIBLE)
    • buildbot 2y ago
      This (Nvidia’s) triton predates openAI’s by a few years.
  • jgraettinger1 2y ago
    > For example, imagine a high volume, low latency, synchronous computer vision inference service. I'm not in this space and this is probably too simplistic, but I would think pairing asyncio to do all IO (reading / decoding requests and preparing them for inference) coupled with asyncio.to_thread'd calls to do_inference_in_C_with_the_GIL_released(my_prepared_request), would get you nearly all of the performance benefit using current Python.
• deleted 2y ago
  [deleted]
• imron 2y ago
  > Removing the GIL sounds like it will make typical Python programs slower and will introduce a lot of complexity? There is a lot of Python code that either explicitly (or implicitly) relies on the GIL for correctness in multithreaded programs. I myself have even written such code, explicitly relying on the GIL as synchronization primitive. Removing the GIL will break that code in subtle and difficult to track down ways. The good news is that a large percentage of this code will stay running on older versions of python (2.7 even) and so will always have a GIL around. Some of it however will end up running on no-GIL python and I don't envy the developers who will be tasked tracking down the bugs - but probably they will run on modern versions of python using --with-gil or whatever other flag is provided to enable the GIL. The benefit to the rest of the world then is that future programs will be able to take advantage of multiple cores with shared memory, without needing to jump through the hoops of multi-process Python. Python has been feeling the pain of the GIL in this area for many years already, and removing the GIL will make Python more viable for a whole host of applications.
• deleted 2y ago
  [deleted]