6 ms·

For Python there is no native support for threads because of GIL. You can get around this through multiprocessing etc but these are hacks

pjmlp 9y ago
Only relevant on CPython, other implementations don't have a GIL.
- gebruikersnaam 9y ago
  PyPy has a GIL : http://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil-why http://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil-...
  - pjmlp 9y ago
    Ok, but ZipPy, Jython and IronPython do not.
    - weberc2 9y ago
      Have Jython and IronPython reached compatibility with Python 3 yet? Last I checked they hadn't, or it was still in alpha.
      - pjmlp 9y ago
        I don't know, but Python 3 still isn't relevant to many people. For example, I have to explicitly install 2.7 to be able to use Cocos2d-x build scripts.
        weberc2 9y ago
        Maybe, but it seems like a bad idea to invest in Python 2 (which is teaching is ended of life) just to get parallelism when there are other programming languages that have great concurrency stories and a bright future. That said, if PyPy3 ditches the GIL, let me know!
        throwaway7645 9y ago
        Someone posted an article awhile back...people are moving over by a significant amount now.
        sli 9y ago
        Yeah, "not relevant to many people" is a wildly false claim. If it were anywhere near true, we wouldn't see the level of Python 3 support that we have in the major packages (95.8%). http://py3readiness.org/ http://py3readiness.org/
      - throwaway7645 9y ago
        Yes, but ain't nobody really using Jython or IronPython. Yes some people are, but I'd bet a very small group of the greater Python whole. You also introduce incompatibilities that way. Perl6 has this in the base implementation, so I don't have to switch to a lesser maintained distribution.
    - dragonwriter 9y ago
      Jython project's latest release and news post was mid-2015; the project would seem to be pining for the fjords, so to speak. Jython and IronPython are both limited to Python 2.x, ZipPy is WIP without a stable release.
  - reubano 9y ago
    But not PyPySTM
askvictor 9y ago
There is native support for threads, it's just that they don't run concurrently.
- nothrabannosir 9y ago
  At this point the word is so far removed from the meaning in this thread you might as well say a haberdasher supports threads.
  - askvictor 9y ago
    There are valid reasons for programming with threads with only a single CPU (indeed when I learnedb thread programming in the 90s pretty much only single core cpus were available). For instance, IO blocking.
- weberc2 9y ago
  While we're being pedantic, they do run concurrently (for example, dispatching a dozen concurrent HTTP requests), they just the GIL prevents most CPU work from executing in parallel.
- dragonwriter 9y ago
  They do run in parallel, so long as all but one are running low-level library code which has released the GIL, but you can't call back into the runtime without grabbing the GIL.
e12e 9y ago
Afaik erlang has a lock as well - in fact I think most green thread implementations do: what you want/need at a minimum is one os process or thread per cpu core that works as a scheduler and then some form of (co-operative) scheduling with only user-mode/low-overhead context switching?
- yellowapple 9y ago
  That doesn't sound right re: Erlang; its everything-is-immutable model is supposed to avoid the need for a GIL. Erlang processes are also preemptive, so there's no possibility for a process to lock up the whole VM (unless it calls into a long-running NIF, but native code is always dangerous in a "crash BEAM if you don't get it right" kind of way).
  - dragonwriter 9y ago
    IIRC, Erlang uses shared mutable state internally, though its not exposed, so it needs to lock around accesses to that, but this isn't equivalent to the Python GIL.
- brightball 9y ago
  Scheduling and locking aren't the same thing. Here's a brief excerpt from Joe Armstrong that I found on SO while looking for a good explanation. > [Erlang] is a concurrent language – by that I mean that threads are part of the programming language, they do not belong to the operating system. That's really what's wrong with programming languages like Java and C++. It's threads aren't in the programming language, threads are something in the operating system – and they inherit all the problems that they have in the operating system. One of the problems is granularity of the memory management system. The memory management in the operating system protects whole pages of memory, so the smallest size that a thread can be is the smallest size of a page. That's actually too big. > If you add more memory to your machine – you have the same number of bits that protects the memory so the granularity of the page tables goes up – you end up using say 64kB for a process you know running in a few hundred bytes. With Erlang, there are threads running concurrently across multiple CPU's and a preemptive scheduler within each of those threads. This is all transparent through use of Erlang processes. Essentially, many concurrent schedulers managing hundreds of thousands of processes that start out at 0.5kb each. A JVM thread is 1024kb by comparison and a goroutine is 2kb. You're getting legitimate concurrency with Erlang, as well as isolation. There are only so many things that a single CPU can do at the same time and that's where the scheduler becomes necessary. The scheduler also ensure that if you fire off 100,000 processes and one of them is CPU intensive that it can't hog the processor and preventing the others allocated to that processor from executing.
anonacct37 9y ago
You might have chosen bad words, but people reading this should know this statement it false. Python has threads. You can create them and (if you're using linux) see them in the /proc filesystem. A more accurate statement is: "python bytecode can't execute concurrently w/o something like multiprocessing". The key difference? You can do multithreaded I/O all day long, which is a pretty important use case of threading. You can also hand off work to C and release the GIL, although this isn't as common. Example: XML parsing, regex, numeric work. FWIW all these gotchas are one of the reasons I love just having complete threading support built in to my language.
- chrisseaton 9y ago
  > A more accurate statement is: "python bytecode can't execute concurrently No this isn't any more accurate. Python bytecode does execute concurrently if you have multiple threads. It just doesn't execute in parallel.
- ensiferum 9y ago
  In fact it is quite common. If you look into the c modules you see a pattern where they release the gil before performing some operation and then reacquire it

pjmlp 9y ago

Only relevant on CPython, other implementations don't have a GIL.

gebruikersnaam 9y ago
PyPy has a GIL : http://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil-why http://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil-...
- pjmlp 9y ago
  Ok, but ZipPy, Jython and IronPython do not.
  - weberc2 9y ago
    Have Jython and IronPython reached compatibility with Python 3 yet? Last I checked they hadn't, or it was still in alpha.
    - pjmlp 9y ago
      I don't know, but Python 3 still isn't relevant to many people. For example, I have to explicitly install 2.7 to be able to use Cocos2d-x build scripts.
      - weberc2 9y ago
        Maybe, but it seems like a bad idea to invest in Python 2 (which is teaching is ended of life) just to get parallelism when there are other programming languages that have great concurrency stories and a bright future. That said, if PyPy3 ditches the GIL, let me know!
      - throwaway7645 9y ago
        Someone posted an article awhile back...people are moving over by a significant amount now.
        sli 9y ago
        Yeah, "not relevant to many people" is a wildly false claim. If it were anywhere near true, we wouldn't see the level of Python 3 support that we have in the major packages (95.8%). http://py3readiness.org/ http://py3readiness.org/
    - throwaway7645 9y ago
      Yes, but ain't nobody really using Jython or IronPython. Yes some people are, but I'd bet a very small group of the greater Python whole. You also introduce incompatibilities that way. Perl6 has this in the base implementation, so I don't have to switch to a lesser maintained distribution.
  - dragonwriter 9y ago
    Jython project's latest release and news post was mid-2015; the project would seem to be pining for the fjords, so to speak. Jython and IronPython are both limited to Python 2.x, ZipPy is WIP without a stable release.
- reubano 9y ago
  But not PyPySTM

askvictor 9y ago

There is native support for threads, it's just that they don't run concurrently.

nothrabannosir 9y ago
At this point the word is so far removed from the meaning in this thread you might as well say a haberdasher supports threads.
- askvictor 9y ago
  There are valid reasons for programming with threads with only a single CPU (indeed when I learnedb thread programming in the 90s pretty much only single core cpus were available). For instance, IO blocking.
weberc2 9y ago
While we're being pedantic, they do run concurrently (for example, dispatching a dozen concurrent HTTP requests), they just the GIL prevents most CPU work from executing in parallel.
dragonwriter 9y ago
They do run in parallel, so long as all but one are running low-level library code which has released the GIL, but you can't call back into the runtime without grabbing the GIL.

e12e 9y ago

Afaik erlang has a lock as well - in fact I think most green thread implementations do: what you want/need at a minimum is one os process or thread per cpu core that works as a scheduler and then some form of (co-operative) scheduling with only user-mode/low-overhead context switching?

yellowapple 9y ago
That doesn't sound right re: Erlang; its everything-is-immutable model is supposed to avoid the need for a GIL. Erlang processes are also preemptive, so there's no possibility for a process to lock up the whole VM (unless it calls into a long-running NIF, but native code is always dangerous in a "crash BEAM if you don't get it right" kind of way).
- dragonwriter 9y ago
  IIRC, Erlang uses shared mutable state internally, though its not exposed, so it needs to lock around accesses to that, but this isn't equivalent to the Python GIL.
brightball 9y ago
Scheduling and locking aren't the same thing. Here's a brief excerpt from Joe Armstrong that I found on SO while looking for a good explanation. > [Erlang] is a concurrent language – by that I mean that threads are part of the programming language, they do not belong to the operating system. That's really what's wrong with programming languages like Java and C++. It's threads aren't in the programming language, threads are something in the operating system – and they inherit all the problems that they have in the operating system. One of the problems is granularity of the memory management system. The memory management in the operating system protects whole pages of memory, so the smallest size that a thread can be is the smallest size of a page. That's actually too big. > If you add more memory to your machine – you have the same number of bits that protects the memory so the granularity of the page tables goes up – you end up using say 64kB for a process you know running in a few hundred bytes. With Erlang, there are threads running concurrently across multiple CPU's and a preemptive scheduler within each of those threads. This is all transparent through use of Erlang processes. Essentially, many concurrent schedulers managing hundreds of thousands of processes that start out at 0.5kb each. A JVM thread is 1024kb by comparison and a goroutine is 2kb. You're getting legitimate concurrency with Erlang, as well as isolation. There are only so many things that a single CPU can do at the same time and that's where the scheduler becomes necessary. The scheduler also ensure that if you fire off 100,000 processes and one of them is CPU intensive that it can't hog the processor and preventing the others allocated to that processor from executing.

anonacct37 9y ago

You might have chosen bad words, but people reading this should know this statement it false. Python has threads. You can create them and (if you're using linux) see them in the /proc filesystem. A more accurate statement is: "python bytecode can't execute concurrently w/o something like multiprocessing". The key difference? You can do multithreaded I/O all day long, which is a pretty important use case of threading. You can also hand off work to C and release the GIL, although this isn't as common. Example: XML parsing, regex, numeric work. FWIW all these gotchas are one of the reasons I love just having complete threading support built in to my language.

chrisseaton 9y ago
> A more accurate statement is: "python bytecode can't execute concurrently No this isn't any more accurate. Python bytecode does execute concurrently if you have multiple threads. It just doesn't execute in parallel.
ensiferum 9y ago
In fact it is quite common. If you look into the c modules you see a pattern where they release the gil before performing some operation and then reacquire it