5 ms·

They may be fast in C++, but not in the context of CPython. Here are the dirty details. Note that fine-grained locking has also been tried before: https://dabe

• nemetroid 2y ago
  Thanks for the link, that's an interesting read. Actually the referenced PyMutex is a good old pthread_mutex_t, the same you'd use in C or C++. But I shouldn't have written so surely. Although uncontested locks are very fast, if the loop is tight enough, adding locks will be significant. However, PEP 703 specifically points out that performance-critical container operations (__getitem__/iteration) avoid locking, so I'm still highly skeptical that those locks are the cause of the 30-50%. https://peps.python.org/pep-0703/#optimistically-avoiding-locking https://peps.python.org/pep-0703/#optimistically-avoiding-lo...
  • tialaramex 2y ago
    The pthread_mutex_t is focused on compatibility at any cost. So while you're right that the C++ stdlib chooses this too, it's not actually a good choice for performance. But I think you're right be sceptical that somehow this is to blame for the Python perf leak.
• tialaramex 2y ago
  One of the things this spends some time on that was already obsolete in 2011 is using a pool of locks. In 1994 locks are a limited OS resource, Python can't afford to sprinkle millions of them in the codebase. But long before 2011 Linux had the futex, so locks only need to be aligned 32-bit integers. In 2012 Windows gets a similar feature but it can do bytes instead of 32-bit integers if you want. If a Linux process wants a million locks that's fine, that's just 4MB of RAM now.