5 ms·

Lua version of the code is incomprehensible. Could it be that if you implement the same thing in Python it will also run faster?

by andy-x 5y ago

byteface 5y ago
you mean with buffers i.e. array.array . i thought that.
munch117 5y ago
Much of the difference is due to rewriting complex numbers into pairs of floating-point variables. Probably because Lua doesn't have complex numbers? The same rewrite in Python would not speed things up, it would slow things down. It's more than that, though. It looks like the Lua code does only 49 repeats where the Python code does 200. It's clear that this isn't just a Python->Lua rewrite, this is a rewrite that was followed by a lot of optimisation work on the Lua code. Despite that, I kinda buy it. I can believe that you won't get speeds in pure-Python code for this particular problem that are competitive with a JIT. numpy doesn't help, because the number of iterations varies for each point, so you can't easily parallelise. And speaking of parallelisation, he has a point about threading: Doing the computations in pure-Python would not perform well over multiple threads, you need Cython or something. And that something can apparently be Lua.
- kzrdude 5y ago
  It turns out using numpy is enough for Python to parallelize this adequately using threads, see this code: https://news.ycombinator.com/item?id=28817357 https://news.ycombinator.com/item?id=28817357
  - munch117 5y ago
    It's not the same algorithm: That code completes all 200 iterations unconditionally. For some parts of the input domain, the numpy speed-up is enough to compensate and make it faster anyway, but for other parts it will be slower.
    - kzrdude 5y ago
      Agree, even if it compensates by supplying a gradually shrinking compute mask, but not to all operations. But an effect is there, less populated chunks compute faster.
kzrdude 5y ago
No. Lua here uses luajit, a jit compiler and interpreter for Lua which Python can not beat. Python has the numba package, which is in the same direction, less easy to use, more restricted, and I don't know if numba can approach luajit's speed here, we'd have to try it out. (After working on the numpy code posted in this thread, I think Python is pretty competitive, but it requires quite a few contortions - rewriting it using numpy.)

Lua version of the code is incomprehensible. Could it be that if you implement the same thing in Python it will also run faster?

by andy-x 5y ago

Lua version of the code is incomprehensible. Could it be that if you implement the same thing in Python it will also run faster?

byteface 5y ago

you mean with buffers i.e. array.array . i thought that.

munch117 5y ago

Much of the difference is due to rewriting complex numbers into pairs of floating-point variables. Probably because Lua doesn't have complex numbers? The same rewrite in Python would not speed things up, it would slow things down. It's more than that, though. It looks like the Lua code does only 49 repeats where the Python code does 200. It's clear that this isn't just a Python->Lua rewrite, this is a rewrite that was followed by a lot of optimisation work on the Lua code. Despite that, I kinda buy it. I can believe that you won't get speeds in pure-Python code for this particular problem that are competitive with a JIT. numpy doesn't help, because the number of iterations varies for each point, so you can't easily parallelise. And speaking of parallelisation, he has a point about threading: Doing the computations in pure-Python would not perform well over multiple threads, you need Cython or something. And that something can apparently be Lua.

kzrdude 5y ago
It turns out using numpy is enough for Python to parallelize this adequately using threads, see this code: https://news.ycombinator.com/item?id=28817357 https://news.ycombinator.com/item?id=28817357
- munch117 5y ago
  It's not the same algorithm: That code completes all 200 iterations unconditionally. For some parts of the input domain, the numpy speed-up is enough to compensate and make it faster anyway, but for other parts it will be slower.
  - kzrdude 5y ago
    Agree, even if it compensates by supplying a gradually shrinking compute mask, but not to all operations. But an effect is there, less populated chunks compute faster.

kzrdude 5y ago

No. Lua here uses luajit, a jit compiler and interpreter for Lua which Python can not beat. Python has the numba package, which is in the same direction, less easy to use, more restricted, and I don't know if numba can approach luajit's speed here, we'd have to try it out. (After working on the numpy code posted in this thread, I think Python is pretty competitive, but it requires quite a few contortions - rewriting it using numpy.)