4 ms·

I added an implementation [1] in diesel [2][3], which uses select.epoll (or libev, on non-Linux systems) and got a around 150x speedup [4]. I only repeated the

I added an implementation [1] in diesel [2][3], which uses select.epoll (or libev, on non-Linux systems) and got a around 150x speedup [4]. I only repeated the tests a few times (but they were all close) and didn't install the Go compiler so I could test against Go (I'd be interested to see how this stacks up on your machine). Like you say in your post, it's nice to have something wrap up the bother of epoll for you. [1] https://github.com/wmoss/Key-Value-Polyglot https://github.com/wmoss/Key-Value-Polyglot [2] diesel.io [3] https://github.com/jamwt/diesel https://github.com/jamwt/diesel [4] The first run is against the diesel one wmoss@wmoss-mba:~/etc/Key-Value-Polyglot$ time python test.py real 0m0.134s user 0m0.040s sys 0m0.020s wmoss@wmoss-mba:~/etc/Key-Value-Polyglot$ time python test.py real 0m20.164s user 0m0.096s sys 0m0.072s

jamwt 15y ago
also, the diesel one is half the LOC /shameless plug :-) (for the record, on my machine the go comparison was 97ms vs. 173ms, so pure python + diesel was 1.78x slower)
codeape 15y ago
I lack knowledge on networking/event-based systems on a fundamental level. Here's what I don't understand: * test.py is sequential: It first does 500 sets then 500 gets, all in one thread, using a single connection to the server. * The socket handling function (memg.py:handle_con/memg-diesel.py:handle_con) is called once. There is no parallell execution going on. * So why is the memg-diesel.py code so much faster? What makes the code for sending and receiving data to/from the socket so much faster? Could someone please explain to me why an epoll-based solution is so much faster?
chmike 15y ago
What is the difference between diesel and gevent ? Note that gevent 1.0 uses libev.
- willbmoss 15y ago
  In a nutshell, gevent monkey patches the socket library, whereas diesel doesn't. This means that you can use any (previously) blocking libraries with gevent, whereas, in diesel you have to write them again. The upside of the rewriting is that it creates a more coherent (and opinionated) ecosystem.
- jamwt 15y ago
  http://diesel.io/faq http://diesel.io/faq

jamwt 15y ago

also, the diesel one is half the LOC /shameless plug :-) (for the record, on my machine the go comparison was 97ms vs. 173ms, so pure python + diesel was 1.78x slower)

codeape 15y ago

I lack knowledge on networking/event-based systems on a fundamental level. Here's what I don't understand: * test.py is sequential: It first does 500 sets then 500 gets, all in one thread, using a single connection to the server. * The socket handling function (memg.py:handle_con/memg-diesel.py:handle_con) is called once. There is no parallell execution going on. * So why is the memg-diesel.py code so much faster? What makes the code for sending and receiving data to/from the socket so much faster? Could someone please explain to me why an epoll-based solution is so much faster?

chmike 15y ago

What is the difference between diesel and gevent ? Note that gevent 1.0 uses libev.

willbmoss 15y ago
In a nutshell, gevent monkey patches the socket library, whereas diesel doesn't. This means that you can use any (previously) blocking libraries with gevent, whereas, in diesel you have to write them again. The upside of the rewriting is that it creates a more coherent (and opinionated) ecosystem.
jamwt 15y ago
http://diesel.io/faq http://diesel.io/faq