6 ms·

As I read it that's one AWS server of unknown size serving ~500rps of an application of unknown complexity. Tripled when using PyPy (whatever that is). Maybe a

As I read it that's one AWS server of unknown size serving ~500rps of an application of unknown complexity. Tripled when using PyPy (whatever that is). Maybe a very small instance serving something fairly complex?

bkeroack 12y ago
Yeah, I read it as 500 rps per instance, which is not unreasonable for AWS.
valarauca1 12y ago
How complex? In one second a modern processor can do ~1 billion operations (ish, some are faster, some are slower, sometimes multiple are done in the same clock tick). Even if its slow, core2 architecture. This means they have the time for about ~200 million instructions per request (Ignoring internal disk I/O, or network I/O). That amount of work is insane! :.:.: I want to say their doing something fundamentally wrong. And it has nothing to do with their language.
- tachyonbeam 12y ago
  The i7 can dispatch 4 instructions per cycle. In practice, I find it can realistically execute about 2 per cycle. So at 2GHz, that's closer to 4 billion instructions per second, or ~800M instructions per request as per your calculation. The slowness of their system can probably be blamed on slow database access, or some kind of initialization cost they're paying for every request (i.e.: calling into a binary like in the old CGI days, initializing the Python VM every time).
  - valarauca1 12y ago
    Like I said something fundamentally wrong. With their approach. Un-Indexed databases, databases far away from front end servers (in terms of network topography), or weird VM things with python. Something is bad, and if changing languages solved their problems, they are just sweeping an issue under the rug. It'll hit them in the face later, and harder. Be it developer knowledge, or architectural choices. It'll surface again, they'll (hopefully) be large, and the problem will sting harder too.
- tomwilde 12y ago
  Unless you're a mathematician or theoretical physicist the gross of your CPU time will be spent waiting for IO. Reading from disk, writing to the network, synchronizing, etc. They're probably just aggregating 2 or 3 APIs, maybe hitting a database and then adding it all together. That description can apply to almost any and all web applications and is inherently IO-bound.
  - copx 12y ago
    >Unless you're a mathematician or theoretical physicist [...] ..or Dwarf Fortress player ;)
    - valarauca1 12y ago
      Oooh thank you for reminding me. I recently upgraded to a i7-4790k I've been meaning to jump back into DF now that I should have much better single threaded performance. I honestly haven't played since 2008 on my parents home pentium3.
  - falcolas 12y ago
    > That description can apply to almost any and all web applications and is inherently IO-bound. If their system was truly just IO bound, then moving to D wouldn't help them.
    - waps 12y ago
      That's not true. Well it's true in a very narrow technical sense, but it's not really true. For example, the amount of housekeeping python does in order to execute a function call is staggering. It leads to all sorts of nice functionality, but nevertheless (plus C++/D does it almost entirely without housekeeping. Either no housekeeping, or 1 level of indirection). Python has so many indirections for a function call it hardly even makes sense to talk about it in numbers of indirects. Assembly hello world on my machine : 86,607 cpu cycles (of which < 20 actually in the program) Syscalls used by the assembly version : 2 (write and exit) Python hello world on my machine (.pyc was available) : 59,099,731 instructions (including half a million branch misses) Syscalls used by python to execute 'print "hello, world"' : 1139 (each of which causes a program reschedule) These programs do the same thing. Programmers often forget that things they take for granted are not in fact free, they may not even be O(1). Memory allocation. Subprocess execution. Function calls in scripting languages. Syscalls. Writing to files. Allocation of bytes on disks. All of these things come at a really, really high cost, and most not even O(1) costs (e.g. memory allocation is O(N^2) on a busy server as long as things actually fit in main memory, and O(N^4) or even worse when using virtual memory). Sadly using memory does not even have bounded complexity. At some point, just attempting to use virtual memory might cause virtual memory to be allocated just for the lookup. This is generally referred to as "thrashing" and you're very likely to have rebooted your machine before this completes because it'll be frozen for minutes, sometimes hours, if this happens. Likewise the memory model is useful, but huge. Strings in C++ take one byte + the actual contents of the string. Strings in python take up 60 bytes + twice the length of the string. And that's assuming you just set a variable to the string. If you construct the string, the difference is going to be much bigger. The point here is that things that are io-bound (esp. memory bound) in python may be cpu-bound in C++ or D, simply because you avoid doing all the indirections that higher level languages do.
- deleted 12y ago
  [deleted]
loxs 12y ago
Yep, I could readily point to my Erlang application that can serve thousands of requests per second, doing real work. While simultaneously handling 10k+ websocket connections. All that on a single machine (16G RAM, 8 cores). Does that mean that Erlang is the best? Of course not! It only means that Erlang was specifically designed for this sole use case, which only happens to encompass the whole "web" thingie. It would be an overkill to use Erlang for single-threaded software that requires number crunching speed or for any kind of system scripting. Comparing Erlang to D or Python is stupid. As is (IMHO) using any of the latter for massively parallel servers.

bkeroack 12y ago

Yeah, I read it as 500 rps per instance, which is not unreasonable for AWS.

valarauca1 12y ago

How complex? In one second a modern processor can do ~1 billion operations (ish, some are faster, some are slower, sometimes multiple are done in the same clock tick). Even if its slow, core2 architecture. This means they have the time for about ~200 million instructions per request (Ignoring internal disk I/O, or network I/O). That amount of work is insane! :.:.: I want to say their doing something fundamentally wrong. And it has nothing to do with their language.

tachyonbeam 12y ago
The i7 can dispatch 4 instructions per cycle. In practice, I find it can realistically execute about 2 per cycle. So at 2GHz, that's closer to 4 billion instructions per second, or ~800M instructions per request as per your calculation. The slowness of their system can probably be blamed on slow database access, or some kind of initialization cost they're paying for every request (i.e.: calling into a binary like in the old CGI days, initializing the Python VM every time).
- valarauca1 12y ago
  Like I said something fundamentally wrong. With their approach. Un-Indexed databases, databases far away from front end servers (in terms of network topography), or weird VM things with python. Something is bad, and if changing languages solved their problems, they are just sweeping an issue under the rug. It'll hit them in the face later, and harder. Be it developer knowledge, or architectural choices. It'll surface again, they'll (hopefully) be large, and the problem will sting harder too.
tomwilde 12y ago
Unless you're a mathematician or theoretical physicist the gross of your CPU time will be spent waiting for IO. Reading from disk, writing to the network, synchronizing, etc. They're probably just aggregating 2 or 3 APIs, maybe hitting a database and then adding it all together. That description can apply to almost any and all web applications and is inherently IO-bound.
- copx 12y ago
  >Unless you're a mathematician or theoretical physicist [...] ..or Dwarf Fortress player ;)
  - valarauca1 12y ago
    Oooh thank you for reminding me. I recently upgraded to a i7-4790k I've been meaning to jump back into DF now that I should have much better single threaded performance. I honestly haven't played since 2008 on my parents home pentium3.
- falcolas 12y ago
  > That description can apply to almost any and all web applications and is inherently IO-bound. If their system was truly just IO bound, then moving to D wouldn't help them.
  - waps 12y ago
    That's not true. Well it's true in a very narrow technical sense, but it's not really true. For example, the amount of housekeeping python does in order to execute a function call is staggering. It leads to all sorts of nice functionality, but nevertheless (plus C++/D does it almost entirely without housekeeping. Either no housekeeping, or 1 level of indirection). Python has so many indirections for a function call it hardly even makes sense to talk about it in numbers of indirects. Assembly hello world on my machine : 86,607 cpu cycles (of which < 20 actually in the program) Syscalls used by the assembly version : 2 (write and exit) Python hello world on my machine (.pyc was available) : 59,099,731 instructions (including half a million branch misses) Syscalls used by python to execute 'print "hello, world"' : 1139 (each of which causes a program reschedule) These programs do the same thing. Programmers often forget that things they take for granted are not in fact free, they may not even be O(1). Memory allocation. Subprocess execution. Function calls in scripting languages. Syscalls. Writing to files. Allocation of bytes on disks. All of these things come at a really, really high cost, and most not even O(1) costs (e.g. memory allocation is O(N^2) on a busy server as long as things actually fit in main memory, and O(N^4) or even worse when using virtual memory). Sadly using memory does not even have bounded complexity. At some point, just attempting to use virtual memory might cause virtual memory to be allocated just for the lookup. This is generally referred to as "thrashing" and you're very likely to have rebooted your machine before this completes because it'll be frozen for minutes, sometimes hours, if this happens. Likewise the memory model is useful, but huge. Strings in C++ take one byte + the actual contents of the string. Strings in python take up 60 bytes + twice the length of the string. And that's assuming you just set a variable to the string. If you construct the string, the difference is going to be much bigger. The point here is that things that are io-bound (esp. memory bound) in python may be cpu-bound in C++ or D, simply because you avoid doing all the indirections that higher level languages do.
deleted 12y ago
[deleted]

loxs 12y ago

Yep, I could readily point to my Erlang application that can serve thousands of requests per second, doing real work. While simultaneously handling 10k+ websocket connections. All that on a single machine (16G RAM, 8 cores). Does that mean that Erlang is the best? Of course not! It only means that Erlang was specifically designed for this sole use case, which only happens to encompass the whole "web" thingie. It would be an overkill to use Erlang for single-threaded software that requires number crunching speed or for any kind of system scripting. Comparing Erlang to D or Python is stupid. As is (IMHO) using any of the latter for massively parallel servers.