6 ms·

Trouble? Node.js has linear speedup over multiple cores for web servers. See http://nodejs.org/docs/v0.8.4/api/cluster.html http://nodejs.org/docs/v0.8.4/api/cl

by ryah 14y ago

Trouble? Node.js has linear speedup over multiple cores for web servers. See http://nodejs.org/docs/v0.8.4/api/cluster.html http://nodejs.org/docs/v0.8.4/api/cluster.html for more info.

aphyr 14y ago
It's parallel in the same sense that any POSIX program is: Node pays a higher cost than real parallel VMs in serialization across IPC boundaries, not being able to take advantage of atomic CPU operations on shared data structures, etc. At least it did last time I looked. Maybe they're doing some shm-style magic/semaphore stuff now. Still going to pay the context switch cost.
- ryah 14y ago
  it's all serialization - but that's not a bottleneck for most web servers. i'd love to hear your context-switching free multicore solution. this is the sanest and most pragmatic way server a web server from multiple threads
  - aphyr 14y ago
    Threads and processes both require a context switch, but on posix systems the thread switch is considerably less expensive. Why? Mainly because the process switch involves changing the VM address space, which means a TLB shootdown: all that hard-earned cache has to be fetched from DRAM again. You also pay a higher cost in synchronization: every message shared between processes requires crossing the kernel boundary. So not only do you have a higher memory use for shared structures and higher CPU costs for serialization, but more cache churn and context switching. it's all serialization - but that's not a bottleneck for most web servers. I disagree, especially for a format like JSON. In fact, every web app server I've dug into spends a significant amount of time on parsing and unparsing responses. You certainly aren't going to be doing computationally expensive tasks in Node, so messaging performance is paramount. i'd love to hear your context-switching free multicore solution. I claimed no such thing: only that multiprocess IPC is more expensive. Modulo syscalls, I think your best bet is gonna be n-1 threads with processor affinities taking advantage of cas/memory fence capabilities on modern hardware. this is the sanest and most pragmatic way server a web server from multiple threads What is this I can't even.
    - aphyr 14y ago
      Note--think I'm wrong about these types of process switches requiring a TLB shootdown. It think it's just cache invalidation.
  - aphyr 14y ago
    Don't believe me? Try it: Node.js: https://gist.github.com/3200829 https://gist.github.com/3200829 Clojure: https://gist.github.com/3200862 https://gist.github.com/3200862 Note that I picked the really small messages here--integers, to give node the best possible serialization advantage. $ time node cluster.js Finished with 10000000 real 3m30.652s user 3m17.180s sys 1m16.113s Note the high sys time: that's IPC. Node also uses only 75% of each core. Why? $ pidstat -w | grep node 11:47:47 AM 25258 48.22 2.11 node 11:47:47 AM 25260 48.34 1.99 node 96 context switches per second. Compare that to a multithreaded Clojure program which uses a LinkedTransferQueue--which eats 97% of each core easily. Note that the times here include ~3 seconds of compilation and jvm startup. $ time lein2 run queue 10000000 "Elapsed time: 55696.274802 msecs" real 0m58.540s user 1m16.733s sys 0m6.436s Why is this version over 3 times faster? Partly because it requires only 4 context switches per second. $ pidstat -tw -p 26537 Linux 3.2.0-3-amd64 (azimuth) 07/29/2012 _x86_64_ (2 CPU) 11:52:03 AM TGID TID cswch/s nvcswch/s Command 11:52:03 AM 26537 - 0.00 0.00 java 11:52:03 AM - 26540 0.01 0.00 |__java 11:52:03 AM - 26541 0.01 0.00 |__java 11:52:03 AM - 26544 0.01 0.00 |__java 11:52:03 AM - 26549 0.01 0.00 |__java 11:52:03 AM - 26551 0.01 0.00 |__java 11:52:03 AM - 26552 2.16 4.26 |__java 11:52:03 AM - 26553 2.10 4.33 |__java And queues are WAY slower than compare-and-set, which involves basically no context switching: $ time lein2 run atom 10000000 "Elapsed time: 969.599545 msecs" real 0m3.925s user 0m5.944s sys 0m0.252s $ pidstat -tw -p 26717 Linux 3.2.0-3-amd64 (azimuth) 07/29/2012 _x86_64_ (2 CPU) 11:54:49 AM TGID TID cswch/s nvcswch/s Command 11:54:49 AM 26717 - 0.00 0.00 java 11:54:49 AM - 26720 0.00 0.01 |__java 11:54:49 AM - 26728 0.01 0.00 |__java 11:54:49 AM - 26731 0.00 0.02 |__java 11:54:49 AM - 26732 0.00 0.01 |__java TL;DR: node.js IPC is not a replacement for a real parallel VM. It allows you to solve a particular class of parallel problems (namely, those which require relatively infrequent communication) on multiple cores, but shared state is basically impossible and message passing is slow. It's a suitable tool for problems which are largely independent and where you can defer the problem of shared state to some other component, e.g. a database. Node is great for stateless web heads, but is in no way a high-performance parallel environment.
  - rektide 14y ago
    I prefer someone keep rolling for sanity loss & resume the work on isolates! Web serving is OK & all, but I'd love if node could be an ideal runtime for petri-nets and webworker meshes too.
  - bascule 14y ago
    It's more than that. Several processes with small, independently garbage collected heaps are not as efficient as a single process with a large heap, parallel threads, and a modern concurrent GC (e.g. the JVM's ConcurrentMarkSweep GC) In addition to that, processes severely inhibit the usefulness of in-process caches. Where threads would allow a single VM to have a large in-process cache, processes generally prevent such collaboration and mean you can only have multiple, duplicated, smaller in-process caches. (Yes, you could use SysV shared memory, but that's also fraught with issues) The same goes for any type of service you would like to run inside a particular web server that could otherwise be shared among multiple threads.
KirinDave 14y ago
You know, ryah; I like your work ethic, I like your enthusiasm, I think you're a cool guy and it's great your project has so much traction. But you say things like this and it worries me. Because a lot of people look up to you and either you said this because you feel defensive about your project or you said it because you genuinely don't understand the cases we're talking about here. And this is a problem because a lot of people look up to you and what you say, so when you say something as baffling as this response, you run the risk of leading a lot of people astray. I was sort of at a loss for how to reply in the time I have to spare for Hacker News, but thankfully Aphyr did for me. But let me clarify what I said a bit, since I was a bit terse: The problem Node.js has is a social one. A lot of node hackers take the stance, "I thought Node.js solved the problems threads presented," (https://groups.google.com/d/msg/nodejs/eVBOYiI_O_A/kv6iiDyy9ZwJ https://groups.google.com/d/msg/nodejs/eVBOYiI_O_A/kv6iiDyy9...) like there is a single axis of superiority and Node.js sits above the methods that came before. But the reality is that Node.js is really just another possible implementation in the Ruby/Python/Pike/-Perl-¹ space, and shares most of the same characteristics as those languages. So you have a lot of people who are aces at front-end programming in the browser thinking they have a uniformly superior tool for tackling high-performance server problems, but really they don't have that; they just have a tool with familiar syntax. And so they fearlessly (and perhaps admirably) charge into the breech of platform programming without realizing that the way people scale big projects involves a lot of tools, a lot of thought about failure modes, and a lot of well-established algorithms with very specific tradeoffs. And so this is Node.js's problem. It's just another gun in the gunfight, but its community thinks its a cannon. In a world where high-performance parallel VMs like Java or Erlang have very powerful and helpful languages like Clojure or Scala on top, we're in a funny situation. It becomes increasingly difficult to justify all these GIL-ridden implementations of languages. Which is not to say these implementations don't have their place (and Node.js is hardly the first javascript implementation outside of a browser), but increasingly they are losing their place in the pieces of your code expected to shuffle data around efficiently in the backend of modern distributed applications. ¹ Correction, perl 6 doesn't plan share this behavior. What I read suggests it's not done yet.
- rektide 14y ago
  I admire Vert.x for trying to bring deployability to the separate-non-shared-event-loop world (aka: where Node is): Vert.x has "verticles," which are instantiated multiple times but share no data. It's very similar to node's cluster execution, except Vert.x is a thorough answer to the deployment problem (going as so far as to bake in Hazelcast if you want to scale out from one multi-threaded process to multiple likely-cross-machine processes). Yet Node itself can not and should not solve the deployment problem: node is a javascript runtime, and contrary to earlier claims I'd declare not opinionated, not one to make this decision for us. The scaling out story is indeed not easy: even tasks like validating session credentials need to be mastered by the team (persist creds into a data-store, or use group communication/pubsub: building for Node is a lot like building for multi-machine Java). The level of DIY-it-ness here are indeed colossal. What I'd contrast against your view- and I agree with most of your premise, that node is extremely seductive and dangerous and many are apt to get in way way way over their head- is that the comforts you describe are what kill these other languages, what strange and prevent us from becoming better more understanding programmers. Ruby, python, php, less so perl, the webdev that goes on there happens by and large at extreme levels of abstraction: developers flock to the known explored center, the tools with the most, the places that seem safest. The dangerous dangerous scenario presented by most good web development tools is that it is the tools that know how to run things. Contrary to the charge into the breech throw up ad-hoc platforms in production every day mentality (of node), these (ruby, php, python) platforms stagnate, they fall to the ruin as their tooling strives towards ever reaching greater heights: the tools accrue more and more responsibility, there are better carved out & expected ways to do things, and incidental complexity, the scope of what must be known, how far one has to travel, to get from writing a page to it getting shipped over the wire or executing, balloons. If anything, Node's core lesson to the world has been about how much is not required. Connect, the only & extremely extremely low-lifed common denominator of Node web world, is the meager-est, tiniest smallest iota of a pluggable middleware system (if only Senchalabs had been courteous enough to be more up front about it being a complete and total rip off Commons-Chain & to not add a thing, I would not bloody loath it). That pattern? bool execute(Context context). Did you handle this request? No? Ok, next. You need to deploy a bunch of processes on a bunch of boxes? You an probably write up something perfectly adequate in a week. Don't have a week? Go find a module: certainly Substack has at least one for whatever your cause (here it's Fleet, https://github.com/substack/fleet https://github.com/substack/fleet). Node modules are wonderful. They all have some varyingly long list of dependencies, usually the tree is 4-8 different things, but the total amount of code being executed from any given module is almost always short of a couple dozen KB: your engineering team can come in and understand anything in a day or three, and gut it and rebuild it in another day or two. Modules, unlike how development processes have shaped up in hte past decade, are wonderfully delightfully stand-alone: there are no frameworks, no crazy deployment systems, no bloody tooling one is writing to: it's just a couple of functions one can use. The surface area, what is shown, is tiny, is isolated, is understandable, there's no great deep mesh. This runs so contrary to the Drupal, to the Rails, to the Cakes or Faces of the world where one is not writing a language, they're at the eight degree of abstraction writing tools for a library that implements enhancements for a framework that is a piece of an ioc container that runs on a application server that runs on a web server that runs in a runtime that actually does something with the OS. We need to get more developers willing to charge into the breech and break out a gun fight. This stuff is not that complicated,* and the tools we have are hiding that fact from us more often than not. So, I admire and love approaches like Vert.x, that take the reactor pattern (what powers Node) and blow it up to the n-th degree, that solve deployment challenges, but at the same time I don't think there is a huge amount of magic there: most node developers have not advanced their runtimes to the level of parity that is called for yet, but this I do not see as a colossal problem. Node, shockingly, even when hideously under tooled, under supported, under op'ed, seems to stand up and not fall over in, in a vast amount of cases. Problems of the rich, good problems to have, when your node system is having worrisome performance problems: most projects will not scale this big, Node will just work, and hopefully you have enough actual genuine talent with enough big picture understanding wtc on your side to not be totally frozen out when your traffic goes up 10x in two days and no one anticipated it. Node is not a land for hand holding, and I don't think that's a bad thing: I think it'll help us listen better to our machines, to not follow our toolings lead into the breech, but to consider what it is we really actually are building for ourselves.

by ryah 14y ago

Trouble? Node.js has linear speedup over multiple cores for web servers. See http://nodejs.org/docs/v0.8.4/api/cluster.html http://nodejs.org/docs/v0.8.4/api/cluster.html for more info.

aphyr 14y ago

It's parallel in the same sense that any POSIX program is: Node pays a higher cost than real parallel VMs in serialization across IPC boundaries, not being able to take advantage of atomic CPU operations on shared data structures, etc. At least it did last time I looked. Maybe they're doing some shm-style magic/semaphore stuff now. Still going to pay the context switch cost.

ryah 14y ago
it's all serialization - but that's not a bottleneck for most web servers. i'd love to hear your context-switching free multicore solution. this is the sanest and most pragmatic way server a web server from multiple threads
- aphyr 14y ago
  Threads and processes both require a context switch, but on posix systems the thread switch is considerably less expensive. Why? Mainly because the process switch involves changing the VM address space, which means a TLB shootdown: all that hard-earned cache has to be fetched from DRAM again. You also pay a higher cost in synchronization: every message shared between processes requires crossing the kernel boundary. So not only do you have a higher memory use for shared structures and higher CPU costs for serialization, but more cache churn and context switching. it's all serialization - but that's not a bottleneck for most web servers. I disagree, especially for a format like JSON. In fact, every web app server I've dug into spends a significant amount of time on parsing and unparsing responses. You certainly aren't going to be doing computationally expensive tasks in Node, so messaging performance is paramount. i'd love to hear your context-switching free multicore solution. I claimed no such thing: only that multiprocess IPC is more expensive. Modulo syscalls, I think your best bet is gonna be n-1 threads with processor affinities taking advantage of cas/memory fence capabilities on modern hardware. this is the sanest and most pragmatic way server a web server from multiple threads What is this I can't even.
  - aphyr 14y ago
    Note--think I'm wrong about these types of process switches requiring a TLB shootdown. It think it's just cache invalidation.
- aphyr 14y ago
  Don't believe me? Try it: Node.js: https://gist.github.com/3200829 https://gist.github.com/3200829 Clojure: https://gist.github.com/3200862 https://gist.github.com/3200862 Note that I picked the really small messages here--integers, to give node the best possible serialization advantage. $ time node cluster.js Finished with 10000000 real 3m30.652s user 3m17.180s sys 1m16.113s Note the high sys time: that's IPC. Node also uses only 75% of each core. Why? $ pidstat -w | grep node 11:47:47 AM 25258 48.22 2.11 node 11:47:47 AM 25260 48.34 1.99 node 96 context switches per second. Compare that to a multithreaded Clojure program which uses a LinkedTransferQueue--which eats 97% of each core easily. Note that the times here include ~3 seconds of compilation and jvm startup. $ time lein2 run queue 10000000 "Elapsed time: 55696.274802 msecs" real 0m58.540s user 1m16.733s sys 0m6.436s Why is this version over 3 times faster? Partly because it requires only 4 context switches per second. $ pidstat -tw -p 26537 Linux 3.2.0-3-amd64 (azimuth) 07/29/2012 _x86_64_ (2 CPU) 11:52:03 AM TGID TID cswch/s nvcswch/s Command 11:52:03 AM 26537 - 0.00 0.00 java 11:52:03 AM - 26540 0.01 0.00 |__java 11:52:03 AM - 26541 0.01 0.00 |__java 11:52:03 AM - 26544 0.01 0.00 |__java 11:52:03 AM - 26549 0.01 0.00 |__java 11:52:03 AM - 26551 0.01 0.00 |__java 11:52:03 AM - 26552 2.16 4.26 |__java 11:52:03 AM - 26553 2.10 4.33 |__java And queues are WAY slower than compare-and-set, which involves basically no context switching: $ time lein2 run atom 10000000 "Elapsed time: 969.599545 msecs" real 0m3.925s user 0m5.944s sys 0m0.252s $ pidstat -tw -p 26717 Linux 3.2.0-3-amd64 (azimuth) 07/29/2012 _x86_64_ (2 CPU) 11:54:49 AM TGID TID cswch/s nvcswch/s Command 11:54:49 AM 26717 - 0.00 0.00 java 11:54:49 AM - 26720 0.00 0.01 |__java 11:54:49 AM - 26728 0.01 0.00 |__java 11:54:49 AM - 26731 0.00 0.02 |__java 11:54:49 AM - 26732 0.00 0.01 |__java TL;DR: node.js IPC is not a replacement for a real parallel VM. It allows you to solve a particular class of parallel problems (namely, those which require relatively infrequent communication) on multiple cores, but shared state is basically impossible and message passing is slow. It's a suitable tool for problems which are largely independent and where you can defer the problem of shared state to some other component, e.g. a database. Node is great for stateless web heads, but is in no way a high-performance parallel environment.
- rektide 14y ago
  I prefer someone keep rolling for sanity loss & resume the work on isolates! Web serving is OK & all, but I'd love if node could be an ideal runtime for petri-nets and webworker meshes too.
- bascule 14y ago
  It's more than that. Several processes with small, independently garbage collected heaps are not as efficient as a single process with a large heap, parallel threads, and a modern concurrent GC (e.g. the JVM's ConcurrentMarkSweep GC) In addition to that, processes severely inhibit the usefulness of in-process caches. Where threads would allow a single VM to have a large in-process cache, processes generally prevent such collaboration and mean you can only have multiple, duplicated, smaller in-process caches. (Yes, you could use SysV shared memory, but that's also fraught with issues) The same goes for any type of service you would like to run inside a particular web server that could otherwise be shared among multiple threads.

KirinDave 14y ago

You know, ryah; I like your work ethic, I like your enthusiasm, I think you're a cool guy and it's great your project has so much traction. But you say things like this and it worries me. Because a lot of people look up to you and either you said this because you feel defensive about your project or you said it because you genuinely don't understand the cases we're talking about here. And this is a problem because a lot of people look up to you and what you say, so when you say something as baffling as this response, you run the risk of leading a lot of people astray. I was sort of at a loss for how to reply in the time I have to spare for Hacker News, but thankfully Aphyr did for me. But let me clarify what I said a bit, since I was a bit terse: The problem Node.js has is a social one. A lot of node hackers take the stance, "I thought Node.js solved the problems threads presented," (https://groups.google.com/d/msg/nodejs/eVBOYiI_O_A/kv6iiDyy9ZwJ https://groups.google.com/d/msg/nodejs/eVBOYiI_O_A/kv6iiDyy9...) like there is a single axis of superiority and Node.js sits above the methods that came before. But the reality is that Node.js is really just another possible implementation in the Ruby/Python/Pike/-Perl-¹ space, and shares most of the same characteristics as those languages. So you have a lot of people who are aces at front-end programming in the browser thinking they have a uniformly superior tool for tackling high-performance server problems, but really they don't have that; they just have a tool with familiar syntax. And so they fearlessly (and perhaps admirably) charge into the breech of platform programming without realizing that the way people scale big projects involves a lot of tools, a lot of thought about failure modes, and a lot of well-established algorithms with very specific tradeoffs. And so this is Node.js's problem. It's just another gun in the gunfight, but its community thinks its a cannon. In a world where high-performance parallel VMs like Java or Erlang have very powerful and helpful languages like Clojure or Scala on top, we're in a funny situation. It becomes increasingly difficult to justify all these GIL-ridden implementations of languages. Which is not to say these implementations don't have their place (and Node.js is hardly the first javascript implementation outside of a browser), but increasingly they are losing their place in the pieces of your code expected to shuffle data around efficiently in the backend of modern distributed applications. ¹ Correction, perl 6 doesn't plan share this behavior. What I read suggests it's not done yet.

rektide 14y ago
I admire Vert.x for trying to bring deployability to the separate-non-shared-event-loop world (aka: where Node is): Vert.x has "verticles," which are instantiated multiple times but share no data. It's very similar to node's cluster execution, except Vert.x is a thorough answer to the deployment problem (going as so far as to bake in Hazelcast if you want to scale out from one multi-threaded process to multiple likely-cross-machine processes). Yet Node itself can not and should not solve the deployment problem: node is a javascript runtime, and contrary to earlier claims I'd declare not opinionated, not one to make this decision for us. The scaling out story is indeed not easy: even tasks like validating session credentials need to be mastered by the team (persist creds into a data-store, or use group communication/pubsub: building for Node is a lot like building for multi-machine Java). The level of DIY-it-ness here are indeed colossal. What I'd contrast against your view- and I agree with most of your premise, that node is extremely seductive and dangerous and many are apt to get in way way way over their head- is that the comforts you describe are what kill these other languages, what strange and prevent us from becoming better more understanding programmers. Ruby, python, php, less so perl, the webdev that goes on there happens by and large at extreme levels of abstraction: developers flock to the known explored center, the tools with the most, the places that seem safest. The dangerous dangerous scenario presented by most good web development tools is that it is the tools that know how to run things. Contrary to the charge into the breech throw up ad-hoc platforms in production every day mentality (of node), these (ruby, php, python) platforms stagnate, they fall to the ruin as their tooling strives towards ever reaching greater heights: the tools accrue more and more responsibility, there are better carved out & expected ways to do things, and incidental complexity, the scope of what must be known, how far one has to travel, to get from writing a page to it getting shipped over the wire or executing, balloons. If anything, Node's core lesson to the world has been about how much is not required. Connect, the only & extremely extremely low-lifed common denominator of Node web world, is the meager-est, tiniest smallest iota of a pluggable middleware system (if only Senchalabs had been courteous enough to be more up front about it being a complete and total rip off Commons-Chain & to not add a thing, I would not bloody loath it). That pattern? bool execute(Context context). Did you handle this request? No? Ok, next. You need to deploy a bunch of processes on a bunch of boxes? You an probably write up something perfectly adequate in a week. Don't have a week? Go find a module: certainly Substack has at least one for whatever your cause (here it's Fleet, https://github.com/substack/fleet https://github.com/substack/fleet). Node modules are wonderful. They all have some varyingly long list of dependencies, usually the tree is 4-8 different things, but the total amount of code being executed from any given module is almost always short of a couple dozen KB: your engineering team can come in and understand anything in a day or three, and gut it and rebuild it in another day or two. Modules, unlike how development processes have shaped up in hte past decade, are wonderfully delightfully stand-alone: there are no frameworks, no crazy deployment systems, no bloody tooling one is writing to: it's just a couple of functions one can use. The surface area, what is shown, is tiny, is isolated, is understandable, there's no great deep mesh. This runs so contrary to the Drupal, to the Rails, to the Cakes or Faces of the world where one is not writing a language, they're at the eight degree of abstraction writing tools for a library that implements enhancements for a framework that is a piece of an ioc container that runs on a application server that runs on a web server that runs in a runtime that actually does something with the OS. We need to get more developers willing to charge into the breech and break out a gun fight. This stuff is not that complicated,* and the tools we have are hiding that fact from us more often than not. So, I admire and love approaches like Vert.x, that take the reactor pattern (what powers Node) and blow it up to the n-th degree, that solve deployment challenges, but at the same time I don't think there is a huge amount of magic there: most node developers have not advanced their runtimes to the level of parity that is called for yet, but this I do not see as a colossal problem. Node, shockingly, even when hideously under tooled, under supported, under op'ed, seems to stand up and not fall over in, in a vast amount of cases. Problems of the rich, good problems to have, when your node system is having worrisome performance problems: most projects will not scale this big, Node will just work, and hopefully you have enough actual genuine talent with enough big picture understanding wtc on your side to not be totally frozen out when your traffic goes up 10x in two days and no one anticipated it. Node is not a land for hand holding, and I don't think that's a bad thing: I think it'll help us listen better to our machines, to not follow our toolings lead into the breech, but to consider what it is we really actually are building for ourselves.