6 ms·

Technology Brief for Apple's Grand Central Multicore Paradigm [pdf]

42 points by nickb 17y ago

swannodette 17y ago
While all the Apple hardware announcements look great, I think this is possibly an even more significant announcement. C/Objective-C/C++ gets lambdas and closures! It looks like Apple has a very interesting single machine concurrency story here. Right now I'm really enjoying Clojure- it's lockless approach to single machine concurrency is plenty, plenty fun. Unfortunately the downside of learning Clojure is that many other popular languages look downright archaic to me in this regard now. Now reading over this document, whodathunk, concurrency programming might be as fun in C as it is in Clojure!
- Gary_W_Longsine 17y ago
  Related to this is another key performance technology in Snow Leopard, OpenCL: http://images.apple.com/macosx/technology/docs/OpenCL_TB_brief_20090608.pdf http://images.apple.com/macosx/technology/docs/OpenCL_TB_bri... .
jmtulloss 17y ago
First of all, this is awesome. I am missing some details though. Are these true closures? What are the scoping rules? Can you read data declared outside of your block? Can you alter it? Is the only way to keep alterations safe to use a private queue? I guess time (or a closer reading) will tell.
- swannodette 17y ago
  More information about the implementation of blocks here: http://lists.cs.uiuc.edu/pipermail/cfe-dev/2008-August/002670.html http://lists.cs.uiuc.edu/pipermail/cfe-dev/2008-August/00267...
gchpaco 17y ago
This encourages a very different style of multicore programming than is common on desktops today, closer to that of grid systems. Very interesting to see how this develops.
- Gary_W_Longsine 17y ago
  Apple may have been inspired in part by their experience with XGrid [http://www.apple.com/server/macosx/technology/xgrid.html http://www.apple.com/server/macosx/technology/xgrid.html], based on the NeXTSTEP application Zilla [http://www.omnigroup.com/mailman/archive/macosx-talk/2000-September/028352.html http://www.omnigroup.com/mailman/archive/macosx-talk/2000-Se...], and perhaps by PDO (Portable Distributed Objects) [http://en.wikipedia.org/wiki/NeXTSTEP http://en.wikipedia.org/wiki/NeXTSTEP]. I mention these somewhat obscure influences in response to your grid observation. Obviously there are several other (probably more immediate) influences on GCD and OpenCL as well (LLVM, OpenGL, various academic research on parallelized programming, etc.)
agocke 17y ago
I wonder how this will affect the legacy methods of concurrency in OS X -- specifically POSIX threads. I know I wrote a number of C apps using POSIX threads that worked well in OS X and in Linux. While this is nice and interesting for Mac only programs, I hope that I don't have to rewrite my cross platform apps to take advantage of the API.
- wmf 17y ago
  Obviously pthread support won't be removed; pthread programs just won't play as nice with each other as Grand Central programs.
  - agocke 17y ago
    Well I didn't really mean to suggest that it would be removed, mainly about what kind of performance hit I would take by using pthreads instead of Grand Central. In addition, how well will pthread programs and GC programs work together. Normally, the threading is handled by the scheduler in a straightforward model, but if the GC model modifies the scheduler model, I'm not really sure what the results will be.
    - Gary_W_Longsine 17y ago
      You won't "take a hit", but you will "fail to utilize the hardware as efficiently as you could". Both GCD and OpenCL have their roots in open extensions to C. Both offer potentially significant performance gains for certain types of tasks. If enough developers find it advantageous to use either blocks or OpenCL, these technologies will likely show up on other platforms. Regarding the scheduler, I'm not sure enough details are known outside of Apple at this point to know what the results would be on Snow Leopard. Your milage would obviously vary when porting to some other OS. It's easy to imagine that running a PThreads app might hog some cores, or be punished inappropriately by the scheduler. These things can happen when running more than one threaded app on a system today, without GCD. This behavior, even if not documented, should be relatively easy to explore with some simple demo apps, though, without porting your entire app. And, uhm... you will need to refactor your application to take advantage of the new API. (Was that a trick question?) Apple's guidance indicates that you can make modest changes for big gains by focusing the refactoring effort on the parts of the app that do the most computationally. For some apps this can be a small amount of work for a large gain, particularly if you can take advantage of OpenCL as well as GCD.
miratom 17y ago
A Job Queue is a great thing to have, especially when it provided for you, but it's hardly "revolutionary."
perneto 17y ago
This sounds inspired by SEDA (Staged Event-Driven Architecture). http://www.eecs.harvard.edu/~mdw/proj/seda/ http://www.eecs.harvard.edu/~mdw/proj/seda/ It was about time someone made good use of it, too! Is there more info available? The white paper is nice, but not too packed with detail.

42 points by nickb 17y ago

While all the Apple hardware announcements look great, I think this is possibly an even more significant announcement. C/Objective-C/C++ gets lambdas and closures! It looks like Apple has a very interesting single machine concurrency story here. Right now I'm really enjoying Clojure- it's lockless approach to single machine concurrency is plenty, plenty fun. Unfortunately the downside of learning Clojure is that many other popular languages look downright archaic to me in this regard now. Now reading over this document, whodathunk, concurrency programming might be as fun in C as it is in Clojure!

Gary_W_Longsine 17y ago
Related to this is another key performance technology in Snow Leopard, OpenCL: http://images.apple.com/macosx/technology/docs/OpenCL_TB_brief_20090608.pdf http://images.apple.com/macosx/technology/docs/OpenCL_TB_bri... .

jmtulloss 17y ago

First of all, this is awesome. I am missing some details though. Are these true closures? What are the scoping rules? Can you read data declared outside of your block? Can you alter it? Is the only way to keep alterations safe to use a private queue? I guess time (or a closer reading) will tell.

swannodette 17y ago
More information about the implementation of blocks here: http://lists.cs.uiuc.edu/pipermail/cfe-dev/2008-August/002670.html http://lists.cs.uiuc.edu/pipermail/cfe-dev/2008-August/00267...

gchpaco 17y ago

This encourages a very different style of multicore programming than is common on desktops today, closer to that of grid systems. Very interesting to see how this develops.

Gary_W_Longsine 17y ago
Apple may have been inspired in part by their experience with XGrid [http://www.apple.com/server/macosx/technology/xgrid.html http://www.apple.com/server/macosx/technology/xgrid.html], based on the NeXTSTEP application Zilla [http://www.omnigroup.com/mailman/archive/macosx-talk/2000-September/028352.html http://www.omnigroup.com/mailman/archive/macosx-talk/2000-Se...], and perhaps by PDO (Portable Distributed Objects) [http://en.wikipedia.org/wiki/NeXTSTEP http://en.wikipedia.org/wiki/NeXTSTEP]. I mention these somewhat obscure influences in response to your grid observation. Obviously there are several other (probably more immediate) influences on GCD and OpenCL as well (LLVM, OpenGL, various academic research on parallelized programming, etc.)

agocke 17y ago

I wonder how this will affect the legacy methods of concurrency in OS X -- specifically POSIX threads. I know I wrote a number of C apps using POSIX threads that worked well in OS X and in Linux. While this is nice and interesting for Mac only programs, I hope that I don't have to rewrite my cross platform apps to take advantage of the API.

wmf 17y ago
Obviously pthread support won't be removed; pthread programs just won't play as nice with each other as Grand Central programs.
- agocke 17y ago
  Well I didn't really mean to suggest that it would be removed, mainly about what kind of performance hit I would take by using pthreads instead of Grand Central. In addition, how well will pthread programs and GC programs work together. Normally, the threading is handled by the scheduler in a straightforward model, but if the GC model modifies the scheduler model, I'm not really sure what the results will be.
  - Gary_W_Longsine 17y ago
    You won't "take a hit", but you will "fail to utilize the hardware as efficiently as you could". Both GCD and OpenCL have their roots in open extensions to C. Both offer potentially significant performance gains for certain types of tasks. If enough developers find it advantageous to use either blocks or OpenCL, these technologies will likely show up on other platforms. Regarding the scheduler, I'm not sure enough details are known outside of Apple at this point to know what the results would be on Snow Leopard. Your milage would obviously vary when porting to some other OS. It's easy to imagine that running a PThreads app might hog some cores, or be punished inappropriately by the scheduler. These things can happen when running more than one threaded app on a system today, without GCD. This behavior, even if not documented, should be relatively easy to explore with some simple demo apps, though, without porting your entire app. And, uhm... you will need to refactor your application to take advantage of the new API. (Was that a trick question?) Apple's guidance indicates that you can make modest changes for big gains by focusing the refactoring effort on the parts of the app that do the most computationally. For some apps this can be a small amount of work for a large gain, particularly if you can take advantage of OpenCL as well as GCD.

miratom 17y ago

A Job Queue is a great thing to have, especially when it provided for you, but it's hardly "revolutionary."

perneto 17y ago

This sounds inspired by SEDA (Staged Event-Driven Architecture). http://www.eecs.harvard.edu/~mdw/proj/seda/ http://www.eecs.harvard.edu/~mdw/proj/seda/ It was about time someone made good use of it, too! Is there more info available? The white paper is nice, but not too packed with detail.