5 ms·
You are right, my wording was incorrect. I'll try to put what I wanted to say in the words of Dr. Thomas Puzak, IBM: "Everyone knows Amdahl's law, but quickly f
by CookWithMe 14y ago
You are right, my wording was incorrect. I'll try to put what I wanted to say in the words of Dr. Thomas Puzak, IBM: "Everyone knows Amdahl's law, but quickly forgets it".
That said, Amdahl's law does NOT apply to problems, it applies to programs: https://en.wikipedia.org/wiki/Amdahl%27s_law https://en.wikipedia.org/wiki/Amdahl%27s_law
What I was trying to say is that, from my experience, if you take an existing program and try to parallelize it with minimal effort (e.g. parallelize all loops), then you may end up with having a serious percentage of serial code. (Or you don't, because your data is large enough, see Gustafson's law: https://en.wikipedia.org/wiki/Gustafson%27s_Law https://en.wikipedia.org/wiki/Gustafson%27s_Law )
However, if you put in more effort and re-architect the program, you'll usually worry about a lot of problems, but Amdahl's law is not one of them. Yes, you didn't "avoid" Amdahl's law (and you can find counter-examples where Amdahl's law is a real concern).
I actually heard a talk about ordering in in-memory databases. It basically comes down to Gustafson's law: If your dataset is rather small, the overhead of parallelization (some of which can be attributed to Amdahl's law) isn't worth it, you just do it on one core. But because your dataset is small, it is so fast that nobody cares. If your dataset is large, each core sorts a subset. Then you start aggregating, and the final aggregation is serial. However, because you already have sorted lists, you can use a different algorithm that merges this very quickly (but would perform bad on unsorted data). I can't remember any specific numbers, but the bottom line was that even with a very beefy server that had 4 sockets and plenty of cores, the serial part wasn't an issue in terms of overall performance. Keep in mind that the author talked about a Multi-core myth and not a Many-core myth.
Again, I don't argue that near-linear speedup is possible for most problems. Most often it's not. But naming Amdahl's law as the only reason is wrong.