11 ms·

The article is very optimistic about memory availability per cycle, reality is way worse. As an example, on my Macbook Air 2011 with ~10 GB/s of maximum ram ba

The article is very optimistic about memory availability per cycle, reality is way worse. As an example, on my Macbook Air 2011 with ~10 GB/s of maximum ram bandwidth, random access to memory can take 100 time more than a sequential one. This in C, with full optimizations and using a very low overhead read loop. Using the same metrics of the author: best case: ~ 3 bytes per cycle (around 6 Gigabyte per second of available bandwidth) worst case: ~ 0.024 bytes per cycle (every scheduler, prefetch, already open column mostly defied) Note that worst case uses 10 seconds (!) to read and sum in a random way all the cells of an array of 100.000.000 of 4 byte integers, exactly once. Main loop is light enough not to influence the test. That's about 40 megabytes per second out of 6.000 available. What can I say.. CPU designers are truly wizards!

rayiner 9y ago
Right, the article focuses on bandwidth while in many cases it's about latency of dependent memory operations (which has improved by even a lower factor than what's described in the article). Your typical pointer chasing code won't come anywhere near the maximum memory bandwidth mentioned in the article (50 gb/sec).
logicallee 9y ago
>worst case: ~ 0.024 bytes per cycle (every scheduler, prefetch, already open column mostly defied) >That's about 40 megabytes per second Wow, that is an explosive conclusion. It's very hard for me to come to terms with. 40 MB per second is the sustained read of a spinning platter hard drive http://hdd.userbenchmark.com/ http://hdd.userbenchmark.com/ (click any line)† (Did I say 40? I meant 160 MB/sec...) So more like 1/4 of the sequential read speed off of a spinning platter of rust. Ten seconds is insane. I don't care how many times you're bouncing back and forth and invalidating caches and pipelines and prefetches and schedulers, you simply shouldn't be able to ruin things that badly. It is off from what I would expect by (easily) an order of magnitude. I know you say that the main loop is very light - but aren't there other aspects to your build system and operating system that might be affecting this test? To say something very obvious, couldn't the Operating System scheduler not be giving your process the appropriate number of cycles? There is a lot more that I could say in this direction but let's just do something simpler: -> Could you try your experiment without an operating system? For example here are some people who booted a raspberry pi without an operating system - https://www.google.com/search?q=chess+without+an+operating+system+raspberry+pi https://www.google.com/search?q=chess+without+an+operating+s... Perhaps before going that far you could simply boot into a Linux image that was simply not compiled with any hardware support to do anything. (After all you really don't need to do anything except return to the shell.) Or simply see what happens if you boot into Linux and try it. If you get an instantly different result simply booting Linux on the same hardware then you instantly have an explosive blog post: "summing 100k 4-byte integers randomly takes 10 seconds on Mac OS X but only 1 second under Linux". I realize there is a HUGE difference (HUGE) between sequential and random. But I just wanted to get across how insanely slow 40 MB/second straight to RAM is. That should not be possible, no matter how much you defy caches and scheduling and so forth, unless you get the Mac to swap pages out of RAM onto an SSD or something! So not using an operating system would really help here. Could you try it? I'm not saying I don't believe you but - wow, that is insane. † I just noticed you wrote "Macbook air 2011". If you want to look at 2011 hard drive speeds, a quick glance still sees some quoting 140 MB/sec so it still seems correct to me, but I just quoted 2017 figures.
- JoachimSchipper 9y ago
  Your parent poster doesn't provide details, but this is not really crazy. From https://gist.github.com/jboner/2841832 https://gist.github.com/jboner/2841832, main memory latency is 100 ns; so we get 10 million fetches per second. You could get 40 MB by e.g. assuming that you have 4 one-byte requests in flight at the same time. As they say, RAM is the new disk.
  - nortiero 9y ago
    Hi, I've posted a few details and a sample program over here. As you correctly notice, latency is a big issue on modern machines, way worse (in proportion) than it was in the late eighties. But as soon as the wheels start turning, they spit out a lot of bytes, for sure!
- nortiero 9y ago
  Hi! Memory is definitely an issue today, especially with big fat server processors... Here is the small program I wrote, first allocates an array, then writes a random placed linked list that touches all the array cells (e.g. start->|4|6|2|5|3|end , so it goes to cell 1, then 4, then 5, then 3,2,6 and done. Then it reads the same array from start to finish, just to sum contents. This is fast and goes to 6-8 GB/s, depending on unrelated memory pressure from video and os tasks. This is the advertised speed. So I don't think it's a swap issue (and my SSD is way faster :-) I've also tried with smaller and bigger arrays. Just ensure it stays in real memory. Moving around in a big array is very hard on the memory controller: - no prefetch possible; - row and column changes at every access (almost), so commands have to be issued to burst terminate, close row, close bank, precharge maybe, open bank, activate row, fetch address, and who knows; - no advantage from interleave; - no advantage from a fat 64 or 128 bit bus; It is the combined time needed for each and every memory read (due to randomization) that cause performance to drop. This is the code I've been using. I've modified it to work on unices (OSX has an issue with timeval struct), but have not tested on them. It compiles, may need to be altered a bit. Launch it with ./a.out <array_size> <random seed>. You will note that, as soon as the array moves out of caches, hell breaks loose. Works with gcc or clang. https://godbolt.org/g/qX39tL https://godbolt.org/g/qX39tL

rayiner 9y ago

Right, the article focuses on bandwidth while in many cases it's about latency of dependent memory operations (which has improved by even a lower factor than what's described in the article). Your typical pointer chasing code won't come anywhere near the maximum memory bandwidth mentioned in the article (50 gb/sec).

logicallee 9y ago

>worst case: ~ 0.024 bytes per cycle (every scheduler, prefetch, already open column mostly defied) >That's about 40 megabytes per second Wow, that is an explosive conclusion. It's very hard for me to come to terms with. 40 MB per second is the sustained read of a spinning platter hard drive http://hdd.userbenchmark.com/ http://hdd.userbenchmark.com/ (click any line)† (Did I say 40? I meant 160 MB/sec...) So more like 1/4 of the sequential read speed off of a spinning platter of rust. Ten seconds is insane. I don't care how many times you're bouncing back and forth and invalidating caches and pipelines and prefetches and schedulers, you simply shouldn't be able to ruin things that badly. It is off from what I would expect by (easily) an order of magnitude. I know you say that the main loop is very light - but aren't there other aspects to your build system and operating system that might be affecting this test? To say something very obvious, couldn't the Operating System scheduler not be giving your process the appropriate number of cycles? There is a lot more that I could say in this direction but let's just do something simpler: -> Could you try your experiment without an operating system? For example here are some people who booted a raspberry pi without an operating system - https://www.google.com/search?q=chess+without+an+operating+system+raspberry+pi https://www.google.com/search?q=chess+without+an+operating+s... Perhaps before going that far you could simply boot into a Linux image that was simply not compiled with any hardware support to do anything. (After all you really don't need to do anything except return to the shell.) Or simply see what happens if you boot into Linux and try it. If you get an instantly different result simply booting Linux on the same hardware then you instantly have an explosive blog post: "summing 100k 4-byte integers randomly takes 10 seconds on Mac OS X but only 1 second under Linux". I realize there is a HUGE difference (HUGE) between sequential and random. But I just wanted to get across how insanely slow 40 MB/second straight to RAM is. That should not be possible, no matter how much you defy caches and scheduling and so forth, unless you get the Mac to swap pages out of RAM onto an SSD or something! So not using an operating system would really help here. Could you try it? I'm not saying I don't believe you but - wow, that is insane. † I just noticed you wrote "Macbook air 2011". If you want to look at 2011 hard drive speeds, a quick glance still sees some quoting 140 MB/sec so it still seems correct to me, but I just quoted 2017 figures.

JoachimSchipper 9y ago
Your parent poster doesn't provide details, but this is not really crazy. From https://gist.github.com/jboner/2841832 https://gist.github.com/jboner/2841832, main memory latency is 100 ns; so we get 10 million fetches per second. You could get 40 MB by e.g. assuming that you have 4 one-byte requests in flight at the same time. As they say, RAM is the new disk.
- nortiero 9y ago
  Hi, I've posted a few details and a sample program over here. As you correctly notice, latency is a big issue on modern machines, way worse (in proportion) than it was in the late eighties. But as soon as the wheels start turning, they spit out a lot of bytes, for sure!
nortiero 9y ago
Hi! Memory is definitely an issue today, especially with big fat server processors... Here is the small program I wrote, first allocates an array, then writes a random placed linked list that touches all the array cells (e.g. start->|4|6|2|5|3|end , so it goes to cell 1, then 4, then 5, then 3,2,6 and done. Then it reads the same array from start to finish, just to sum contents. This is fast and goes to 6-8 GB/s, depending on unrelated memory pressure from video and os tasks. This is the advertised speed. So I don't think it's a swap issue (and my SSD is way faster :-) I've also tried with smaller and bigger arrays. Just ensure it stays in real memory. Moving around in a big array is very hard on the memory controller: - no prefetch possible; - row and column changes at every access (almost), so commands have to be issued to burst terminate, close row, close bank, precharge maybe, open bank, activate row, fetch address, and who knows; - no advantage from interleave; - no advantage from a fat 64 or 128 bit bus; It is the combined time needed for each and every memory read (due to randomization) that cause performance to drop. This is the code I've been using. I've modified it to work on unices (OSX has an issue with timeval struct), but have not tested on them. It compiles, may need to be altered a bit. Launch it with ./a.out <array_size> <random seed>. You will note that, as soon as the array moves out of caches, hell breaks loose. Works with gcc or clang. https://godbolt.org/g/qX39tL https://godbolt.org/g/qX39tL