10 ms·

RAM Disks and Saving Your SSD from AFL Fuzzing

25 points by michaelrash 12y ago

NKCSS 12y ago
I'm not sure it's really that big of a problem. If you look at http://techreport.com/review/27436/the-ssd-endurance-experiment-two-freaking-petabytes http://techreport.com/review/27436/the-ssd-endurance-experim... you can see that even consumer drives vastly outperform the given rates.
- CSDude 12y ago
  They are not regular consumer ssd drives, they are the most expensive ones (among the consumer ssd drives)
  - kyriakos 12y ago
    The difference in price is not that great
  - vegardx 12y ago
    They are regular consumer drives. They are even markedet and priced as that. They don't even have super capacitors, which more and more consumer drives are coming equipped with standard these days.
JonathonW 12y ago
I'm not sure how one would get statistics on this, but I can't help but wonder how much of those ~2,000 writes/sec are actually getting written to disk. Basically, the same property that gives AFL fuzzing such high write amplification (it's repeatedly changing only a few bytes within a single sector of a file) should also mean that the data in question is mostly staying in cache-- the OS is going to flush that cache out to disk periodically, but it's probably not going to be flushing that cache after every write; that would be pretty slow and silly. My 840 EVO tracks its total number of LBAs written, and makes it available as a SMART attribute. Assuming your drive does the same, it might be interesting to see how quickly that number goes up while fuzzing versus how quickly it goes up with a more "normal" workload.
- nkurz 12y ago
  I can't help but wonder how much of those ~2,000 writes/sec are actually getting written to disk Approximately zero would be my fairly confident guess, although I'm more familiar with the Linux page cache than OSX. For repeated writes to the same small file without an explicit sync, I don't think there should be any difference between a RAM disk and a regular drive. One way to test would be to measure the number of writes/sec to the the RAM disk, the SSD, and a slow spinning drive --- my theory says they should be very close to the same. But I'd love to learn by being shown otherwise.
  - michaelrash 12y ago
    Remember from the article that AFL is being run from a VM. So, my guess is that OS X (host) doesn't know anything about whether what is being written on the VM is a small file or not. There are two possibilities: 1) either the OS X activity monitor is actually measuring disk+cache when it says "disk writes" or 2) disk writes really are happening as fast as shown in the graphs. My bet is the later. What saves things here is the usage of the RAM disk on the guest, since at least that is a clean mechanism for really only using RAM instead of disk - and that carries through from the guest to the host.
    - nkurz 12y ago
      You make good points, and I hadn't considered how the VM would affect things. I'm still doubtful that frequent rewrites of the same file would actually hit the flash of the SSD, but with much less certainty. If it is writing this frequently, this seems like a place for an easy optimization of the VM. Or maybe I'm misunderstanding, and it's writing a series of small files rather than rewriting the same one?
      - michaelrash 12y ago
        On the VM, AFL is rewriting the same small file millions of times, and the fuzzed program is reading this file from disk (as opposed to stdin). I think the main question is why does the OS X activity monitor show a massive spike in disk writes when AFL is started and a corresponding decrease when it is stopped? This is 100% reproducible, and the magnitude of the change is unmistakable. It is absolutely clear that the spike in disk writes is due to AFL activity. Further, this same spike does not happen when running under a RAM disk on the VM. I'm confident you would see the same result. So, either the OS X activity monitor has some special way to measure caching in the VM and that is what it is showing (surely this is unlikely), or the writes really are hitting the SSD when not running under the RAM disk. If not the later, what other (realistic) possibility is there?

25 points by michaelrash 12y ago

NKCSS 12y ago

I'm not sure it's really that big of a problem. If you look at http://techreport.com/review/27436/the-ssd-endurance-experiment-two-freaking-petabytes http://techreport.com/review/27436/the-ssd-endurance-experim... you can see that even consumer drives vastly outperform the given rates.

CSDude 12y ago
They are not regular consumer ssd drives, they are the most expensive ones (among the consumer ssd drives)
- kyriakos 12y ago
  The difference in price is not that great
- vegardx 12y ago
  They are regular consumer drives. They are even markedet and priced as that. They don't even have super capacitors, which more and more consumer drives are coming equipped with standard these days.

JonathonW 12y ago

I'm not sure how one would get statistics on this, but I can't help but wonder how much of those ~2,000 writes/sec are actually getting written to disk. Basically, the same property that gives AFL fuzzing such high write amplification (it's repeatedly changing only a few bytes within a single sector of a file) should also mean that the data in question is mostly staying in cache-- the OS is going to flush that cache out to disk periodically, but it's probably not going to be flushing that cache after every write; that would be pretty slow and silly. My 840 EVO tracks its total number of LBAs written, and makes it available as a SMART attribute. Assuming your drive does the same, it might be interesting to see how quickly that number goes up while fuzzing versus how quickly it goes up with a more "normal" workload.

nkurz 12y ago
I can't help but wonder how much of those ~2,000 writes/sec are actually getting written to disk Approximately zero would be my fairly confident guess, although I'm more familiar with the Linux page cache than OSX. For repeated writes to the same small file without an explicit sync, I don't think there should be any difference between a RAM disk and a regular drive. One way to test would be to measure the number of writes/sec to the the RAM disk, the SSD, and a slow spinning drive --- my theory says they should be very close to the same. But I'd love to learn by being shown otherwise.
- michaelrash 12y ago
  Remember from the article that AFL is being run from a VM. So, my guess is that OS X (host) doesn't know anything about whether what is being written on the VM is a small file or not. There are two possibilities: 1) either the OS X activity monitor is actually measuring disk+cache when it says "disk writes" or 2) disk writes really are happening as fast as shown in the graphs. My bet is the later. What saves things here is the usage of the RAM disk on the guest, since at least that is a clean mechanism for really only using RAM instead of disk - and that carries through from the guest to the host.
  - nkurz 12y ago
    You make good points, and I hadn't considered how the VM would affect things. I'm still doubtful that frequent rewrites of the same file would actually hit the flash of the SSD, but with much less certainty. If it is writing this frequently, this seems like a place for an easy optimization of the VM. Or maybe I'm misunderstanding, and it's writing a series of small files rather than rewriting the same one?
    - michaelrash 12y ago
      On the VM, AFL is rewriting the same small file millions of times, and the fuzzed program is reading this file from disk (as opposed to stdin). I think the main question is why does the OS X activity monitor show a massive spike in disk writes when AFL is started and a corresponding decrease when it is stopped? This is 100% reproducible, and the magnitude of the change is unmistakable. It is absolutely clear that the spike in disk writes is due to AFL activity. Further, this same spike does not happen when running under a RAM disk on the VM. I'm confident you would see the same result. So, either the OS X activity monitor has some special way to measure caching in the VM and that is what it is showing (surely this is unlikely), or the writes really are hitting the SSD when not running under the RAM disk. If not the later, what other (realistic) possibility is there?