7 ms·

What Every Data Programmer Needs to Know About Disks

46 points by frsyuki 15y ago

vimalg2 15y ago
Converted the Presentation PPTX to PDF for accessibility: http://dl.dropbox.com/u/5137/pdf-link/What%20Every%20Data%20Programmer%20Needs%20to%20Know%20About%20Disks%20Presentation.pdf http://dl.dropbox.com/u/5137/pdf-link/What%20Every%20Data%20...
koopajah 15y ago
Is there a video recorded of the presentation somewhere? This seemed really interesting
- delta1 15y ago
  I was really hoping for a video, actually. Audio would suffice too :)
ntoshev 15y ago
Slide 6: "[Disk] throughput: 768 MB/sec on SATA 3" is misleading, consumer-grade rotating disks barely fill SATA 1 with ~70 MB/sec. SSD are faster in raw transfer, and if you have a RAID you are not limited by a single SATA channel any more. Slide 13: "Laugh at MongoDB" is pretty stupid, mongo makes certain trade-offs that are not for everyone, but they are still meaningful. The presentation doesn't explore the RAID options, which is a shame.
- mevodig 15y ago
  He's referring to the bus speed of SATA 3 as an upper bound -- but like you mentioned, if you throw RAID and such into the mix, I'm not so sure either that it's a very useful number.
CJefferson 15y ago
This talk totally failed to properly discuss the issues of the various kinds of fsync, instead (it seems to me) just making a selection of vague comments. That is a shame, as really properly understanding file integrity is a really important issue, which is very easy to get very wrong. When ext4 came out, I remember a large number of apps losing data, as the metadata and 'normal data' caches can get out of order, meaning that: Write file B. Rename B -> A. Can result (after a crash) in A being empty, as the rename can occur before the contents of B are actually written.
- Swannie 15y ago
  Agreed. These slides are like "Disks are not memory 101". I'd hope the talk went into more details. After an hour or so reading semi-related material I have an understanding that goes beyond these basics :-/
wazoox 15y ago
Paint me unimpressed. "eat my Data" was a way, way better and deeper presentation: http://www.oscon.com/oscon2008/public/schedule/detail/3172 http://www.oscon.com/oscon2008/public/schedule/detail/3172 Get the presentation here: http://flamingspork.com/talks/2007/06/eat_my_data.odp http://flamingspork.com/talks/2007/06/eat_my_data.odp
- Swannie 15y ago
  Thank you for that link. This is a far more interesting presentation than the OT. (Though it does assume some prior knowledge, like what fsync is, etc.)
- vimalg2 15y ago
  Found the video if anyone else is interested (same talk, but from 2007) : http://mirror.linux.org.au/pub/linux.conf.au/2007/video/talks/278.ogg http://mirror.linux.org.au/pub/linux.conf.au/2007/video/talk...

46 points by frsyuki 15y ago

Converted the Presentation PPTX to PDF for accessibility: http://dl.dropbox.com/u/5137/pdf-link/What%20Every%20Data%20Programmer%20Needs%20to%20Know%20About%20Disks%20Presentation.pdf http://dl.dropbox.com/u/5137/pdf-link/What%20Every%20Data%20...

koopajah 15y ago

Is there a video recorded of the presentation somewhere? This seemed really interesting

delta1 15y ago
I was really hoping for a video, actually. Audio would suffice too :)

ntoshev 15y ago

Slide 6: "[Disk] throughput: 768 MB/sec on SATA 3" is misleading, consumer-grade rotating disks barely fill SATA 1 with ~70 MB/sec. SSD are faster in raw transfer, and if you have a RAID you are not limited by a single SATA channel any more. Slide 13: "Laugh at MongoDB" is pretty stupid, mongo makes certain trade-offs that are not for everyone, but they are still meaningful. The presentation doesn't explore the RAID options, which is a shame.

mevodig 15y ago
He's referring to the bus speed of SATA 3 as an upper bound -- but like you mentioned, if you throw RAID and such into the mix, I'm not so sure either that it's a very useful number.

CJefferson 15y ago

This talk totally failed to properly discuss the issues of the various kinds of fsync, instead (it seems to me) just making a selection of vague comments. That is a shame, as really properly understanding file integrity is a really important issue, which is very easy to get very wrong. When ext4 came out, I remember a large number of apps losing data, as the metadata and 'normal data' caches can get out of order, meaning that: Write file B. Rename B -> A. Can result (after a crash) in A being empty, as the rename can occur before the contents of B are actually written.

Swannie 15y ago
Agreed. These slides are like "Disks are not memory 101". I'd hope the talk went into more details. After an hour or so reading semi-related material I have an understanding that goes beyond these basics :-/

wazoox 15y ago

Paint me unimpressed. "eat my Data" was a way, way better and deeper presentation: http://www.oscon.com/oscon2008/public/schedule/detail/3172 http://www.oscon.com/oscon2008/public/schedule/detail/3172 Get the presentation here: http://flamingspork.com/talks/2007/06/eat_my_data.odp http://flamingspork.com/talks/2007/06/eat_my_data.odp

Swannie 15y ago
Thank you for that link. This is a far more interesting presentation than the OT. (Though it does assume some prior knowledge, like what fsync is, etc.)
vimalg2 15y ago
Found the video if anyone else is interested (same talk, but from 2007) : http://mirror.linux.org.au/pub/linux.conf.au/2007/video/talks/278.ogg http://mirror.linux.org.au/pub/linux.conf.au/2007/video/talk...