6 ms·
> Because flash does not overwrite anything, ever. This is repeated multiple times in the article, and I refuse to believe it is true. If NVME/SSDs never overw
by notaplumber 5y ago
> Because flash does not overwrite anything, ever.
This is repeated multiple times in the article, and I refuse to believe it is true. If NVME/SSDs never overwrote anything, they would quickly run out of available blocks, especially on OSs that don't support TRIM.
- cduzz 5y agoThere's nuance to this; the deletes / overwrites are accomplished by bulk wiping entire blocks. Rather than change the paint color in a hallway you have to tear down the house and build a new house in the vacant lot next door that's a duplicate of the original, but with the new hallway paint. To optimize, you keep a bucket of houses to destroy, and a bucket of vacant lots, and whenever a neighborhood has lots of "to be flattened houses" the remaining active houses are copied to a vacant lot and the whole neighborhood is flattened. So, things get deleted, but not in the way people are used to if they imagine a piece of paper and a pencil and eraser.
- zdw 5y agoinspired by that last sentence, the analogy could be rewritten as: - lines on page - pages of paper - whole notebooks and might be easier for people to grok than the earlier houses/paint analogy.
- jazzyjackson 5y agoI don’t know, I like the drama of copying a neighborhood and tearing down the old one xD
- slaymaker1907 5y agoJust to add to the explanation, SSDs are able to do this because they have a layer of indirection akin to virtual memory. This means that what your OS thinks is byte 800000 of the SSD may change it's actual physical location on the SSD over time even in the absence of writes or reads to said location. This is a very important property of SSDs and is a large reason why log structured storage is so popular in recent times. The SSD is very fast at appends, but changing data is much slower.
- hawski 5y agoAre there write-once SSDs? They would have a tremendous capacity. Probably good for long term backups or archiving. Also possibly with a log structured filesystem only.
- wtallis 5y agoI don't think anyone would make literally write-once drives with flash memory; that's more optical disk territory. But zoned SSDs and host-managed SMR hard drives make explicit the distinction between writes and larger-scale erase operations, while still allowing random-access reads.
- eqvinox 5y agoMaking them write-once doesn't increase the capacity; that's mostly limited by how many analog levels you can distinguish on the stored charge, and how many cells you can fit. The management overhead and spare capacity to make SSDs rewritable is –to my knowledge– in the single digit percentages. (Also you need the translation layer even for write-once since flash generally doesn't come 100% defect free. Not sure if manufacturers could try to get it there, but that'd probably drive the cost up massively. And the translation layer is there for rewritable flash anyway... the cost/benefit tradeoff is in favor of just living with a few bugged cells.)
- entangledqubit 5y agoI suspect that hawki was assuming that a WORM SSD would be based on a different non-flash storage medium. I don't know any write once media that has similar read/write access times to an SSD. FWIW, there are WORM microsd cards available but it looks like they still use flash under the hood.
- daniellarusso 5y agoSpoiler alert - This is the plot to ‘The Prestige’.
- wand3r 5y agoI think the explanation is sound maybe (I am not that familiar) but the analogy gets a bit lost when you talk about buckets of houses and buckets of vacant lots. Maybe there is a better analogy or paradigm to view this through.
- cduzz 5y agoI should have been a little more clear -- the urban planner managing the house building / copying and neighborhood destruction (the realtime controller) The rules are: 1) You can build a house kinda quickly 2) You can't modify a house once it is built 3) you can only build a house on a vacant lot 4) you can change the "mailing address" (relative to the physical location) of the house 5) you can only knock down whole blocks of houses at once (not one at a time) 6) each time you flatten a block more crap accumulates in that block until after a while you can't build there anymore. 7) the flatten / rebuild step may be quite slow (because you have lots of houses to build) 8) You can lie and say you built a house before it is finished, if you don't have too many houses to build. (if you've got an SSD with a capacitor / battery or tiny cache and reserved area for that cache) 9) you've lied to the user and you actually have 5-100% more build-able area than you've advertised. 10) you have a finite area so eventually the dead space accumulates to the point where you can no longer safely build. So -- you keep track of vacant lots and "dead" houses (abandoned but not flattened); whenever you've got spare time you will copy blocks with some ratio of "live" to abandoned houses to new lots so the new block only has live houses. These pending / anticipatory compaction/garbage collection operations are what I refer to as "buckets" -- having to compact 300 (neighborhoods) blocks to achieve 300 writes is going to result in glacial performance because of this huge write amplification (behind the scenes the drive is duplicating 100s of mb / gb of data to write a small amount of user modifications) As you might imagine, there are lots of strategies to how to approach this problem, some of which give you an SSD with extremely unpredictable (when full) performance, others will give a much more consistent but "slower" performance.
- aidenn0 5y agoPerhaps they mean it must erase an entire block before writing any data, unlike a disk that can write a single sector at a time?
- dragontamer 5y agoThe issue is that DDR4 is like that too. Not only the 64 byte cache line, but DDR4 requires a transfer to the sense amplifiers (aka a RAS, row access strobe) before you can read or write. The RAS command eradicated the entire row, like 1024 bytes or so. This is because the DDR4 cells only have enough charge for one reliable read, after that the capacitors don't have enough electrons to know if a 0 or 1 was stored. A row close command returns the data from the sense amps back to the capacitors. Refresh commands renew the 0 or 1 as the capacitor can only hold the data for a few milliseconds. ------ The CAS latency statistic assumes that the row was already open. It's a measure of the sense amplifiers and not of the actual data.
- zymhan 5y agoWhat does DDR have to do with NVMe?
- nine_k 5y agoYou can't write a byte, or a word, either. The "fact" that you can do it in your program without disturbing bytes around it is a convenient fiction that the hardware fabricates for you.
- dragontamer 5y agoDDR4 is effectively a block device and not 'random access'. Pretty much only cache is RAM proper these days (aka: all locations have equal access time... that is, you can access it randomly with little performance loss).
- mirker 5y agoI’m confused. What’s the difference between a cache line and a row in RAM? They’re both multiples of bytes. You have data sharing per chunk in either case. The distinction seems to be how big the chunk is not uniformity of access time (is a symmetrical read disk not a block device?)
- Dzugaru 5y agoOf course it does [0]. It's just it assigns writes as evenly as possible (to have as even wear as possible), so log-like internal "file system" is a way to go. https://pages.cs.wisc.edu/~remzi/OSTEP/file-ssd.pdf https://pages.cs.wisc.edu/~remzi/OSTEP/file-ssd.pdf
- eqvinox 5y agoIt's true and untrue depending on how you look at it. Flash memory only supports changing/"writing" bits in one direction, generally from 1 to 0. Erase, as a separate operation, clears entire sectors back to 1, but is more costly than a write. (Erase block size depends on the technology but we're talking MB on modern flash AFAIK, stuff from 2010 already had 128kB.) So, the drives do indeed never "overwrite" data - they mark the block as unused (either when the OS uses TRIM, or when it writes new data [for which it picks an empty block elsewhere]), and put it in a queue to be erased whenever there's time (and energy and heat budget) to do so. Understanding this is also quite important because it can have performance implications, particularly on consumer/low-end devices. Those don't have a whole lot of spare space to work with, so if the entire device is "in use", write performance can take a serious hit when it becomes limited by erase speed. [Add.: reference for block sizes: https://www.micron.com/support/~/media/74C3F8B1250D4935898DB7FE79EB56E7.ashx https://www.micron.com/support/~/media/74C3F8B1250D4935898DB... - note the PDF creation date on that is 2002(!) and it compares 16kB against 128kB size.]
- IshKebab 5y agoBy any reasonable definition they do overwrite data. It's just that they can't overwrite less than a block of data.
- matheusmoreira 5y ago> Understanding this is also quite important because it can have performance implications Security implications too. The storage device cannot be trusted to securely delete data.
- throwaway09223 5y agoThe author clearly explains how this works in the sentence immediately following. "Instead it has internally a thing called flash translation layer (FTL)" ...
- notaplumber 5y agoI unfortunately skimmed over this, isotopp's explanation helped clear things up in my head.
- throwaway09223 5y agoI just saw his post, it's a great explanation. It might also help to keep in mind that both regular disk drives and solid state drives remap bad sectors. Both types of disks maintain an unaddressable storage area which is used to transparently cover for faulty sectors. In a hard drive, faulty sectors are mapped during production and stored in the p-list, and are remapped to sectors in this extra hidden area. Sectors that fail at runtime are recorded in the g-list and are likewise remapped. Writes may usually go to the same place in a hard drive, but it's not guaranteed there either.
- dannyw 5y agoThis is not true anymore for many recent SMR HDDs. They have a translation layer, just like flash storage. This is because for SMR HDDs, each block can either be SMR (higher density, EXTREMELY SLOW WRITES like <10mb/s possible, erases will remove multiple blocks just like flash memory), or normal (standard density, normal write speeds). The controller abstracts this away and does writes as normal, but while the drive is idle, the controller in the background, converts these standard blocks into SMR blocks. This is also why SMR HDDs support TRIM.
- throwaway09223 5y agoThanks for the info that makes a lot of sense. It looks like this tech has emerged in the time since I last did much work with disk drives. Seems it's increasingly a bad idea to presume the implementation of internals.
- isotopp 5y agoFlash has a flash translation layer (FTL). It translates linear block addresses (LBA) into physical addresses ("PHY"). Flash can write blocks at a granularity similar to a memory page (cells, around 4-16 KB). It can erase only sets of blocks, at a much larger granularity (around 512-ish cell sized blocks). The FTL will try to find free pages to write your data to. In the background, it will also try to move data around to generate unused erase blocks and then erase them. In flash, seeks are essentially free. That means that it does no longer matter if blocks are adjacent. Also, because of the FTL, adjacent FTL are not necessarily adjacent on the physical layer. And even if you do not rewrite a block, it may be that the garbage collection moves data around at the PHY layer in order to generate completely empty erase blocks. The net effect is that positioning as seen from the OS no longer matters at all from the OS layer, and that the OS layer has zero control over adjacency and erase at the PHY layer. Rewriting, defragging, or other OS level operations cannot control what happens physically at the flash layer. TRIM is a "blatant layering violation" in the Linus sense: It tells the disk "hardware" what the OS thinks it no longer needs. TRIM'ed blocks can be given up and will not be kept when the garbage collector tries to free up an erase page.
- notaplumber 5y agoThanks for this part, I feel like this was a crucial piece of information I was missing. Also explains my observations about TRIM not being as important as people claim it is, the firmware on modern flash storage seems more than capable of handling this without OS intervention.
- isotopp 5y agoThe GC in the device cleans up. TRIM is useful, it gives the GC important information. TRIM is not that important as long as the device is not full (less than 80%, generally speaking, but it is very easy to produce pathological cases that are way off in either direction). Once the device fills up above that it is crucial.
- anarazel 5y ago> In flash, seeks are essentially free. That means that it does no longer matter if blocks are adjacent. > The net effect is that positioning as seen from the OS no longer matters at all from the OS layer, and that the OS layer has zero control over adjacency and erase at the PHY layer. Rewriting, defragging, or other OS level operations cannot control what happens physically at the flash layer. I don't agree with this. The "OS visible position" is relevant, because it influences what can realistically be written together (multiple larger IOs targeting consecutive LBAs in close time proximity). And writing data in larger chunks is very important for good performance, particularly in sustained write workloads. And sequential IO (in contrast to small random IOs) does influence how the FTL will lay out the data to some degree.