6 ms·

This inspired me to read up on the low-level details of CD structure. I'm curious if anybody scanned an entire CD and shared the results, so that we could work

by d332 2y ago

This inspired me to read up on the low-level details of CD structure. I'm curious if anybody scanned an entire CD and shared the results, so that we could work with a raw image of disc that contains all its quirks, as opposed to the typical .iso format?

sho 2y ago
Coming back to this, having read some of the (great!) replies, I'm going to go out on a limb and say that in theory, this sounds possible, and fun, but highly impractical. I'll assume that by "scan" you mean a high end "flatbed scanner" optical scan which would return a 2D bitmap. It's impractical because the resolution required to retrieve the data "flatbed scanner style" is comically high, perhaps 50k dpi, far beyond the capability of any commercial unit and well into scanning microscope territory. Sure, from my understanding, it looks technically possible. But it would be a very significant and costly project just to assemble the image in the first place. Even if you had that, the resulting file would be hilariously huge (something like 122GB), extremely difficult to work with, and you would be starting from scratch implementing some kind of visual pathfinding helical decoder to painstakingly unravel the linear coil of data the scan just sort of blatted into two dimensions. It's a cool idea. But it's comically, exponentially harder than just using the equipment as intended to just read the laser returns off the disk directly, into a far, far more easily dealt with format. I'm adding that CD scan to my list of things I'd like to do if I ever get really rich.
ssl-3 2y ago
Audio CDs were never ripped/transferred as ISO files. ISO-9660 is a filesystem that came years later, and Redbook audio CDs simply do not contain files. If you want to look at the structure of a whole audio CD, then one way is to rip it with a decent tool (perhaps cdrdao or EAC) and generate a bin/cue file pair as an output.
- d332 2y ago
  But that's not my goal. I'd like to be able to observe every grove, the physical encoding of data, and see if I could implement decoding from scratch. First problem is though that I don't know how to get a microscopic image of the disc.
  - taneliv 2y ago
    You might do well enough with https://en.wikipedia.org/wiki/Cdparanoia https://en.wikipedia.org/wiki/Cdparanoia without needing use different hardware to scan the disc. Instead it relies on the CD drive's ability to report on inaccuracies in keeping in sync with the grooves.
  - ssl-3 2y ago
    You don't need a microscopic image of a disc to do that; a two-dimensional photograph is of essentially no advantage here. All you need is the unmolested data from that disc. The data is arranged on a singular spiral groove starting from the center and slowly winding its way towards the outside. The data is completely linear: It begins at the beginning, and continues to the very end without interruption. This is all akin to (although opposite of) how a single-track vinyl record is physically laid out. The entire CD -- whatever it contains -- is just a continuous string of pits and lands. And to observe that string as it appears on a real disc, all you need to get started is a regular old-school CD player and some appropriate data acquisition gear, and maybe an oscilloscope to help figure out what you're looking at. The optics and basic motor controls are already solved problems, and it doesn't even have to be particularly fast data acquisition gear by today's standards to record what is happening.
    - hunter2_ 2y ago
      The "unmolested data" would still have undergone error correction though, wouldn't it? I don't think a bin/cue rip would contain the redundant stuff, which GP seems interested in, nor the subcodes (of which some are represented in the cue file, while the bin file is PCM audio). And at the risk of taking us well beyond the rainbow books, I'll just leave this here: https://www.psxdev.net/forum/viewtopic.php?f=70&t=1266 https://www.psxdev.net/forum/viewtopic.php?f=70&t=1266
      - ssl-3 2y ago
        There is a layer betwixt the optical reflection and the audio output that exists only as raw signals, before any molestation/error correction occurs. There cannot not be this layer. (And with a sufficiently-old-school CD player, it is probably not even challenging to get to it. The less-integrated the parts are, the better.)
thristian 2y ago
It's really difficult. Unlike floppy disks, where you tell the drive to seek and get back raw magnetic pulses (so you can produce raw flux images), or hard disks where you tell the drive to read an arbitrary sector and get a blob of data (so you can produce sector-level images), the protocol for talking to a CD ROM involves asking for track/sector addresses, which means you have to trust the drive to interpret all the track metadata and error-correction for you - you generally can't just dump the "raw" data and do the interpretation yourself. That's why the most robust CD image format is the BIN/CUE format. The BIN file contains all the sectors the drive allows us to read, the CUE file contains the disc metadata as interpreted for us by the drive firmware. There are some drives which support extra "raw read" commands, but they're incredibly rare and consequently in great demand by CD preservation projects like redump.org. Some people have used the contents of BIN/CUE data to reconstruct what should actually be on the disk, but that's not quite the same thing. Here's a great explanation of the CD structure in all its complexity: https://github.com/higan-emu/emulation-articles/tree/master/compact-discs/structure https://github.com/higan-emu/emulation-articles/tree/master/...
- edude03 2y ago
  Apparently makemkv forum members created some patched firmware that lets you raw read BRs for the sake of extracting metadata that’s intentionally hidden for DRM. Though I’ll have to recheck my understanding since you’re saying you can’t actually raw read disks anyway
- AkBKukU 2y ago
  Even BIN/CUE is not enough. It cannot store subchannel data like CD+G and is only able to hold a single session which breaks bluebook CDs with audio and data. We do not currently have a widely supported CD standard for storing data from a CD that can properly hold all data. Aaru [0] is close, but still has to output back to other formats like BIN/CUE to use the contents of the disc. [0] https://www.aaru.app/#/ https://www.aaru.app/#/

by d332 2y ago

sho 2y ago

Coming back to this, having read some of the (great!) replies, I'm going to go out on a limb and say that in theory, this sounds possible, and fun, but highly impractical. I'll assume that by "scan" you mean a high end "flatbed scanner" optical scan which would return a 2D bitmap. It's impractical because the resolution required to retrieve the data "flatbed scanner style" is comically high, perhaps 50k dpi, far beyond the capability of any commercial unit and well into scanning microscope territory. Sure, from my understanding, it looks technically possible. But it would be a very significant and costly project just to assemble the image in the first place. Even if you had that, the resulting file would be hilariously huge (something like 122GB), extremely difficult to work with, and you would be starting from scratch implementing some kind of visual pathfinding helical decoder to painstakingly unravel the linear coil of data the scan just sort of blatted into two dimensions. It's a cool idea. But it's comically, exponentially harder than just using the equipment as intended to just read the laser returns off the disk directly, into a far, far more easily dealt with format. I'm adding that CD scan to my list of things I'd like to do if I ever get really rich.

ssl-3 2y ago

Audio CDs were never ripped/transferred as ISO files. ISO-9660 is a filesystem that came years later, and Redbook audio CDs simply do not contain files. If you want to look at the structure of a whole audio CD, then one way is to rip it with a decent tool (perhaps cdrdao or EAC) and generate a bin/cue file pair as an output.

d332 2y ago
But that's not my goal. I'd like to be able to observe every grove, the physical encoding of data, and see if I could implement decoding from scratch. First problem is though that I don't know how to get a microscopic image of the disc.
- taneliv 2y ago
  You might do well enough with https://en.wikipedia.org/wiki/Cdparanoia https://en.wikipedia.org/wiki/Cdparanoia without needing use different hardware to scan the disc. Instead it relies on the CD drive's ability to report on inaccuracies in keeping in sync with the grooves.
- ssl-3 2y ago
  You don't need a microscopic image of a disc to do that; a two-dimensional photograph is of essentially no advantage here. All you need is the unmolested data from that disc. The data is arranged on a singular spiral groove starting from the center and slowly winding its way towards the outside. The data is completely linear: It begins at the beginning, and continues to the very end without interruption. This is all akin to (although opposite of) how a single-track vinyl record is physically laid out. The entire CD -- whatever it contains -- is just a continuous string of pits and lands. And to observe that string as it appears on a real disc, all you need to get started is a regular old-school CD player and some appropriate data acquisition gear, and maybe an oscilloscope to help figure out what you're looking at. The optics and basic motor controls are already solved problems, and it doesn't even have to be particularly fast data acquisition gear by today's standards to record what is happening.
  - hunter2_ 2y ago
    The "unmolested data" would still have undergone error correction though, wouldn't it? I don't think a bin/cue rip would contain the redundant stuff, which GP seems interested in, nor the subcodes (of which some are represented in the cue file, while the bin file is PCM audio). And at the risk of taking us well beyond the rainbow books, I'll just leave this here: https://www.psxdev.net/forum/viewtopic.php?f=70&t=1266 https://www.psxdev.net/forum/viewtopic.php?f=70&t=1266
    - ssl-3 2y ago
      There is a layer betwixt the optical reflection and the audio output that exists only as raw signals, before any molestation/error correction occurs. There cannot not be this layer. (And with a sufficiently-old-school CD player, it is probably not even challenging to get to it. The less-integrated the parts are, the better.)

thristian 2y ago

It's really difficult. Unlike floppy disks, where you tell the drive to seek and get back raw magnetic pulses (so you can produce raw flux images), or hard disks where you tell the drive to read an arbitrary sector and get a blob of data (so you can produce sector-level images), the protocol for talking to a CD ROM involves asking for track/sector addresses, which means you have to trust the drive to interpret all the track metadata and error-correction for you - you generally can't just dump the "raw" data and do the interpretation yourself. That's why the most robust CD image format is the BIN/CUE format. The BIN file contains all the sectors the drive allows us to read, the CUE file contains the disc metadata as interpreted for us by the drive firmware. There are some drives which support extra "raw read" commands, but they're incredibly rare and consequently in great demand by CD preservation projects like redump.org. Some people have used the contents of BIN/CUE data to reconstruct what should actually be on the disk, but that's not quite the same thing. Here's a great explanation of the CD structure in all its complexity: https://github.com/higan-emu/emulation-articles/tree/master/compact-discs/structure https://github.com/higan-emu/emulation-articles/tree/master/...

edude03 2y ago
Apparently makemkv forum members created some patched firmware that lets you raw read BRs for the sake of extracting metadata that’s intentionally hidden for DRM. Though I’ll have to recheck my understanding since you’re saying you can’t actually raw read disks anyway
AkBKukU 2y ago
Even BIN/CUE is not enough. It cannot store subchannel data like CD+G and is only able to hold a single session which breaks bluebook CDs with audio and data. We do not currently have a widely supported CD standard for storing data from a CD that can properly hold all data. Aaru [0] is close, but still has to output back to other formats like BIN/CUE to use the contents of the disc. [0] https://www.aaru.app/#/ https://www.aaru.app/#/