14 ms·

The Ghost in the MP3

96 points by fugyk 12y ago

deleted 12y ago
[deleted]
dankoss 12y ago
The information present here isn't really "lost" as much as it can't be heard in the context of the other sounds in the original recording. These forms of audio compression take advantage of auditory masking[1] which means those sounds likely wouldn't be heard in the original. [1] http://en.wikipedia.org/wiki/Auditory_masking http://en.wikipedia.org/wiki/Auditory_masking
- sejje 12y ago
  The information is lost in the sense that it's in the original, and not in the lossy version. Whether or not you can hear it, the information is gone.
  - mdisraeli 12y ago
    If you can't hear it, was it ever information?
    - daeken 12y ago
      We can't see in infrared, but there's clearly information there. Same with infra/ultrasound and other sounds that are buried in our hearing. It may not be pertinent information in the case of music, but it's definitely information.
    - TheOtherHobbes 12y ago
      MP3 is audibly lossy, especially at lower sample rates. So yes - you can hear it's gone. And if you look at a spectrograph, you can see it's gone, too.
    - sejje 12y ago
      If you cover your eyes and can't see me, do I still exist?
intopieces 12y ago
If this kind of thing interests you, I highly recommend the book "MP3: The Meaning of a Format" by Jonathan Sterne [0] [0]https://www.dukeupress.edu/MP3/ https://www.dukeupress.edu/MP3/
chanux 12y ago
Suzanne Vega - Tom's Dinner https://www.youtube.com/watch?v=FLP6QluMlrg https://www.youtube.com/watch?v=FLP6QluMlrg
Buge 12y ago
It gives an error when I try to play the video in Firefox or Chrome.
- rMBP 12y ago
  Safari checking in.
- tveita 12y ago
  It played for me 30 minutes ago, but it doesn't anymore. I thought it was a embedded Youtube video at first, but it's actually a .mov file hosted in Google docs. First time I've noticed that way of hosting, maybe they have a bandwidth limit?
  - eitland 12y ago
    Played fine on my phone (Android) just now. Seems like they are using Vimeo now.
- _jomo 12y ago
  Searched it on YouTube, someone uploaded it 2 minutes ago: https://www.youtube.com/watch?v=DkQ2p5QSbyc https://www.youtube.com/watch?v=DkQ2p5QSbyc
joe_bleau 12y ago
A quick scan didn't reveal to me whether he's time aligning the signals during the subtraction. I've played with listening to the wav-mp3 signal before, and I seem to recall that the mp3 encoder would introduce a little delay. I added a transient pulse in front of the music so that I could (visually) time align the signals before subtracting them.
- mdisraeli 12y ago
  That would explain the phasing/flanging like sound which gives the ghost recording such an eerie feel
- mdisraeli 12y ago
  320kbps with highest quality setting is pretty much an industry standard now, and many DJs, myself included, make use of that. As you've looked into this before, do you know what the similar difference is like for such professional-grade encoding?
  - joe_bleau 12y ago
    (Note: no idea what mp3 encoder Audacity uses, and I'm sure the results will vary with encoder settings as well.) I just fired up Audacity and generated a click track, with the first click at 1 second in. The exported 44.1k wav file, when loaded in Audacity, shows the click at exactly 44100 samples in. The exported mp3 file, when loaded, shows the click to be around 46357 samples in. (It's a bit hard to measure, because the encoded has smeared the pulse.) Somewhere between 51-52 ms late relative to the wav file. Listening to the wav and mp3 ticks summed, the delay is obvious--they are not in sync at all. Adding 2257 samples of silence to the front end of the wav file puts them back in audible sync.
- egypturnash 12y ago
  He is probably dealing with this, given that the audio piece is not just "tomsdiner.wav - timsdiner.mp3". There's a lot of processing happening after that: ---- Verse one finds the narrator in a bustling diner, making observations about her environment. The focus of this text is external to it's author, as opposed to later verses which exist in a more subjective, internal space. Using different settings to harvest the lost material, I was able to isolate both clear, pitched content and more ephemeral transient signals. Using the python library headspace, and a reverb model of a small diner, I began to construct a virtual 3-d space. Beginning by fragmenting and scrambling the more transient material, I applied head related transfer functions to simulate the background conversation one might hear in a diner. Tracking the amplitude of the original melody in the verse, I applied a loose amplitude envelope to these signals. Thus, a remnant of the original vocal line comes through in its amplitude contour. Having constructed this background, prominent pitches from the original melody appear and disappear, located variously in this virtual space. These ephemeral sounds hint at a familiar melody, playing with aural memory and imagination, a flickering apparition hovering at the border of consciousness. ---- - found near the bottom of http://theghostinthemp3.com/theghostinthemp3.html http://theghostinthemp3.com/theghostinthemp3.html
magwhyr 12y ago
on vimeo: https://vimeo.com/107845118 https://vimeo.com/107845118
im3w1l 12y ago
The file we are watching is lossily compressed. So we are watching the lossy compression of a delta between original and lossy compression. How good is the lossy compression at capturing that delta?
MrJagil 12y ago
How does he get the information lost in compression? Does he put the compressed and uncompressed version on two different tracks with one phase-flipped?
daturkel 12y ago
This was posted almost a year ago and enjoyed a decent conversation in the comments [0] I'll repeat my comment from the time: "For what it's worth, the presence of seemingly significant signal in the difference between the original and compressed tracks does not necessarily mean that significant sonic/perceptual loss has occurred. Operating correctly, the encoder is designed to cut not just sounds that the human ear cannot hear in general (e.g. sounds above 22kHZ) but also sounds which may not be perceptible in context (e.g. the quietest signals in a loud section). So if you find something beautiful about the ghost tracks (and I think there is something beautiful to find), don't immediately jump to concluding that mp3 is awful for cutting these sounds—they might be hardly perceptible when added to the mix. Of course, at high-compression rates mp3 does begin to significantly degrade fidelity. Edit: all of this is not to put down the project—I still think it's pretty cool as art and as a demonstration of the encoder, I just didn't want people to think that this was some sort of massive failing of mp3." [0]: https://news.ycombinator.com/item?id=7955917 https://news.ycombinator.com/item?id=7955917

96 points by fugyk 12y ago

deleted 12y ago

[deleted]

The information present here isn't really "lost" as much as it can't be heard in the context of the other sounds in the original recording. These forms of audio compression take advantage of auditory masking[1] which means those sounds likely wouldn't be heard in the original. [1] http://en.wikipedia.org/wiki/Auditory_masking http://en.wikipedia.org/wiki/Auditory_masking

sejje 12y ago
The information is lost in the sense that it's in the original, and not in the lossy version. Whether or not you can hear it, the information is gone.
- mdisraeli 12y ago
  If you can't hear it, was it ever information?
  - daeken 12y ago
    We can't see in infrared, but there's clearly information there. Same with infra/ultrasound and other sounds that are buried in our hearing. It may not be pertinent information in the case of music, but it's definitely information.
  - TheOtherHobbes 12y ago
    MP3 is audibly lossy, especially at lower sample rates. So yes - you can hear it's gone. And if you look at a spectrograph, you can see it's gone, too.
  - sejje 12y ago
    If you cover your eyes and can't see me, do I still exist?

intopieces 12y ago

If this kind of thing interests you, I highly recommend the book "MP3: The Meaning of a Format" by Jonathan Sterne [0] [0]https://www.dukeupress.edu/MP3/ https://www.dukeupress.edu/MP3/

chanux 12y ago

Suzanne Vega - Tom's Dinner https://www.youtube.com/watch?v=FLP6QluMlrg https://www.youtube.com/watch?v=FLP6QluMlrg

Buge 12y ago

It gives an error when I try to play the video in Firefox or Chrome.

rMBP 12y ago
Safari checking in.
tveita 12y ago
It played for me 30 minutes ago, but it doesn't anymore. I thought it was a embedded Youtube video at first, but it's actually a .mov file hosted in Google docs. First time I've noticed that way of hosting, maybe they have a bandwidth limit?
- eitland 12y ago
  Played fine on my phone (Android) just now. Seems like they are using Vimeo now.
_jomo 12y ago
Searched it on YouTube, someone uploaded it 2 minutes ago: https://www.youtube.com/watch?v=DkQ2p5QSbyc https://www.youtube.com/watch?v=DkQ2p5QSbyc

joe_bleau 12y ago

A quick scan didn't reveal to me whether he's time aligning the signals during the subtraction. I've played with listening to the wav-mp3 signal before, and I seem to recall that the mp3 encoder would introduce a little delay. I added a transient pulse in front of the music so that I could (visually) time align the signals before subtracting them.

mdisraeli 12y ago
That would explain the phasing/flanging like sound which gives the ghost recording such an eerie feel
mdisraeli 12y ago
320kbps with highest quality setting is pretty much an industry standard now, and many DJs, myself included, make use of that. As you've looked into this before, do you know what the similar difference is like for such professional-grade encoding?
- joe_bleau 12y ago
  (Note: no idea what mp3 encoder Audacity uses, and I'm sure the results will vary with encoder settings as well.) I just fired up Audacity and generated a click track, with the first click at 1 second in. The exported 44.1k wav file, when loaded in Audacity, shows the click at exactly 44100 samples in. The exported mp3 file, when loaded, shows the click to be around 46357 samples in. (It's a bit hard to measure, because the encoded has smeared the pulse.) Somewhere between 51-52 ms late relative to the wav file. Listening to the wav and mp3 ticks summed, the delay is obvious--they are not in sync at all. Adding 2257 samples of silence to the front end of the wav file puts them back in audible sync.
egypturnash 12y ago
He is probably dealing with this, given that the audio piece is not just "tomsdiner.wav - timsdiner.mp3". There's a lot of processing happening after that: ---- Verse one finds the narrator in a bustling diner, making observations about her environment. The focus of this text is external to it's author, as opposed to later verses which exist in a more subjective, internal space. Using different settings to harvest the lost material, I was able to isolate both clear, pitched content and more ephemeral transient signals. Using the python library headspace, and a reverb model of a small diner, I began to construct a virtual 3-d space. Beginning by fragmenting and scrambling the more transient material, I applied head related transfer functions to simulate the background conversation one might hear in a diner. Tracking the amplitude of the original melody in the verse, I applied a loose amplitude envelope to these signals. Thus, a remnant of the original vocal line comes through in its amplitude contour. Having constructed this background, prominent pitches from the original melody appear and disappear, located variously in this virtual space. These ephemeral sounds hint at a familiar melody, playing with aural memory and imagination, a flickering apparition hovering at the border of consciousness. ---- - found near the bottom of http://theghostinthemp3.com/theghostinthemp3.html http://theghostinthemp3.com/theghostinthemp3.html

magwhyr 12y ago

on vimeo: https://vimeo.com/107845118 https://vimeo.com/107845118

im3w1l 12y ago

The file we are watching is lossily compressed. So we are watching the lossy compression of a delta between original and lossy compression. How good is the lossy compression at capturing that delta?

MrJagil 12y ago

How does he get the information lost in compression? Does he put the compressed and uncompressed version on two different tracks with one phase-flipped?

daturkel 12y ago

This was posted almost a year ago and enjoyed a decent conversation in the comments [0] I'll repeat my comment from the time: "For what it's worth, the presence of seemingly significant signal in the difference between the original and compressed tracks does not necessarily mean that significant sonic/perceptual loss has occurred. Operating correctly, the encoder is designed to cut not just sounds that the human ear cannot hear in general (e.g. sounds above 22kHZ) but also sounds which may not be perceptible in context (e.g. the quietest signals in a loud section). So if you find something beautiful about the ghost tracks (and I think there is something beautiful to find), don't immediately jump to concluding that mp3 is awful for cutting these sounds—they might be hardly perceptible when added to the mix. Of course, at high-compression rates mp3 does begin to significantly degrade fidelity. Edit: all of this is not to put down the project—I still think it's pretty cool as art and as a demonstration of the encoder, I just didn't want people to think that this was some sort of massive failing of mp3." [0]: https://news.ycombinator.com/item?id=7955917 https://news.ycombinator.com/item?id=7955917