10 ms·

The way this works (and I'm obviously taking a high level view here) is by comparing what is being played to what is being captured. There is an inherent latenc

The way this works (and I'm obviously taking a high level view here) is by comparing what is being played to what is being captured. There is an inherent latency in between what is called the capture stream (the mic) and the reverse stream (what is being output to the speakers, be it people taking or music or whatever), and by finding this latency and comparing, one can cancel the music from the speech captured. Within a single process, or tree of processes that can cooperate, this is straightforward (modulo the actual audio signal processing which isn't) to do: keep what you're playing for a few hundreds milliseconds around, compare to what you're getting in the microphone, find correlations, cancel. If the process aren't related there are multiple ways to do this. Either the OS provides a capture API that does the cancellation, this is what happens e.g. on macOS for Firefox and Safari, you can use this. The OS knows what is being output. This is often available on mobile as well. Sometimes (Linux desktop, Windows) the OS provides a loopback stream: a way to capture the audio that is being played back, and that can similarly be used for cancellation. If none of this is available, you mix the audio output and perform cancellation yourself, and the behaviour your observe happens. Source: I do that, but at Mozilla and we unsurprisingly have the same problems and solutions.

Log_out_ 2y ago
At the lowest level its a fouriertransform over a systems (your room the echochambers response is know from some testsound )and the expected output going through that transform on its way to the mic is subtracted. Most socks and machines have dedicated systems for that. The very same chip produces the echo of the surroundings.
gpvos 2y ago
It just seems more logical for the OS to do that, rather than the application. Basically every application that uses microphone input will want to do this, and will want to compensate for all audio output of the device, not just its own. Why does the OS not provide a way to do this?
- hpen 2y ago
  How sure are you that Basically every application wants this? So should there be a flag at the os level for enabling the cancellation? How do you control that flag?
  - gregschlom 2y ago
    It would be trivial to pass that flag in whatever API the application calls to request access to the microphone stream.
  - kalleboo 2y ago
    > How do you control that flag? https://developer.apple.com/documentation/avfaudio/avaudiosession/mode https://developer.apple.com/documentation/avfaudio/avaudiose...
- duped 2y ago
  > Basically every application that uses microphone input will want to do this The OS doesn't have more information about this than applications and it's not that obvious whether an application wants the OS to fuck around with the audio input it sees. Even in the applications where this might be the obvious default behavior, you're wrong - since most listeners don't use loudspeakers at all, and this is not a problem when they wear headphones. And detecting that (also, is the input a microphone at all?) is not straightforward. Not all audio applications are phone calls.
  - dheera 2y ago
    The OS can have multiple sound input devices for the application to choose from, "raw" and "fuckarounded with"
    - duped 2y ago
      That doesn't make sense in the context of default devices. MacOS's AVKit (or is it CoreAudio?) APIs that configure the streams created on the device makes way more sense, since it's a property of the audio i/o stream and not the devices.
generalizations 2y ago
Is there any way to apply this outside the browser? Like, is there a version of this that can be used with Pulseaudio?
- correct-horse 2y ago
  To spare others from googling: https://docs.pipewire.org/page_module_echo_cancel.html https://docs.pipewire.org/page_module_echo_cancel.html https://wiki.archlinux.org/title/PipeWire/Examples#Echo_cancellation https://wiki.archlinux.org/title/PipeWire/Examples#Echo_canc... If you're still on pulseaudio for some reason, it ships with a similar module named "module-echo-cancel": https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Modules/ https://www.freedesktop.org/wiki/Software/PulseAudio/Documen...
  - generalizations 2y ago
    Huh, thanks. I was interested in this probably 6-8 years ago, and when I went digging the stackoverflow answer mentioned elsewhere in this thread [0] was as far as I got. I guess the tech has progressed since then. [0] https://stackoverflow.com/questions/21795944/remove-known-audio-output-from-microphone-input https://stackoverflow.com/questions/21795944/remove-known-au...
    - varjag 2y ago
      It was there 8 years ago.
Johnie 2y ago
This reminds me of: >The missile knows where it is at all times. It knows this because it knows where it isn't. By subtracting where it is from where it isn't, or where it isn't from where it is (whichever is greater), it obtains a difference, or deviation https://knowyourmeme.com/memes/the-missile-knows-where-it-is https://knowyourmeme.com/memes/the-missile-knows-where-it-is
- CasperH2O 2y ago
  Up to a point that text makes a lot of sense for describing a PID controller, which is a form of control that only really looks at error and tries to get it to zero.
  - constantcrying 2y ago
    >a PID controller, which is a form of control that only really looks at error As the name implies the PID controller relies on proportional, integral and derivative information about the error. What you mean is a purely P controller, which just relies on the error. Missiles are also not guided by a PID controller, that would be silly. They (or the guidance computer in the airplane) has to take into account the trajectory of the target and guide the missile in a way to intercept that target, which is not something you can accomplish with just a PID controller.
    - notfish 2y ago
      It wouldn’t surprise me at all if early heat seeking missiles used just a PID controller, since a big part of what makes PID attractive is the ability to implement it with electrical components. Take a pair of IR photodiodes and wire them such that their difference is the error of your PID control, wire the output of the PID to the steering on your missile, and suddenly you have a missile that points at the nearest IR target (on one axis of course). Modern missiles do better than this, but a missile wired this way with a proximity fuse would hit the target a reasonable amount of the time. Not silly at all if you haven’t invented microcontrollers yet.
wormius 2y ago
For a little more context on negative feedback to those who want to know more (I believe this is what you're referring to?) Here's a short historical interview with Harold Black from AT&T on his discovery/invention of the negative feedback technique for noise reduction. It's not super explanatory but a nice historical context: https://youtu.be/iFrxyJAtJ7U?si=8ONC8N2KZwq3Jfsq https://youtu.be/iFrxyJAtJ7U?si=8ONC8N2KZwq3Jfsq Here's a more indepth circuit explanation: https://youtu.be/iFrxyJAtJ7U?si=8ONC8N2KZwq3Jfsq https://youtu.be/iFrxyJAtJ7U?si=8ONC8N2KZwq3Jfsq IIRC the issue was AT&T was trying to get cross-country calling, but to make the signal carry further you needed a louder signal. Amplifying the signal also the distortion. So Harold came up with this method that ultimately allowed enough signal reduction to allow calls to cross the country within the power constraints available. For some reason I recall something about transmission about Denver being a cut off point before the signal was too degraded... But I'm too old and forgetful so I could be misremembering something I read a while ago. If anyone has more specific info/context/citations that'd be great. Since this is just "hearsay" from memory, but I think it's something like this.

Log_out_ 2y ago

At the lowest level its a fouriertransform over a systems (your room the echochambers response is know from some testsound )and the expected output going through that transform on its way to the mic is subtracted. Most socks and machines have dedicated systems for that. The very same chip produces the echo of the surroundings.

gpvos 2y ago

It just seems more logical for the OS to do that, rather than the application. Basically every application that uses microphone input will want to do this, and will want to compensate for all audio output of the device, not just its own. Why does the OS not provide a way to do this?

hpen 2y ago
How sure are you that Basically every application wants this? So should there be a flag at the os level for enabling the cancellation? How do you control that flag?
- gregschlom 2y ago
  It would be trivial to pass that flag in whatever API the application calls to request access to the microphone stream.
- kalleboo 2y ago
  > How do you control that flag? https://developer.apple.com/documentation/avfaudio/avaudiosession/mode https://developer.apple.com/documentation/avfaudio/avaudiose...
duped 2y ago
> Basically every application that uses microphone input will want to do this The OS doesn't have more information about this than applications and it's not that obvious whether an application wants the OS to fuck around with the audio input it sees. Even in the applications where this might be the obvious default behavior, you're wrong - since most listeners don't use loudspeakers at all, and this is not a problem when they wear headphones. And detecting that (also, is the input a microphone at all?) is not straightforward. Not all audio applications are phone calls.
- dheera 2y ago
  The OS can have multiple sound input devices for the application to choose from, "raw" and "fuckarounded with"
  - duped 2y ago
    That doesn't make sense in the context of default devices. MacOS's AVKit (or is it CoreAudio?) APIs that configure the streams created on the device makes way more sense, since it's a property of the audio i/o stream and not the devices.

generalizations 2y ago

Is there any way to apply this outside the browser? Like, is there a version of this that can be used with Pulseaudio?

correct-horse 2y ago
To spare others from googling: https://docs.pipewire.org/page_module_echo_cancel.html https://docs.pipewire.org/page_module_echo_cancel.html https://wiki.archlinux.org/title/PipeWire/Examples#Echo_cancellation https://wiki.archlinux.org/title/PipeWire/Examples#Echo_canc... If you're still on pulseaudio for some reason, it ships with a similar module named "module-echo-cancel": https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Modules/ https://www.freedesktop.org/wiki/Software/PulseAudio/Documen...
- generalizations 2y ago
  Huh, thanks. I was interested in this probably 6-8 years ago, and when I went digging the stackoverflow answer mentioned elsewhere in this thread [0] was as far as I got. I guess the tech has progressed since then. [0] https://stackoverflow.com/questions/21795944/remove-known-audio-output-from-microphone-input https://stackoverflow.com/questions/21795944/remove-known-au...
  - varjag 2y ago
    It was there 8 years ago.

Johnie 2y ago

This reminds me of: >The missile knows where it is at all times. It knows this because it knows where it isn't. By subtracting where it is from where it isn't, or where it isn't from where it is (whichever is greater), it obtains a difference, or deviation https://knowyourmeme.com/memes/the-missile-knows-where-it-is https://knowyourmeme.com/memes/the-missile-knows-where-it-is

CasperH2O 2y ago
Up to a point that text makes a lot of sense for describing a PID controller, which is a form of control that only really looks at error and tries to get it to zero.
- constantcrying 2y ago
  >a PID controller, which is a form of control that only really looks at error As the name implies the PID controller relies on proportional, integral and derivative information about the error. What you mean is a purely P controller, which just relies on the error. Missiles are also not guided by a PID controller, that would be silly. They (or the guidance computer in the airplane) has to take into account the trajectory of the target and guide the missile in a way to intercept that target, which is not something you can accomplish with just a PID controller.
  - notfish 2y ago
    It wouldn’t surprise me at all if early heat seeking missiles used just a PID controller, since a big part of what makes PID attractive is the ability to implement it with electrical components. Take a pair of IR photodiodes and wire them such that their difference is the error of your PID control, wire the output of the PID to the steering on your missile, and suddenly you have a missile that points at the nearest IR target (on one axis of course). Modern missiles do better than this, but a missile wired this way with a proximity fuse would hit the target a reasonable amount of the time. Not silly at all if you haven’t invented microcontrollers yet.

wormius 2y ago

For a little more context on negative feedback to those who want to know more (I believe this is what you're referring to?) Here's a short historical interview with Harold Black from AT&T on his discovery/invention of the negative feedback technique for noise reduction. It's not super explanatory but a nice historical context: https://youtu.be/iFrxyJAtJ7U?si=8ONC8N2KZwq3Jfsq https://youtu.be/iFrxyJAtJ7U?si=8ONC8N2KZwq3Jfsq Here's a more indepth circuit explanation: https://youtu.be/iFrxyJAtJ7U?si=8ONC8N2KZwq3Jfsq https://youtu.be/iFrxyJAtJ7U?si=8ONC8N2KZwq3Jfsq IIRC the issue was AT&T was trying to get cross-country calling, but to make the signal carry further you needed a louder signal. Amplifying the signal also the distortion. So Harold came up with this method that ultimately allowed enough signal reduction to allow calls to cross the country within the power constraints available. For some reason I recall something about transmission about Denver being a cut off point before the signal was too degraded... But I'm too old and forgetful so I could be misremembering something I read a while ago. If anyone has more specific info/context/citations that'd be great. Since this is just "hearsay" from memory, but I think it's something like this.