7 ms·

This may call out to ffmpeg for pre-processing. If you're reluctant to running that on you Mac straight, you can use this wrapper script to have ffmpeg run in a

by ndr_ 2y ago

This may call out to ffmpeg for pre-processing. If you're reluctant to running that on you Mac straight, you can use this wrapper script to have ffmpeg run in a docker instance: https://gist.github.com/ndurner/636d37fd83aed4b875cdb66653017ae7 https://gist.github.com/ndurner/636d37fd83aed4b875cdb6665301... However, I found that Whisper is thrown off by background music in a prodcast - and will not recover. (That was with the mlx-community/whisper-large-v3-mlx checkpoint, OP uses distil-whisper-large-v3). I concluded for myself that Whisper might be used in larger processing pipelines that will handle such - can someone provide insights about that? The podcast I used it on was https://www.heise.de/news/KI-Update-Deep-Dive-Was-taugen-KI-Suchmaschinen-9850904.html https://www.heise.de/news/KI-Update-Deep-Dive-Was-taugen-KI-.... I ended up using Google Gemini, which handled it well. (Blog post: https://ndurner.github.io/mlx-whisper-gemini https://ndurner.github.io/mlx-whisper-gemini)

krageon 2y ago
I use a noise filter pass (really just https://github.com/richardpl/arnndn-models/blob/master/bd.rnnn https://github.com/richardpl/arnndn-models/blob/master/bd.rn... and some speech band filtering after) before doing any processing in whisper. It's worked well for me when using dirty audio (music in the background, environmental noise, etc). When there is music, you either almost can't hear it at all or you'll only hear particularly clear parts featuring singing.