13 ms·
Ask HN: Where are the good resources for learning audio processing?
I'm trying to program a harmonizer, like Jacob Collier's one built by MIT's Ben Bloomberg. I am looking for good, accessible resources on pitch shifting (whilst still sounding natural) and other terms I've heard like format shifting.
Where are some good resources for this for somebody with extensive programming experience but no experience in audio processing?
- grobibi 2y agohttp://blogs.zynaptiq.com/bernsee/time-pitch-overview/ http://blogs.zynaptiq.com/bernsee/time-pitch-overview/ Not sure if it's useful. It's probably going to involve granular synthesis.
- i_am_proteus 2y agohttps://ccrma.stanford.edu/~jos/ https://ccrma.stanford.edu/~jos/ Not much to say that Julius doesn't... open course materials for (almost) everything you might need in audio processing.
- fxtentacle 2y agoGreat link, thanks :)
- planewave 2y agoSecond anything from CCRMA, the inventors of FM synthesis and still the one top programs in the country/world.
- worstspotgain 2y agoThirded! CCRMA and its people are awesome (way more so than the rest of Stanford.)
- anymouse123456 2y agoThis comment helped get me over the barrier to take a closer look...
- geokon 2y agoI've used it as a reference or to gain a different perspective on something I'm familiar with - but it's generally way too terse to learn from Ex: https://ccrma.stanford.edu/~jos/mdft/Bessel_Functions.html https://ccrma.stanford.edu/~jos/mdft/Bessel_Functions.html
- worstspotgain 2y agoThat doesn't look terse to me, though it does require familiarity with the subject. "The last expression can be interpreted as the Fourier superposition of the sinusoidal harmonics of [expression], i.e., an inverse Fourier series sum. In other words, [expression] is the amplitude of the k-th harmonic in the Fourier-series expansion of the periodic signal x_m(t)." Many of the concepts are hyperlinked for reference. With the required familiarity, I would much rather read this than something that took seven pages to get to the point - say by assuming that the reader is unfamiliar with a premise out of an abundance of caution.
- cozzyd 2y agoJulius is a national treasure. I learned immensely from his class and textbooks...
- Chrupiter 2y agoWhen I wanted to make a python application to separate a song into the source instruments I used this: https://www.coursera.org/learn/audio-signal-processing https://www.coursera.org/learn/audio-signal-processing. I studied signal processing as a Computer Engineer student but I didn't really get it at the time, with that course I understood what I could do in practice.
- bwanab 2y agoIt's not going to directly teach you how to build a harmonizer, but this guy has a series of incredible videos on audio processing that might be helpful: https://www.youtube.com/playlist?list=PL-wATfeyAMNoirN4idjev6aRu8ISZYVWm https://www.youtube.com/playlist?list=PL-wATfeyAMNoirN4idjev...
- zengid 2y agoPretty good Audio Developer Conference talk on it here: https://www.youtube.com/watch?v=fJUmmcGKZMI https://www.youtube.com/watch?v=fJUmmcGKZMI
- Archit3ch 2y agoAlso check out the blog (https://signalsmith-audio.co.uk/writing/2023/stretch-design/ https://signalsmith-audio.co.uk/writing/2023/stretch-design/) and corresponding library on GitHub.
- aj7 2y agoUse LabView as a calculation engine to do experiments. The advantage is you get system-like diagrams.
- bwestergard 2y agoYou should learn Supercollider, starting with Eli Fieldsteel's tutorials.
- an_aparallel 2y agoHey - one of the industry standard time stretching library is "elastique" by Zynaptiq (licensed, not open source). Used by Ableton, FL Studio etc. If you want to peak into some source code - you can look into Rubberband library: https://breakfastquay.com/rubberband/ https://breakfastquay.com/rubberband/ Rubberband is one of the time stretching/pitch shifting algorithms used in Reaper. You can download reaper trial and listen to the results with different parameters to see how you can tweak the code and if that gets any results you're happy with: https://www.reaper.fm/ https://www.reaper.fm/
- IndySun 2y ago>Hey - one of the industry standard time stretching library is "elastique" by Zynaptiq. The company is Zplane, not Zynaptiq. Easy mistake, there is a little overlap. https://licensing.zplane.de/ https://licensing.zplane.de/
- an_aparallel 2y agoI mix these 2 up constantly, thanks! :)
- a-dub 2y agohttp://www.ee.columbia.edu/~dpwe/ http://www.ee.columbia.edu/~dpwe/ https://github.com/librosa/librosa https://github.com/librosa/librosa
- planewave 2y agoProfessor Puckett, inventor of Max and PureData (the two top visual programming languages for DSP) has a book, The Theory and Practice of Electronic Music, with interactive examples written in PD, this one probably has an example exercise for a pitch shifter [0] I often recommend also Music and Computers originally out of Columbia. [1] [0]http://msp.ucsd.edu/techniques.htm http://msp.ucsd.edu/techniques.htm [1]https://musicandcomputersbook.com/ https://musicandcomputersbook.com/
- fxtentacle 2y agoAudio is half art, half science. That's why I'd try to find someone with experience. Back in university, I heard lectures on FFT and its applications to audio signal processing. So open access university courses would be the second place I'd look. The approach I always try first is to ask people I know if they can recommend a conference/meetup. For example, the annual JUCE events appear to be chock full with VST plugin developers. There's also private schools like SAE where you (or your employer) can pay for you to have an hour with one of their lecturers to ask questions.
- the__alchemist 2y agoI would pick up a microcontroller dev board that has a mic built in (Eg one of the STM32 discoveries). Also get a "codec" dev board. (Or alternatively, use the MCU's onboarod DAC). Get it to receive audio, process it using DSP, then output it, and/or save to memory. This will really force you to understand it.
- spacechild1 2y agoBad advice. I have no idea how using a microcontroller would help someone understand pitchshifting algorithms.
- p1esk 2y agoWhy not just use a regular laptop for this? There’s a ton of low level sound processing libraries for every OS.
- dsp_person 2y agoI find [1] a good reference. A con is the examples are in matlab, but it's clear enough between the text and matlab code to write your own implementation. Also [2] is a decent book for overall dsp concepts. [1] DAFX - Digital Audio Effects (Second Edition) Edited by Udo Zölzer https://dafx.de/DAFX_Book_Page_2nd_edition/index.html https://dafx.de/DAFX_Book_Page_2nd_edition/index.html [2] Understanding Digital Signal Processing, Richard Lyons
- cushychicken 2y agoZolzer’s book is the best out there on the topic that I know of.
- a-dub 2y agolyons is a good intro but maybe a bit handwavey at times (although my copy is an edition from the 90s). consider maybe backing it up with one of the textbooks like oppenheim (the classic) or manolakis (one that i think i remember liking).
- octetta 2y agoThis dude has nice content too : https://youtube.com/@lantertronics?si=eXjrDn8zlHwQogdX https://youtube.com/@lantertronics?si=eXjrDn8zlHwQogdX
- ivanjermakov 2y ago2021 (120 comments): https://news.ycombinator.com/item?id=27273706 https://news.ycombinator.com/item?id=27273706
- cageface 2y agoThe Will Pirkle books have a lot of good info and code to get you started: https://www.willpirkle.com https://www.willpirkle.com Audio programming is a lot of fun but it's the most challenging domain I've ever worked in. You have to be very careful with what you do on the audio thread. No locks, no memory allocation etc. Messing this up can result in some really ugly audio artifacts.
- jarmitage 2y agoReal-time audio programming 101: time waits for nothing http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing http://www.rossbencina.com/code/real-time-audio-programming-... C++ for Real-Time Audio Programming: https://learn.bela.io/tutorials/c-plus-plus-for-real-time-audio-programming/course-introduction/ https://learn.bela.io/tutorials/c-plus-plus-for-real-time-au...
- peterbmarks 2y agoIf you are interested in Apple platforms check out https://www.audiokit.io https://www.audiokit.io
- mikewarot 2y agoGnu Radio can easily handle audio I/O as well as it does IQ signals from SDR front ends. It's cross platform and you just build flow graphs, which then can be executed.
- Ylpertnodi 2y agowww.airwindows.com may help.
- squeaky-clean 2y agoChris @ airwindows is super nice to release all his plugins and source code for free. But the code quality is really bad. That doesn't matter if you're using a plugin in the production of a song and it works well. But for learning dsp, it's a bad resource.
- mottosso 2y agoI was curious what you meant and went to have a look. At first all seemed well, until I got to the actual audio processing part. :) https://github.com/airwindows/airwindows/blob/master/plugins/WinVST/BassDrive/BassDriveProc.cpp https://github.com/airwindows/airwindows/blob/master/plugins... Then again, maybe this is the norm for audio engineers? Not my field.
- al2o3cr 2y agoThe documentation for that specific module even calls it out as "painfully hard-coded biquad filter code", YMMV. I'd guess those files aren't what the author actually edits - there are commits that suggest that they are _generated_ from "boilerplate", and even a few files that seem to have failed interpolations ("__MyCompanyName__" in some copyright lines, for instance) A lot of files also seem to have duplicated code, down to individual comments. For instance, the comment on line 24 reoccurs on line 344 of this effect: https://github.com/airwindows/airwindows/blob/master/plugins/WinVST/Doublelay/DoublelayProc.cpp https://github.com/airwindows/airwindows/blob/master/plugins... and in the Mac AU version on line 267: https://github.com/airwindows/airwindows/blob/master/plugins/MacAU/Doublelay/Doublelay.cpp#L267 https://github.com/airwindows/airwindows/blob/master/plugins... and in the Linux VST version on lines 24 and 344: https://github.com/airwindows/airwindows/blob/master/plugins/LinuxVST/src/Doublelay/DoublelayProc.cpp#L24 https://github.com/airwindows/airwindows/blob/master/plugins...
- djaychela 2y ago
- MaysonL 2y agoCheck out officehours.global – a lot of audio people hang out there.
- checker659 2y agoAudio Anecdotes series
- rhizome31 2y agoGreat channel on fundamentals of digital audio: https://www.youtube.com/@akashmurthy https://www.youtube.com/@akashmurthy
- geekraver 2y agohttps://www.audiolabs-erlangen.de/resources/MIR/FMP/C0/C0.html https://www.audiolabs-erlangen.de/resources/MIR/FMP/C0/C0.ht... might have something
- bbloomberg 2y agoHello :) all of the resources mentioned here are great! One step I’d add to the learning part (and it’s what we did when building Jacob’s) is to spend a lot of time trying out existing implementations to determine what you like and don’t like. For example, many of them don’t have great low end. Some are “sluggish” and need external enveloping. Getting a sense for what’s out there can help to provide a North Star when you write your own. Some classics are the Eventide H3000, IZotope Vocal Synth, TC Voice Live, Antares Harmony Engine, and Soundtoys Little Alterboy.
- jkalsdjf209 2y agoGreat discord where many audio programmers hang out, may be able to answer your specific questions when you get to the more detailed areas. https://www.theaudioprogrammer.com/discord https://www.theaudioprogrammer.com/discord A blog post about an open source C++ pitch shifting library: https://signalsmith-audio.co.uk/writing/2023/stretch-design/ https://signalsmith-audio.co.uk/writing/2023/stretch-design/ And accompanying ADC talk: https://www.youtube.com/watch?v=fJUmmcGKZMI https://www.youtube.com/watch?v=fJUmmcGKZMI
- RossBencina 2y agoLearn how to search and read the research literature, most of the rest is just dabbling in the shallows. MTG Barcelona has been doing R&D for Yamaha since the 90s. They have published a lot of work on time-frequency transformation and have certainly implemented harmonizers and time stretchers. Look for papers and thesis by Jordi Bonada, Alex Loscos and certainly others too: https://www.upf.edu/web/mtg/research/publications https://www.upf.edu/web/mtg/research/publications Needless to say pitch shifting is nothing new, so going back to research publications from the 90s may help. Publications might be found in early conferences of DAFX, ICMC, IEEE Mohonk, ACM multimedia, JAES, etc etc. Try keywords like "waveform similarity overlap add" WSOLA, "Lent's algorithm". The musicdsp mailing list has discussed pitch shifting many times over the years. Participants have included engineers with fairly intimate familiarity with algorithms employed by Eventide, z-plane, etc. I would search the archives (you may need to do some digging to find all of the archives dating back to the late 90s). Maybe look at expired patents from Creative Labs, Eventide and Antares if you feel comfortable exposing yourself to that literature.
- brudgers 2y agoI'm trying to program a harmonizer Why? Not questioning your motivations. Rather I’m curious what they are.