[0] https://mitpress.mit.edu/books/audio-programming-book
Edit: My goal is to make a mini-synth which takes input from the computer keyboard.
——
The founder, and sole developer of Mutable Instruments (a well known modular audio/synth maker) wrote about her recommendations on how to get started in audio programming here [0]. Much recommended for the cited books, and general/business advice on making a hardware audio product.
[0] https://modwiggler.com/forum/viewtopic.php?p=2858814#p285881...
——
My own post about this was from a mini thread on HN about programming sound that may also interest you:
https://news.ycombinator.com/item?id=20491766
——
Last note! Another synth maker personally recommended the same DSP book as noted in the comment above (on Modwiggler), so it’s probably worth checking out!
Udo Zolzer's "DAFX"
The Computer Music Tutorial by Curtis Roads is a good companion here if you want inner workings. You might be able to snag a PDF copy online to see if it's for you.
Audio Programming Book is entirely in C and can be a bit of a pain to get things working in C if you're new to programming (although highly worth the initial struggle if you're looking at it with a career in mind), although I'd still recommend stepping from Max/MSP into MATLAB/Python, which focuses on functionality rather than programming itself.
Python is also bit more beginner friendly, although C++ will probably end up being your long term bet.
I'd pick up the Pyo library for python, and/or a tutorial like this one, after dabbling with Max/MSP a bit: https://python.plainenglish.io/build-your-own-python-synthes...
Then if you want to get even deeper into the DSP, you could go with pretty much any of the other resources the others have mentioned here. My personal faves are the Steven Smith DSP book, Julius Smith's resources (with its runnable MATLAB scripts), and DAFX book by Udo Zolzer.
- "Introduction to Computer Music"[0] by Nick Collins: very focused on audio programming for musical applications but you can easily extrapolate for other domains. Lots of pseudo-code algorithms are provided.
- "The Scientist and Engineer's Guide to Digital Signal Processing"[1] by Steven W. Smith: the bread and butter for everything concept related to DSP. Everything is explained in a crystal-clear fashion, lots of sample code to get you started with digital filters and analysis (although written in BASIC but easily translatable to C/C++ or even Python).
Also there is this one article "Real-time audio programming 101: time waits for nothing"[2] by Ross Bencina (creator of AudioMulch, Portaudio, etc.) about the basics concepts of real-time audio programming
[0] https://www.wiley.com/en-gb/Introduction+to+Computer+Music-p...
[2] http://www.rossbencina.com/code/real-time-audio-programming-...
I wanted to help myself and other folks develop better intuitions around the material, particularly focusing on short animations to develop better visual intuition along with working code examples of the material in the books.
Books covered (with a notebook for each chapter):
* Musimathics volumes 1 & 2 by Gareth Loy
* Introduction to the DFT by Julius Smith
* Introduction to Digital Filters by Julius Smith
* Physical Audio Signal Processing by Julius Smith
also a couple not directly about audio but helpful for the domain:
* Coding the Matrix by Philip Klein
* Accelerated C++ by Andrew Koenig and Barbara Moo
Hope someone gets some value from these - have fun!
I am comfortable writing low level efficient real-time code in C (I'm an ex game dev). I understand most synthesis techniques in pretty good detail. I could probably write a book on subtractive synthesis, and I have implemented a complete functioning FM synth.
I get the fundamental concepts around DSP: Nyquist, sampling, Fourier analysis, etc.
But when it comes to sitting down and, say, designing and implementing a filter from scratch, I'm pretty well stuck. Part of it is that most of the material I've found either goes straight to heavy calculus (which I can kind of hack but am pretty rusty on) or to MATLAB (which I don't have).
Likewise, implementing a simple digital sawtooth or square oscillator analytically is trivial. Sample playback is also a piece of cake. But doing those without aliasing seems to jump from super-easy mode all the way to hardcore nightmare. All of a sudden I'm trying to understand what minBLEPs are and other stuff that feels over my head.
Any recommendations on how to get over this hump? Should I get MATLAB?
Web Audio API and JUCE are the modern building blocks for mostly everything out there; if you're familiar with these, that's a really good place to start.
If you're looking for something accessible, SOUL is an excellent domain-specific scripting language that leverages JUCE underneath; see https://github.com/julianstorer for more context.
I'm not as sold on Max/MSP or Pd, though they're fun to play with for a brief moment.
https://m.youtube.com/watch?v=tgamhuQnOkM&t=77s
https://m.youtube.com/watch?v=OSCzKOqtgcA&t=2s
I asked myself too how I should get started with Audio Programming in 2019. I ended up building my own little synth in the progress and I ended up talking to people from Ableton and even gave a talk at Native Instrument's headquarters here in Berlin.
I blogged about my experience and my product here: https://timdaub.github.io/2020/02/19/wasm-synth/
* JUCE for making VSTs. https://juce.com/
* ELK Audio for hardware / audio. https://elk.audio/
* Find open source projects that meet your needs / desires.
It ccomes with many Xcode playgrounds for interactive exploration of the SDK.
This book teaches you DSP and also how to apply those DSP algorithms by employing them inside an audio plugin.
I started with Csound and Cabbage framework[1] which I used to create some VST plugins.
Then I moved to javascript lands (as everyone already has runtime called web browser) and I am creating sort of DAW using webaudio. I used Tone.js, but it was somewhat confusing, so I started my own library[2].
It's not finished yet but I have already nice and usable demo - comb organ playable with computer or MIDI keyboard with built-in "tape recorder"[3]. Good news is that is open source so you can actually stole some ideas from me.
[1]: https://cabbageaudio.com/
[2]: https://github.com/severak/cyber-music-studio/blob/main/js/h...
[3]: https://severak.github.io/cyber-music-studio/regenschori.htm...
Covered on HN late last year https://news.ycombinator.com/item?id=24940624 ("How to create minimal music with code in any programming language")
Also related to bytebeats: https://soundcloud.com/robertskmiles/bitshift-variations-in-...
I don't know if you're a musician yourself but I'd highly recommend downloading a DAW and playing around with the plugins to get a sense of common audio processing modules, you can perhaps make it a near-term goal to create an EQ or filter plugin that you can install and use in the DAW. IIRC I think Ableton Lite should have a free demo, Reaper is another well-regarded free DAW from what I've heard
Also, you can learn a ton by looking at various other higher level languages do it: Csound, SuperCollider, PureData are all open source.
Oli Larkin made a brilliant page of resources on github: https://github.com/olilarkin/awesome-musicdsp
In my head, I'm imagining a more programmatic approach to "arrangement of samples and loops," as in, instead of, or perhaps complementing, the classic "grid" or "buttons" view -- something like,e.g.:
- Specify 4/4 @ 95bpm for the piece
- Sample 1, a drum loop stretched/and or pitched
- Sample 2, a hit on the 2nd beat of each interval
- Sample 3, a simple electric piano melody, repeating over X intervals.
- Sample 4, a pad with an echo/reverb
etc. Now, what should be easy to do is to switch out, e.g. which drum loop, or change your piano to a trumpet. I get that MIDI is okay for this -- but is of course better for the latter situation and not great for effects and audio samples.
Your Abletons and so forth are not bad at this, of course, but - and maybe this what I'm describing -- Ableton's features -- but doable in vim, perhaps with the ability to create functions et al on the fly, which strikes me as a potentially powerful way to "group" musical sounds/ideas/intervals/melodies/chords/effects etc etc
I thought I might throw my hat in the ring of shameless self promotion since you might be interested:
I'm working on a project called Elementary Audio, which is a brand new javascript runtime and framework for writing native audio applications– meaning desktop apps, like perhaps the mini-synth you're hoping to make, as well as audio plugins (VST, AU, AAX, etc) and even embedded hardware, all in JavaScript.
The project is still in an early stage (and macOS only right now). I'm planning to share it more widely here on HN once it's ready for beta, but I thought this thread might be interested in it before then.
Website: https://www.elementary.audio/
And here's an example MIDI Synth: https://github.com/nick-thompson/elementary/blob/master/exam...
You could even spin up an Electron frontend to take input from your computer keyboard and send those over to Elementary to render the synth sounds. It's actually an example project I was considering doing myself but you might beat me to it at this point!
Lastly, I'll say briefly that there are a lot of tools out there (as we're seeing in these comments), different things work for different people, and there's no single "right" way to start audio programming– find the path that makes it fun for you, because it's such an awesome field to get into.
But if you're using an existing computer music programming environment like Csound (used in The Audio Programming Book), SuperCollider, or PureData, you can make a mini-synth which takes input from the computer keyboard without any of that stuff! (Stay away from Max/MSP; it's proprietary.)
The most fun I've had with audio programming has been bytebeat, though: http://canonical.org/~kragen/bytebeat. I didn't invent bytebeat but I did write that page about it.
Also, there are a lot of the hassles involved in audio latency on CTSS-derived OSes like MacOS, Linux (especially Android), or Microsoft Windows; it's easy to end up with 100+ ms of latency, and hard to do better than 10 ms, and either of those is a super shitty musician experience. If you do your stuff on an Arduino or similar microcontroller instead, those problems evaporate, and your computer music system will fit into an effects pedal. You need a DAC and, if you're taking audio input, an ADC; the built-in PWM and ADC in a standard AVR may not be sufficient. (And, beware, a 31.25 kHz PWM carrier can go right through a stereo system and burn out your venue's tweeters if it's turned up high enough. Filtering is required.)
Now that you've read a book, pick an extremely simple task and just do it. For example, maybe try writing a command-line program that will just play a square wave through the speakers. Then, build up your skills by making options for sawtooth, triangle, and sine waves. Finally, in the callback from the audio API, poll the keyboard to decide what tones to play. ;)
FWIW: A few years ago I wrote a functional keyboard and mouse synth program that just did sine, triangle, square waves and harmonics. It sounded a lot like an organ. (I later lost interest.) What helped was that I found a wrapper for MacOS CoreAudio that dramatically simplified the API so all I needed to do was provide a stream of floats. (I don't remember the name.)
Don't fall under the illusion that you can just read books and at the end you'll know how to do this. You'll need to work through simplified tasks to build your skills and understanding before you can write your mini-synth.
A book I thought was great has no code in it, but is FULL of very useful info about audio and computing it. Computer Music by Dodge and Jerse, ISBN 0028646827 (1997) ... a little pricey, might be hard to find, but very hard to beat.
[1] I haven't seen Richard's book (I'm sure it's great - and I will see it, thanks for the tip) ... he just might be pitching this idea in there!
You can absolutely make a nice synth that takes input from pretty much anything you can attach to a computer.
And as everyone else has said - theAudioProgrammer is good too.
(Edit) - Pure Data (PD) is an open source alternative to Max which is free. I've heard people recommend it but never used it myself.
Sound on sound has a series on synthesizer concepts. https://www.soundonsound.com/series/synth-secrets
Dsp guide is a great resource. http://dspguide.com/
Music dsp also has some good information and implementations. https://www.musicdsp.org/en/latest/index.html
Ross Bencina also has some good articles about the realtime aspect. Basically, you shouldn't use locks and should become intimately familiar with ring buffers :)
- Musimathics Vol I & II: This will teach you the basics of Music and DSP in a very approachable way. I wish I had these when I started writing audio software.
- The Art of VA Filter Design: https://www.native-instruments.com/fileadmin/ni_media/downlo...
If you are programming in C, these libraries are very useful:
- PortAudio: Cross platform audio playback
- libsndfile: Easy and cross platform sound file io
- ladspa: Really simple plugin API, useful if you want to make or use audio plugins without all the boilerplate of vst or lv2
I'm using the Unity game engine, and am leaning heavily on already-implemented FFT algorithms available on the Unity AudioSource component, but the fun part is taking the spectrum data, dividing it up into bands, and doing visuals with those.
Here is a short video showing a simple audio visualization demo I created: https://www.youtube.com/watch?v=kYt0dcUY3i8
It led me down a bit of a rabbit hole with watching videos about Fast Fourier Transforms, and learning more about audio specific programming.
What is your background, and which parts of audio programming would you like to focus on? e.g. are you interested in writing your own low-level DSP code, or more interested in hooking together higher-level blocks?
If you're from a web programming background, you can have a lot of fun with the WebAudio API [1]. It provides high level building blocks like oscillators, filters, effects, etc., which you can hook together. It has some rough edges but it's a great starting point.
Tone.js [2] is a great way to get started with WebAudio, it provides some useful abstractions that let you build fun things quickly, and there are some great examples of WebAudio creativity out there e.g. Blokdust [3]. If you want to go lower level, all major browsers now support the AudioWorklet API [4], which lets you write your own custom DSP algorithms and have them run efficiently within WebAudio.
However, WebAudio has limitations in terms of performance, and you can't use a WebAudio synth as a plugin in your music production software of choice (well, you might be able to get a browser which runs as a plugin, but it's not going to be ideal!).
If you want to write professional audio code, you can't really avoid working with C++ - it's the industry standard approach. This is largely because C++ allows programmers to very carefully design the performance characteristics of their code - specifically, you can avoid doing anything which might take an unknown amount of time (e.g. allocating new memory, locking) while processing on your audio thread - audio is realtime and very low latency, so you need to be sure your code is going to run in the amount of time you think it will.
Personally, I think JUCE [5] is a great starting point. It abstracts away some of the complexities of both C++ and audio APIs, allowing you to build cross platform audio apps and plugins without having to worry about platform-specific quirks. They have a pretty good selection of tutorials too [6]. An alternative you could look into is iPlug2 [7]. There are a couple of books by Will Pirkle on the topic of programming audio effects and synths with C++ - the author uses his own framework, but the same principles apply to other frameworks. Make sure you get the new editions if you do get these.
You'll probably want to take time to learn some C++ basics first if you don't already know it, it's a hard language to master! I'm actually working on an integration right now to allow you to build web UIs for JUCE apps, which takes away the complexity of also building the UI in C++ - I'll hopefully have something to share in the next few weeks.
If you want something a bit more straightforward and you're happy just supporting MacOS and iOS, AudioKit [8] is worth a look. It's kind of in between WebAudio and JUCE in terms of complexity I'd say - you write your code in Swift and it provides a load of high level building blocks, but you can drop down to a lower level if you need to write your own stuff in C++. Also the CoreAudio/CoreMIDI APIs on Apple platforms are very powerful, though I've struggled to find great documentation.
One other option you could look at are specialist audio languages like Supercollider [9] and Faust [10], or visual programming tools like Max [11] or Pure Data [12]. I'm not very well versed in any of these, but I've heard good things about Supercollider especially.
Phew, hope that helps! As you can probably tell, I'm really interested in this stuff so please give me a shout with any questions!
Edit: one more resource I remembered, there's a Youtube channel dedicated to learning audio programming, mostly using JUCE, called The Audio Programmer [13]. I haven't actually watched much content so can't say too much about that, but I know the guy behind it and he's great, and also runs a great community for people interested in audio programming [14].
[1] https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_A...
[4] https://developer.mozilla.org/en-US/docs/Web/API/AudioWorkle...
[5] https://juce.com
[6] https://juce.com/learn/tutorials
[7] https://github.com/iPlug2/iPlug2
[9] https://supercollider.github.io/
[13] https://www.youtube.com/channel/UCpKb02FsH4WH4X_2xhIoJ1A
Learn C++ Real-Time Audio Programming with Bela - https://hackaday.io/course/171240-learn-c-real-time-audio-pr.... The course does recommend buying the Bela development board from https://shop.bela.io/ but I don't think it's a hard requirement.
[0] - https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_A...
If you want a really gentle start, try Sassy: https://sol-hsa.itch.io/sassy
If you want this to play nicely musically you want low latency and I'd look for programming languages made for audio. Just watch a few introductions and pick what you like best.
More professionally I'm not experienced, but maybe writing a VST MIDI instrument would make sense.
If it's just about the sound maybe you should start at the DAW end and use Bitwigs Grid or Ableton (possibly with max/msp).
If you are a Ruby programmer on Linux, you could use this rubygem I wrote: https://github.com/mike-bourgeous/mb-sound
A video about using that gem to make a synthesizer: https://m.youtube.com/watch?v=aS43s6TWnIY&feature=youtu.be
Part of a long-running experiment of mine to make educational videos about sound, which I hope might help you on your audio programming journey: https://www.youtube.com/playlist?list=PLpRqC8LaADXnwve3e8gI2...
There have been recent posts to HN about the difficulty of reading key-up events from the terminal. I used MIDI and a separate MIDI keyboard app for my video demo.
It might be worth studying the sources of some working soft synths such as Helm, Surge or ZynAddSubFx or mixed firmware-hardware synths such as TSynth. The latter in particular is a low cost polyphonic external module built around a Teensy microcontroller, so the code base isn't that big, and it sounds great.
https://surge-synthesizer.github.io/
https://github.com/zynaddsubfx/zynaddsubfx
[0] https://sonic-pi.net/ [1] https://github.com/sonic-pi-net/sonic-pi
It's not as powerful as something like PureData but it does give you a simple and intuitive introduction to DSP and audio synthesis.
Tons of great code to read and projects to understand ..
Also, the Basic Synth book is a dream:
- https://news.ycombinator.com/item?id=27269589 - Elk OS – Audio Operating System
I generally learn best by having a real thing to build; and then finding the resources I need when I get stuck.
I buy books aspirationally (I have bought the Audio Programming Book as well come to think of it) - so I often end up with many books on subjects that interests me, without learning much from them.
If you don't want to do realtime synthesis, you can also dive in the (uncompressed) WAV file format and make programs that generate sound files.
Another thing that might be nice (but I personally haven't tried) is to synthesize your own drums, 808-style. Google 'kick synthesis' for example.
https://online.stanford.edu/courses/sohs-ymusic0001-audio-si...
See also this book about computational music synthesis:
Also, I second the suggestion to join the Audio Programmer community.