I've been going down this rabbit hole lately and it's really fascinating.
Workflow 1: feed a sawtooth oscillator through filters controllable with knobs. Eventually I realized that it will always sound mechanical, no matter how many filters you stack. That led to
Workflow 2: feed a sawtooth oscillator through a convolution reverb that uses a custom impulse response. For impulse responses, use random sounds downloaded from the internet (like wood strikes), or mixtures of existing instrument sounds. But that felt limiting, so I moved on to
Workflow 3: generate an impulse response wav file with Python, and use that in a convolution reverb to filter the sawtooth. This gave me some more interesting and configurable echoes, but then why start with an oscillator at all? So
Workflow 4: write code to generate the sound on the fly, as a sequence of samples. This way I can mimic some nice properties of physical sounds, like "the nth harmonic has initial amplitude 1/n and decay time 1/n". Also I can get inharmonicity, smearing of harmonics (like in PADsynth algorithm), and other nice things that are out of reach if you start with periodic oscillators.
If I could go back and give advice to my months-younger self, I'd tell me to skip oscillators and filters, and jump straight into generating the sound with code. You need to learn some math about how sound works, but then you'll be unstoppable. For example, here's a short formula I came up with a month ago for generating a pluck-like sound: https://jsfiddle.net/yd4nv5Ls/ It's much simpler than doing the same with prebuilt bricks.
The whole experience made me suspect that there's an alternative approach to building modular synths, based on physical facts about sound (as opposed to either starting with oscillators, or going all the way to digital modeling of strings and bows). It would be similar to physically based rendering in graphics: for example, it would enforce a physically correct relationship between how high a harmonic is and how long it rings, and maybe some other relationship about what happens to harmonics at the start of the sound, etc. But I'm not an expert and can't figure out fully how such a synth would work.
It seems to be easy to implement digitally, but it really isn't, because a lot of the nuances and non-linearities that add weight and colour to synthesis with real electronics aren't present in simple digital emulations.
For pure DSP the choice is more or less between open additive, modal (which is a kind of constrained additive), AI-constrained additive, which is what Google have been playing with, and physical modelling, which is digital modelling of strings and bows.
If you want to "enforce a physically correct relationship" between etc you're going to want AI-constrained additive or physical modelling.
The aesthetics of all of this are a different topic altogether.
I was thinking of relationships like these:
1) Decay time of nth partial falls as a certain formula of n.
2) Frequency of nth partial is slightly different from n * fundamental, by a factor which is a formula of n.
3) Spectrum of nth partial isn't a delta function, but a hump whose width is a formula of n.
All these ideas come from physical effects, but you can use them to generate sounds directly, without any physical modeling or AI. My hunch is that could be more such ideas and they could play together nicely.
Yamaha experimented with this in the 90s with their VL series physical modeling synths, but it never caught on, mostly I think because if you want to have convincing results, you really need alternative midi controllers like a breath controller for woodwind and brass intruments.
An alternative take on why physical modeling synths never really caught on is GigaSampler. It was the first sampler (as far as I can remember) that could playback samples from hard disk, by only keeping the first second or so of the samples in memory. This made it possible to have sampled instruments where for example each key of a piano was sampled at various velocity/loudness values. Resulting in a sampled piano that could span multiple gigabytes. At a time where 128MB of RAM was still quite a lot, this was quite revolutionary. While physical modeling can produce convincing sounds with a potential expressiveness that no sample based instrument will ever match, it's base sound still doesn't sound as 'real' as a properly sampled instrument, recorded in a nice room with good microphones.
 simple overview, including some soft synth alternatives: https://www.musicradar.com/news/tech/blast-from-the-past-yam...
 example breath controller: https://www.akaipro.com/ewi5000
 Review of Gigasampler's successor: https://www.soundonsound.com/reviews/tascam-gigastudio-4
(It helps that some of my favorite producers and composers have used Pianoteq - for me that's Guy Sigsworth and Thomas G:Son, but the Ludovico Einaudi endorsement really clinches it for me.)
I'd much rather be able to just plug MIDI into a plug-in to get say a saxophone line for a song than having to buy a top tier saxophone, learn to play it in a perfectly soundproofed room with great microphone, DAC, ect.
You could also ask someone who already knows how to play the sax to do it for you, and use a midi based sax sound until you have the score perfected as a stopgap.
They are also developing a completely new type of multidimensional embouchure sensing mouthpiece with which to play the instruments. It should be easy to learn but offer deep potential for expressiveness.
Also note that the goal of these instruments is not to faithfully emulate the timbre of any existing instrument (for that use a sampler) but to emulate dynamic behavior with is where the true expressiveness of wind instruments comes from.
Disclosure: I am developing the mouthpiece.
The famous Karplus–Strong algorithm used a burst of white noise as the excitation, but I've had more success using an asymetric triangle-shaped impulse that resembles the shape of the drawn string.
A couple of friends and I are experimenting with collaborating using Github and pull requests.
1. I was using VCV Rack on Windows, but I think my friend was not, so you may need to tweak Git's line ending settings
2. Some changes you just don't want to merge. The other person could have different audio output settings. There can also be rounding errors in the knob values so sometimes the values can be edited slightly even if they were 1.0 before, even if you don't think you changed them, which makes merges a little noisier. But `git add -p` made short work of cleaning that up.
Edit: This is also assuming that you coordinate who takes turns editing at one given time - "ping pong" is easier than free for all.
That said, I got myself a Moog Mother-32 recently, which is a hardware semi-modular synth, and I found it much more musical than software alternatives. I could immediately create much more musical pieces than I could with the software – even though I have MIDI peripherals for the software and the software can create any module I can imagine.
Another good option is SunVox; it's part modular synth, part tracker in a really cool UI. It includes a bunch of simple examples that demonstrate how to use the synth modules.
Back to the topic at hand. I've played with software gizmos, and recently picked up three secondhand units from the Korg Volca range. I'm not particularly musically talented, trying to change that over time, I find the physical controls way way way more intuitive.
I’ve been fooling around with music for about 20 years, but the path I took (trackers, which are sample based, then into VSTs with their huge preset libraries) plus a lack of self-discipline meant that I never really mastered making a sound from scratch on a synth. It’s only in the last few years since getting into hardware that I’ve started to learn this, and it has made such a difference to my music making - I can now often dial in the sound in my head without having to trawl through presets, or I can take a preset and either modify it to my taste, or work out what it’s doing and recreate a similar sound.
If I could go back in time and give myself some advice, it would be to pick up a basic hardware subtractive synth with knobs for each function on it and master making sounds on it from scratch. Something like the Korg Minilogue would be perfect. Alternatively, a good synth on an iPad or creating a good midi mapping for one VST synth and mastering it would do the trick, but I think there’s something about the hands on design of hardware, plus that you’ve invested money in it, that makes it the ideal learning platform (and a lot of fun!). The skills you learn doing this are transferable to any other synth and other areas of music making.
A good comparison would be some visual programming environments and modular buzz-machine type DAW programms, that both look very similar, employing the graph metaphor--nets of generators and filters. I'm not sure wgether that's comparable to functional reactive programming, that I have never tried.
Writing a synth in C would be very different, at any rate, I guess, something I don't even know how to go about in an ergonomic fashion.
If you want to write your own plugins or audio software you’ll need to learn C++ at some point, which I think is a fairly complex language to start with and likely to be frustrating in terms of getting quick results, but your mileage may vary!
A colleague of mine learned C++ as his first “real” language (which is now his job) because he started out playing around in SuperCollider (a DSL for computer music stuff) and wanted to turn his ideas into real plugins, but it can get pretty complex pretty quickly so I’d say you’d have to be pretty motivated to do this!
If you do want to go down this route I would probably start with the aforementioned SuperCollider or something similar (maybe Max/MSP) to get an idea of how the musical side of things works without having to master C++ at the same time, then when you feel confident you can start learning C++ - personally I’d recommend the JUCE framework as it is designed for audio apps, hides some of the complexity/gotchas of C++, and has some good tutorials for beginners: http://juce.com.
DSP stuff can get pretty maths heavy and I’ve not come across anything equivalent to the building blocks supplied by WebAudio/AudioKit for C++, although JUCE does now have some DSP modules such as filters which you can easily use, and there is sample code online... but you’ll probably have to get your hands dirty at some point :) Will Pirkle’s book on audio plugins provides quite a good intro to DSP but the code in the book is, to be honest, pretty outdated/bad style - he is apparently working on an updated version, but if you are able to take the code with a pinch of salt (e.g. write the examples yourself in JUCE rather than using his RackAFX framework) you might find it useful.
Feel free to reach out to me for more advice, my background is in web development but I’ve been working on audio stuff professionally for the last couple of years so have been through the learning process of C++ etc and would be happy to help if I can!
I’d check out that tutorial I posted, once you feel comfortable with the basics of JS you may also want to take a look at https://tonejs.github.io/, which provides a layer on top of WebAudio with useful functionality like synths, sequencers etc. Can save a lot of time!
There are many other great tutorials and open source WebAudio projects out there too. Have fun, hopefully you can post something you have made on here in the not too distant future! Like I say, feel free to give me a shout on email/twitter if you need more advice.
Now 20 years old, still online, this is pretty much a complete course in theory and practice.
People who liked Brian Eno's twitter feed also liked...
Have you seen Reactable? http://reactable.com/
I wish there were one like that. If only we had some companies with big silos of our data, ML tech and server capacities.
I'd suggest getting a simple synth (such as the ES M shown in the second image, or simple free VST synths if you're on windows, such as the PG-8X ) and spending some time playing around with the controls, and getting to know the sounds. There are usually 'Oh, THAT'S what that sound is' moments when playing around with the filters for the first time.
Once you're comfortable with that, move onto more complex synths (such as the Superwave P8) and you'll find that many of them are not really more complex, but just have more of the sections you already know - a bit like learning a channel on a mixer, and then moving to a 72-channel mixer from an 8 channel one.
Modular synths such as VCV rack (mentioned elsewhere here) are really great for experimentation with the architecture of subtractive sound generation (as mentioned elsewhere in this thread), but for many they are intimidating initially, and have setup time cost if you're starting from scratch. I think they are for a certain kind of personality (myself included!), but not for everyone. The reason that the 'standard' architecture which is described in the OP exists is because people habitually used the same setup (osc > filter > amp with lfo and envelopes, as seen to a degree in the 'olympic rings' analogy), and manufacturers wanted to produce simpler, cheaper synths for a wider market. Often you end up making variations on that theme. The great advantage of software modular synths is that you can save (and load!) your setups, and you never run out of cables or modules (or money!).
 - https://sites.google.com/site/mlvst0/
 - http://www.superwavesynths.com/p8
I'm not sure if I need to approach it by trial and error, or there are a few tricks to program it!
I think ML and GANs might be interesting to apply to FM patch design.
I guess the closest software equivalent I’ve seen is Ableton’s Operator but I find the Digitone much more fun, not least because it’s hands-on hardware which encourages exploration!
There is another way to let the machines do the work: interactive evolution. Brian Eno thought of this in 1995 (among others): the system presents you with N candidate patches, you select the ones you like, and it uses your selections to think of N new patches to offer you. Basically it's assisting you in moving forward through the space of patches in search of interesting stuff without having to program them.
Interactive evolution works really well for FM, but systems which provide it are rare. And now for the self-promotion. As it so happens, I just presented a paper on this a few days ago on my system.
EDIT: Great work, adding your paper to my stack of read-todo's...
And also, the tolerances in Synthesizers are actually fairly small and idiomatic synths use quite a few relatively obscure parts i.e. How many introductory electronics books discuss OTAs in any detail( for a slightly terrible example)?
I already know, however it's not easy to find out in one resource: The application for music is sufficiently obscure (Analog synths require much more coaxing than (say) a guitar amp) to warrant dedicated discussion.
You are looking for an "introductory electronic book" that discusses the types of circuits that are used in analog synthesizers. Further, those discussions should be accessible (understandable) to someone with little or no prior understanding of linear circuit theory.
Is that a correct reading of the thing you are seeking? If so then I would start with something like the Sam's OpAmp circuits book. If you aren't put off by mathematics, and your original message suggested you were okay with that, then "The Art of Electronics" (Horowitz and Hill) the first four chapters cover pretty much all of the information you need to know to read any of the schematics on the Moog schematics web site. Both books discuss filters, VCOs and VCAs, and transconductance as well.
As for precision, typically analog synthesizers are not nearly as precise as you might imagine. Like many instruments they were made to have a quality sound which may or may not be strictly accurate in terms of musical representation. One of the nice things about the Moog Model 15 was that you could tune it to different types of scales. You do want thermally stable circuits so that you aren't re-tuning all the time, but setting up in the studio I would typically use anywhere from 5 to 20 minutes with the 'high C' (1046 Hz) signal reference to tune in the various oscillators and amplifiers to get a nice 0 dB signal level at the final output and with the half dozen or so oscillators tuned to match frequencies. Not at all like a "modern" keyboard where you turn it on and blam! you're ready to play.
If I am still misunderstanding what you're asking I would like to understand that. You wrote "The application for music is sufficiently obscure ..." which sounds like you are looking for a specific tie into music in general. However the tie into music is, for the most part, entirely incidental to the mechanics of how these things are built so typically references cover the fundamental properties of these circuits without calling out their musical application which is seems to me to be fairly obvious once you know the fundamentals.