
Procedurally generated electronic music - stevehiehn
http://signalsandsorcery.com/
======
eltoozero
Raymond Scott[0] (1908-1984) comes to mind.

He designed rhythm generators and automatic baseline sequencers. He died
trying to complete building a monster electronic instrument called the
"Electronium" of which only one incomplete and partially functional example
exists, owned by Mark Mothersbaugh.

I only wish his devices were documented or understood such that simulators or
reproductions could be built.

[0]:
[https://en.m.wikipedia.org/wiki/Raymond_Scott](https://en.m.wikipedia.org/wiki/Raymond_Scott)

[1]: [https://youtu.be/0V2TZKcWnXE](https://youtu.be/0V2TZKcWnXE)

[2]: [https://youtu.be/o6VsZiNjjZE](https://youtu.be/o6VsZiNjjZE)

~~~
stevehiehn
Wow, that is awesome. I never heard of that guy.

~~~
uhoh-itsmaciek
Scott was great. The Manhattan Research, Inc. compilation chronicles his
electronic music work, and while a lot of it is outtakes and commercial
jingles, it's fascinating.

Fun fact: Scott also had a jazz combo and, among other things, recorded
Powerhouse [1], which became a cartoon staple.

[1]:
[https://www.youtube.com/watch?v=qaC0vNLdLvY](https://www.youtube.com/watch?v=qaC0vNLdLvY)

------
bandrami
It's interesting: some of the earliest European polyphony could be described
as "procedurally generated"; the cantus firmus was assigned liturgically, and
the descant could be determined by what we would today call an algorithm
(sacred music was at this point considered science rather than a creative
art). Plus ça change...

~~~
ThomPete
Agree.

Music is by and large about structure. What made it worth listening to was
this structure combined with human imperfection in the details and once in a
blue moon melodies and harmonies that sound like they are there without being
actually played.

Of known composers perhaps Bach was at the time the most mathematical of them.

Within the next decade I believe algos will be able to write hit's. Already
today a lot of the things we like to hear is really using technology to create
soundscapes and hooks we would never be able to create as humans.

~~~
stevehiehn
My bet is we will see DAWs that essentially offer composers shortlisted
options to choose from. In my SignalsAndSorcery project I am really just using
machine learning to find composition options that make sense in the give
context (based on my data set). A DAW could just keep presenting a series of
options to choose from and almost anyone could be a composer.

~~~
FigBug
I'm currently working on a DAW, and this is definitely the direction we'd like
to head. In our previous version, we added some basic chord progressions. The
next most likely chords are suggested on statistics from popular music. But
I'm stuck on where to take it next. I have some ideas for a melody generator,
but I don't really know anything about machine learning, etc. When you are
ready to implement you API, I'd be very interested in talking to you. Send me
an email at roland@tracktion.com

~~~
bandrami
I'm interested in this; mind if I tag in too?

------
bane
Here's a winning demoscene production, 256 bytes, on a Commodore 64.

[https://www.youtube.com/watch?v=sWblpsLZ-O8](https://www.youtube.com/watch?v=sWblpsLZ-O8)

Technical discussion here: [https://linusakesson.net/scene/a-mind-is-
born/](https://linusakesson.net/scene/a-mind-is-born/)

~~~
stevehiehn
It blows my mind to think people could do anything with that little RAM. I'm
the author of this project & it chews up about 32gb of ram to pump out one
track.

~~~
bane
Well, yours _is_ using significantly more advanced music sources and makes
much more music.

------
trypt
Academia is significantly behind the times in this area

Artists like Autechre and related links in this thread have been doing this
for years using Max/MSP, to a much greater effect (albeit it's apparently a
hybrid where the humans are at the "control panel" of the generative engine
guiding it in the direction they want)

[https://www.youtube.com/watch?v=wdKIJHXzPkk](https://www.youtube.com/watch?v=wdKIJHXzPkk)

------
mlaretallack
Years ago I remember listening to algomusic on the Amiga. Every so often I do
a search to see if anyone has made a version that works on Linux, but with no
luck. This is getting there, but still feels a bit rough around the edges.

~~~
mlaretallack
A quick search found the the author of AlgoMusic has moved on to a new project

[https://www.soundhelix.com](https://www.soundhelix.com)

Some of the examples sound even better. That's my evening sorted.

~~~
stevehiehn
The soundhelix examples sound really good.

------
kevinr
Oh this is very cool. The structure reminds me of Matthew Brown's "Music for
Shuffle" project:
[http://musicforshuffle.com/2014/04/06/session1/](http://musicforshuffle.com/2014/04/06/session1/)

------
indubitable
Who owns the copyright on procedurally generated music?

Mostly curious since some of these sound better than the ambient tracks you
get in games. Even better, you could constrain the parameters of the
generation to have each instance of a game generate different, but
thematically similar, procedural music.

~~~
FigBug
> 503.03(a) Works-not originated by a human author.

> In order to be entitled to copyright registration, a work must be the
> product of human authorship. Works produced by mechanical processes or
> random selection without any contribution by a human author are not
> registrable.

Possibly nobody, but it depends how much input the program requires to
generate the output. Just press a button and a song comes out -- can't be
copyrighted. User has to enter a lot of parameters to generate output, user
gets get copyright.

~~~
WalterGR
_Just press a button and a song comes out -- can 't be copyrighted._

I wonder: what would it take for a machine-generated work to be considered a
work derived from a human-generated work? While the machine-generated work
still wouldn't be copyrightable[0], would that prevent others from using the
machine-generated work without an appropriate license?

(IANAL.)

[0] «To be copyrightable, a derivative work must incorporate some or all of a
preexisting “work” and add new original copyrightable authorship to that
work.»
[https://www.copyright.gov/circs/circ14.pdf](https://www.copyright.gov/circs/circ14.pdf)
[PDF]

------
p0nce
This was actually much better than I expected. Wondering what kind of neural
net can do this, and what are the outputs. It sounds a bit "phasey" as if
there is an IFFT step.

~~~
stevehiehn
So its not just a single neural net. Its a bunch of different ML algorithms
chained together. I'm essentially automating a 12(ish) piece electronic band.
So first I have a ML algorithm that makes a chord progression. Then another
that makes a bassline from the chords. Then another that makes a beat from the
bassline & the chords and so on. At each step it uses the selected patterns to
predict the next layer to be added. So to answer the question directly you can
use RNN for some timeseries stuff. To decide if rhythm pattern are compatible
I found success with CNN, random forests, and support vector machines. I even
found you can make reasonable chord progressions just using N-grams. What I'm
trying to say is I found I had to make an entire audio production pipeline
that uses ML at each step of the way.

~~~
p1esk
Are you planning to describe all that in more detail? I skimmed the 4 blog
posts you published, but it's still not clear to me how the music is being
generated.

Also, why did you decide to separate the "layers"? Technically, an RNN trained
on similar music should be able to generate all "layers" simultaneously.

~~~
stevehiehn
I will hopefully add more detail eventually. When I say layer I am taking
about a music layer like a melody not a layer of neurons. Most of the music is
midi rendered with vst synths or sound fonts.

~~~
p1esk
Yes, that was what I meant when I said "layers" as well. RNN can generate all
layers at once (more than one note per time step for a single instrument, and
more than one instrument per time step overall). Basically what I'm saying is
if you train it on such complex music, it will learn to generate it.

~~~
stevehiehn
Ah!, I think I understand what you are saying. I'm not sure if you used a NN
before but you will notice that there is an exponential relationship with the
number of inputs and the size of the training set required. Because I don't
have millions of perfect training data I need to break down the problem and
make ML models to deal with specific problems. For example predicting if a
drum beat is compatible with a bassline etc.

~~~
p1esk
What exponential relationship? Can you point me to any papers about it?

It's actually quite easy to generate polyphonic (classical style) music with
an RNN: just predict the entire note frequency range of one time step as a
binary encoded vector. But yes, I can see how for multiple instruments, the
sparsity of inputs might require more training samples.

I'm wondering if the "layered" approach has some advantages, perhaps more
control over some aspects of the music?

~~~
stevehiehn
Ya, there is no official relationship. From my experiments the more inputs you
have the larger the dataset required. I do not know what is best way to do
this but the layered approach is how humans compose music so it made sense to
try that. I'm not sure I agree that its easy to generate polyphonic music. I
would like to be wrong but when you say "easy" I don't think you're referring
to music that follows a harmonic structure or factors in timbre. Most music I
hear follows structure with repeating themes etc. Not just a stream of
conscious series of notes.

~~~
p1esk
Well, I just tried googling "rnn generated music":
[https://youtu.be/A2gyidoFsoI?t=1m44s](https://youtu.be/A2gyidoFsoI?t=1m44s)
Sounds like a pretty good polyphony, but of course, it could be just
overfitting on Bach training samples.

Here are a couple of more sophisticated (and original) pieces:
[https://soundcloud.com/user-95265362/op-1-for-piano-solo-
in-...](https://soundcloud.com/user-95265362/op-1-for-piano-solo-in-d-major)
[https://soundcloud.com/user-95265362/op-21-for-piano-solo-
ge...](https://soundcloud.com/user-95265362/op-21-for-piano-solo-genesis)

~~~
stevehiehn
Those do sound amazing! My goal is to create interactive music. If you noticed
in my UI it can change at any point. In order to do that I think I need to
premeditate the harmonic structure. i.e chord progression. I suspect that
those RNN are not conforming to a predetermined harmonic structure. I doubt if
you ask it say generate a piece following Dmaj9/Amaj/Bmin/Bmin it will
understand that but I dont know.

~~~
p1esk
_if you ask it say generate a piece following Dmaj9 /Amaj/Bmin/Bmin it will
understand that_

Dmaj9/Amaj/Bmin/Bmin will be the seed sequence to the RNN. Of course, the
network might ignore the seed, or if it's overfitting it will produce a chunk
of the training sample which has this sequence in it, but you can definitely
specify the starting point (and you usually do with RNNs).

------
delgaudm
What do the icons mean? I can't figure it out.

~~~
stevehiehn
(project author here) Ya, sorry, its not clear at all. About 6 months ago I
got the system working sorta ok, but i found it a bit boring. So I thought
maybe if the algorithm generates N interchangeable variations associated with
an emotion I could maybe make the music interactive. So the icons are going to
represent a different emotion that a user can switch to at any point.
Obviously its not really work very well. This site is ultimately a tool for me
to test switching between variations. The end goal is not a website but to
expose a music generating api that could be integrated into say, VR/game or
something that might want interactive music.

------
eeZah7Ux
Procedural music is really hard to get right.

------
aswanson
The breaks are unpredictable, kinda dope.

------
NiklasMort
jesus christ, BEWARE ITS LOUD

