I like the concept and the framing of the demo is really cool, but the actual music generated in response to me doing basic things like playing a blues riff or a pentatonic scale was usually just a repeat of some fragment of what I was playing. It felt like something that could be built using a basic heuristic...
Yeah either the examples from the video are cherry-picked, or something's wrong with the deployment, because what I get is way worse than what's in the video.
I really don't like this new trend of throwing Deep Learning at anything without really checking whether a simple model (e.g. markov chains in this case) would give equal results. Because using a simple model wouldn't be "cool" or newsworthy...
Here's a relevant excerpt from research I did on college into this. FWIW, my research was specifically into genetic algorithms and generating similar musical styles as the input music. It didn't perform particularly well but had some surprises.
From my research, a Markov model could potentially be viable for this demo in particular. However, it wouldn't be viable for longer generation of music where you're trying to produce an actual song. The biggest problem being that it doesn't have the concept of melody in it.
--- research after here ---
A Markov model utilized by Herfort and Rehberger [17] uses the immediate previous note in order to determine the next note against a set of probabilities. This is a fairly simple algorithm as the set of rules is not large, but it is extremely short-sighted by nature. This “first order” Markov chain is not capable of determining a probability based on a large sequence of pitches. To counteract this, “N-order” Markov chain models can be determined to look more deeply into the past, but the complexity of the rule set increases exponentially with added depth. Further, every aspect of music that is to be modeled would require its own Markov model. For instance, there would need to be a model for pitch, duration, etc. This leads to a potentially huge Markov model rule set where all chains are independent of each other. Thus, Markov chains are good at composing musical melodies on the micro level (such as a measure) but fail to produce quality music at the macro level.
Just to complete your post, one problem I always have with music generation using "first-order" Markov Chains is that they get really confused by children's music. For example, here's the start of "twinkle twinkle, little star":
CCGG AAG- FFEE DDC-
So the tranistion matrix contails rules like:
P(C -> C) = 50%
P(A -> A) = 50%
P(F -> F) = 50%
P(E -> E) = 50%
Which makes huge loops of the same note quite common. This makes for some really uncanny music.
When you've described the entire new vision of your company as being "doing things with deep learning", you aren't going to bother seeing if the simpler solution works. Just isn't good for business.
Agreed, it seems to either play a note you've played or pick a random interval. The interval and timing perhaps aren't truly random but it makes no difference to the listener's ear. Doesn't appear to have any concept of, or the ability to learn, scales, rhythm, or melody (or chords). Neat idea, but akin to playing piano with a 5-year-old (perhaps not as fun).
even going a simple jingle bells loop (I know, I haven't studied much) get's a pretty much nonsensical repeat response.
on a side note I'd like to have an option to put ai on mute and have the duet run after a session, since it's extremely disorienting for a noob to play with random noise on the feedback loop
I don't think there's a practical limitation on training resources with respect to Music. Maybe the authors didn't get into any of the rubric, but it's certainly out there. Gobs of it.
AI: long pause; holds down one random key for several seconds; boops another random key "Nailed it."
After spending a few minutes playing, I did see evidence of it occasionally tracking my chords (as long as they were very common - e.g. c major, a minor) and tempo (but only matching tempo, as in vaguely matching how fast I was playing, not actually playing in time with me). I played an extended chord sequence and it made one or two nice contributions, but mostly it was just random noise.
I wonder if this is due to lack of training data? Maybe it needs an actual data set of people specifically trading musical phrases with each other, rather than just a set of general music.
It's an amazing idea, and I can actually see it being quite interesting for things like musical phrase creation - supply the first half of a phrase and see what the machine comes up with for the second half, and tweak as appropriate. But currently it's a bit of a case of over promising and under delivering.
Definitely a work in progress. Was like having a goddamn cat following my fingers around the piano. No sense of rhythm or tonality whatsoever. Like, it objectively made whatever I was playing sound worse.
It may "respond" but it's a lot more like throwing electricity through a dead body than working with a competent duet partner.
FWIW, it's not really meant to be playing alongside you. The idea is for you to play a short tune, then wait for the AI to respond with a tune of its own based off of yours.
Well, FWIW as well I tried exactly that as well and it was like it could barely follow what I did (rhythm was off) and then it went in other directions that were not pleasing to the ear. Nor made much of any sense musically. So, again, it's like a cat or child was mimicking my motions just enough to make me dislike the result.
This is fun. It actually feels like I'm taking turns with someone else on the other end of the screen who takes what I play and sends back a riff on the notes. The trick is (as the little intro shows) to only play for a few seconds, don't play simultaneous notes, and then stop and let the AI respond. Once the AI has finished, you can try again. If the AI starts playing before you're done, you've screwed up and should make your little sequence of notes shorter. Basically, the title of this is misleading... if you keep playing expecting the AI to join you in a simultaneous duet, it'll sound like cats fighting inside a piano.
Try playing the "Dukes of Hazard" horn: on your keyboard, press (without overlapping the keypresses) "LJGGGHJKLLLJ" and let the AI respond. More often than not, it'll come back with a vaguely pleasing riff on that sequence using mostly the same notes and in the same tempo.
My immediate reaction was to try and confuse it, so I played "Roll Call" from the Thomas show (kids are obsessed !) It's a bit chromatic so I thought it would have trouble, which it did.
Next, I tried putzing around on the white keys but made sure to never hit a F or F#. I wanted to see if it would try and guess which key I was playing in, C or G major. It seemed to guess C most of the time, though one time I tried hinting at G by starting out on G, it got confused and starting playing both F and F#.
I was just clicking with a mouse, so rhythm was hard to judge but overall this thing did a really bad job "complimenting" my playing!
Are these standard keys for some other piece of music software?
I keep getting confused because I'm so used to the old-style tracker keys (I keep hitting J for Bb and getting B, which isn't helping my C minor improv). Also one octave isn't great, lag is killing the rhythm, and it keeps not waiting for me to finish.
Neat idea though.
(I'm curious to know whether the training set for the version shown in the demo is the same as for the live web version, as the demo sounded way better than anything I'm getting out of it - though obviously they would have cherrypicked the results for the video.)
I played the A-section for Heart and Soul (the song from the movie "Big") and its response was enough to tell me it was not going to be playing actual duets.
I'd be interested in a version that has learned accompanying portions to many popular melodies. If I could play the right hand part of a piece and it played the left hand I'd be very impressed -- also this would actually be useful software for aspiring pianists.
just tried this on my midi keyboard( web midi is awesome!) but the "AI music" response was very underwhelming and sounded bad.
I would be interested if "AI" can help me with different timbres based on the physical environment that music is playing using various sensor like air pressure, humidity, room resonance, maybe even heartbeat, perspiration ect. Make that a new dimension of information coded into the song along with pitch, timing, amp and timbre.
I was excited to play with this, but ultimately left disappointed. I was hoping that my initial melodies would be expanded upon and that I could build a series of riffs that would become increasingly complex but based on the same starting melody, but in the end, the responses from the AI seemed to be fairly random, and didn't seem to be built upon the previous sequences at all. The AI also changed keys a couple of times, which was disconcerting.
To whoever implemented this AI: Can you actually play music yourself? Your AI can't. I tried to play “Satori Maiden” (link: https://www.youtube.com/watch?v=0c9bea_thaM ), and your AI only replied with unmusical nonsense.
The problem is that this experiment might as well just replay a random collection of the N most recent notes played, with durations shuffled around, for all the musicality it demonstrates. He wired a neural network to some midi samples, and it didn't generate anything of use. Yeah, it was an experiment, and there's value in failed experiments, but you don't typically make a big website to present your failed experiment.
Maybe if you could set it to "Blues" or something like that, switching training sets, it would do better, but as is it's a nonsense generator.
There's nothing wrong with experiments that eventually fail, but there's something wrong with carrying out an experiment that you can tell in advance can't possibly succeed.
I'm opposed to this trend of trying to replace actual understanding of the problem at hand with massive amounts of computation. If you want to write a program that writes music, then learn how music is actually written first.
---
@hellofunk: If you want the ML algorithm to actually learn how to write music, at the very least, the programmer must be able to express the goal of producing musical sequences of notes.
I thought the idea was to apply ML in the least intelligent possible way and then say "Well, at least we used ML, and that's cool, right?" when the results are garbage.
I agree with catnaroek - this is a bad example of human learning, never mind a good example of ML.
Why would I? While I use computers to play music all the time, I don't think using computers to write music is a good idea in the first place.
---
@jgalloway___: I'm not sure if I'd call it an “expectation”. I just dislike it when people throw random shit at the wall and see what sticks. When you write music, you should first think how it's going to sound, convince yourself that it's plausibly good-sounding, and only then actually play it just to make sure it sounds good. If you do it the other way around (first play random stuff, then filter out what doesn't sounds good), you're a wanker.
---
@ohitsdom: I'm totally fine with improvisation driven by ideas, not random computation. Coming up with ideas on the fly is good. Passing noise as genuine ideas is not good.
That's fine that it's not your style, but experimentation is a huge source of new music and reinvented ideas. Dismissing any music that wasn't strictly planned seems a little silly.
Look, it couldn't even follow a basic tempo when I was playing. That's not even close to being a "duet" as described. Maybe whoever wrote it has a limited grasp of English, because that experiment is a far, far, far, far, far, far, far cry from a "Duet" experience.
Well, one where you don't look over at your partner and wonder if the have any clue how music works before pushing them off the piano bench.
For those looking for harmony try letting the machine lead .. if you mimic it's style and complement with a hint of your own rhythm it can be satisfying.
This is cool! I've been trying to use ML to automate the entire production pipeline. Choose chords, synths, loops, mix and add FX. http://signalsandsorcery.com/
Im finding extracting audio features from loops the hardest part. The selection of notes is actually not as difficult as choosing a good timbre for the notes.
It does seem like an interesting edge to explore doesn't it? ML techniques combined with spectral analysis could yield some interesting outcomes. The datasets wouldn't be too hard to find either - most people have pretty extensive digital music libraries.
I know this is meant to be a "cool experiment", but from trying this out for a while my impression is:
I could just build a random sequence generator and tell the world "I made this with tensorflow" and people would believe. I mean, none of the response sounded like music. Which is fine, but then again, what is the machine learning for, if it just sounds like random keystrokes?
I think that this response is confused. What is your point? That people can't differentiate random notes from a duet? That people would believe that anything associated with tensorflow was imbued with sophisticated behavior?
How does that relate to this experiment? It's clearly not just playing random notes in the example video...
I played with this for a while and never got the impression that it was random notes. A few times it played something quite musical which I found interesting and cool. There seems to be a lot of haters in here for what I can only imagine is some sort of sour grapes jealousy or some shit.. maybe it's the production quality of this video is out of proportion for the effort put into the experiment, I dunno but the experiment itself is cool.
Like MattRix said, I was also extremely excited about this since I like to improvise on my piano. But it was simply underwhelming and all I did was express that. I didn't say the project itself was stupid. I love the idea itself, that's why I was excited at first.
Also I was throwing a legitimate question. If you can't tell the difference between some random note generator and a deep-learning AI generated notes, what is the point?
I don't know why I'm being called an asshole for asking a question with a counter-argument. I remember a couple of months ago a video went viral which was basically a short film based on a script written by a Tensorflow-based AI. I also loved that idea for the novelty, but had the same feeling as this one--I could just random-generate utterances and call it a day and people will have no idea.
Basically my point is, there are a lot of narrow-domain use cases where deep learning is awesome, but in many cases it's not on par with human ability... yet.
p.s.
Maybe I have worse sense of music than you, but i have tried all kinds of variations. I would try dorian, pentatonic, blues, etc. and the only time where it kind of sounded sane was when you played easily recognizable notes like major scale (which beats the purpose of this whole AI thing). Of course one could call this "avant garde music" but like I said, I can write a random notes generator and call it that too.
We're not "haters". Why is it that if someone disagrees then they must have some ulterior motive? We just think the music sounds bad! That's it!
I wouldn't show this to anyone as an example of the power of AI because it's incredibly underwhelming. It's especially sad in these days where it seems like AI is pushing boundaries all the time.
So in other words, we're not haters, we're just being honest about how disappointed we are.
I've had the most success in my own experiments using many dedicated Models. The first and most important model is a NN that generates a chord progression. After that is much easier to select good notes!
http://signalsandsorcery.com/
I have tried just using N-grams which is pretty good but it doesn't have any structure. My next experiment will be to use the 'word2vec' algorithm the same way it it used for NLP. Meaning first create relationships between chords then create sequences with RNN.
As an experiment I think this is well done. It's fun to mash the keys and see what happens. If there was a feedback loop to register if the user was happy or displeased with the results via facial recognition or other input with expanded training data it could become quite interesting I think.
I'm getting some good results but I'm also a clasically trained musician so if you haven't studied music it may not give you a pleasing result but if you are a trained musician you may get more mileage.
I think it'd sound more interesting if the computer compliments what I play. For example, if I play in C key, the computer tries to guess the chord progression and play the rhythm that matches what I'm playing.
