
Piano Genie: An Intelligent Musical Interface - mariuz
https://magenta.tensorflow.org/pianogenie
======
clvcooke
I was pretty excited to try this out (there is an online demo linked), however
after some playing around I found the experience lacking. Not that things
sounded _bad_ persay but that I found it incompatible with actually playing
anything that sounded nice.

I think part of my disatisfaction is due to pattern recognition. As humans we
recognize patterns EVERYWHERE, even when they don't exist. So what happens (or
what I believe might be happening) is that I think I recognize a pattern, try
to play a note to continue it, then the NN decides otherwise. Although the
resulting sound may be nice to an outsider it breaks my internal model, making
it sound "bad" to me.

I do however like the idea of augmenting insturments to make them easier to
play, I'm personally guitly of having many potential songs in my head but not
having near enough the skill to execute them.

~~~
fenomas
I think the dissatisfaction you're talking about arises necessarily from
projects that generate notes with a network trained on a bunch of MIDI data.
Our ears expect music to have structure (chord progressions, etc) but models
like this only really mimic small-scale features of the source. So any given
phrase of 3-5 notes can sound musical, but over the course of a measure or two
the illusion breaks and it sounds largely indistinguishable from a random walk
of notes.

~~~
organsnyder
So very similar to a Markov text generator, then.

Makes sense, really, since music can convey much more than is evident merely
by looking at the sequence of notes.

~~~
fenomas
That's the analogy I usually make, yeah - it's like generating a story with a
neural network trained on a bunch of stories. You'll get grammatical phrases,
and sometimes sentences, but naturally none of it will make any sense.

------
Cogito
This was fun to play with, but impossible to improvise on for anything other
than a relatively random sequence of notes.

While I could get a bit of a feel for how my actions would influence the
piano, there is still a huge disconnect in there. Even something like trying
to repeat a note can happen in three different ways (same key, key above, or
key below) and there is no indication of which way will work this time.

So improvisation is limited to

\- play high or play low

\- change tempo

\- learn what the machine will do in a situation and try to preempt it;
"previously when I pressed these buttons in this order it did a thing, maybe
if I do it again it will do a similar thing"

Improvisation isn't just randomly hitting notes, it's connecting different
pieces together in new ways. If I play some random notes and I like them, I
want to play them again.

Right now there is no way to craft music with this, just different ways to
poke the machine to see what kind of music gift it gives you. I want to craft
music with this, but I've no idea what that bridge would look like.

------
fredley
This suffers from the limitation of many generative music algorithms/tools in
that you're locked into a key (in this case C major - no black notes are
played at all), which makes it a cool toy and research project, but of no real
use to anyone.

Most of what makes good music good (or interesting music interesting if you'd
prefer) comes from harmonic interplay and shift. Bach was a master of this. If
you listen to the unaccompanied 'cello suites[1] then what makes the music so
fascinating is that even though only one note is being played at a time, the
harmonic landscape is shifting and changing (often in 'unexpected' ways) all
the time.

Doing this with algorithms and making it sound good is _hard_ , and nobody's
(afaik) done it well yet.

1:
[https://www.youtube.com/watch?v=rGgG-0lOJjk](https://www.youtube.com/watch?v=rGgG-0lOJjk)

~~~
IanCal
> (in this case C major - no black notes are played at all),

The demo is definitely not limited in this way, though it may be the case in
the physical version in the video.

------
elaus
It's a fun idea but it really threw me off when playing around with it.

It feels like the proverb 'too many cooks spoil the broth', i.e. both me and
the NN want to decide where the song is going and in the end no one is happy
with the result. I found it especially confusing when trying to play the same
note twice and it changed the note, or when changing the note and it played
the same note again.

But as I said it's an interesting experiment none the less!

------
baldfat
Olafur Arnold is a Neo-Classical Musican from Ice Land. He took this idea
(Minus the garage band) and made two pianos connected with a midi controller
which he hired a programmer to make the software for him.

I seriously think that more and more musicians will employee programmers for
musical reasons.

Here is his Tiny Desk Concert. Pretty amazing stuff.

[https://youtu.be/TpO_8tk6yNQ?t=235](https://youtu.be/TpO_8tk6yNQ?t=235)

~~~
organsnyder
I'm not grasping the significance of this. To my ear, it sounds like the
pianos are both playing basically an echo of the chord that he's sending from
the controller keyboard. Doesn't seem that much different (though I will
concede, much more aesthetically pleasing) from the auto-accompany features
that have been available on keyboards since at least the 1990s.

The MIDI interfaces on the pianos aren't at all novel, either: any piano can
be retrofitted with a PianoDisc system. Just about every pipe organ nowadays
also has MIDI capabilities.

~~~
baldfat
Well if you watch him play he isn't playing a simple chord that is controlling
two pianos. He actually has them randomly produce different sequences
periodically. He plays a chord and the two pianos have two different
arpeggiations going. If you look a little closer you will see the sequences
coming and going.

> The Icelandic multi-instrumentalist and composer's fourth solo LP finds him
> working with Stratus, a piece of computer software he programmed over two
> years in collaboration with Halldór Eldjárn. It uses MIDI values triggered
> by what Arnalds plays on his primary piano (equipped with a Moog Piano Bar)
> to generate different sequences on two player pianos. The process can create
> randomized feedback and lead to progressions that might be unplayable by
> conventional methods. Arnalds has described it as a way to change up his
> creative thinking: "You get ideas that you would never get otherwise, just
> because this inspired those ideas."
> [https://www.residentadvisor.net/news/41922](https://www.residentadvisor.net/news/41922)

> The MIDI interfaces on the pianos aren't at all novel, either: any piano can
> be retrofitted with a PianoDisc system. Just about every pipe organ nowadays
> also has MIDI capabilities.

Well if you listen the pianos they have a totally different sound and it is
due to the mechanical way they were designed. He has stated that he had spent
tens of thousands of dollars and countless hours to get the two pianos to
sound this particular way. This isn't some simple little trick that anyone
could pull off.

------
dpkonofa
This is really neat. I like that they mention Rock Band and Guitar Hero as
those are the first things I immediately thought of for this.

Serious question, though, that I hope will start some debate:

I know some people that would consider this to be a hinderance to music but I,
personally, think it's great that this could potentially be used to bring
people into music that normally would be too intimidated to consider it. Is
this still considered "playing music" since the person using it is still
making choices and decisions even if those choices are aided by the computer
algorithm?

~~~
Ace17
> Is this still considered "playing music" since the person using it is still
> making choices and decisions even if those choices are aided by the computer
> algorithm?

Think about hang drums, diatonic harmonicas, or about the Tenori-on. They make
it basically impossible to play a "wrong" note. But a normal piano was already
a step in this direction.

A piano restricts the pitches to discrete ones, making it harder to play a
"wrong" pitch. Violins, and to some extent, guitars, have continuous pitches.
And there are indeeded some pitch nuances that piano players can at best hope
to hear (e.g C# vs Db), and some pitch effects that a piano player can only
mimic (glissando, vibrato).

However, nobody today would argue that a piano player isn't really "playing
music".

> this could potentially be used to bring people into music that normally
> would be too intimidated to consider it

And this is where the real issue lies IMHO: playing or making music shouldn't
be "intimidating", but the way we teach it certainly doesn't help.

~~~
TheOtherHobbes
Pianos evolved from older instruments where each string had fixed pitch -
probably psalteries, which are related to harps - so there was no stepping
towards removing wrong notes. They were simply different instruments. In fact
the stepping went in the other direction, from a limited note selection in one
scale to a full chromatic scale that could play in any key, wrong notes
included.

Many things that humans do are intimidating - such as running a marathon,
climbing Everest, or inventing Smalltalk.

Removing challenges devalues the personal satisfaction of mastery. Making any
art simple makes it trivial and unexpressive.

So you can't say anything of interest or value with machine music that does
most of the musicking for you. You can make some sort-of-music-like noises,
but that's all.

~~~
Ace17
> Pianos evolved from older instruments where each string had fixed pitch

Interesting. But my point is actually unrelated to history. It's related to
the tradeoff between "easy to play" and "expressive power":

There are some steps you can take, as an instrumentist, in the "easy"
direction, while still keeping enough expressive power.

> Removing challenges devalues the personal satisfaction of mastery.

Surely. It's unclear, though, how instrumentist mastery correlates with
musicality:

Some virtuoso guitar solos can be considered "non-musical". To the other
extreme, some great composers don't use any musical instrument (from classical
music composers, to Amiga sequenced-music composers) - some of them can't even
play their own compositions properly!

> Making any art simple makes it trivial and unexpressive.

This reminds me of an old a running joke on french TV, starring a fake David
Guetta having a custom piano built, with only one key ( a reference to the
intro of "I Gotta Feeling" ). I think it's safe to say that you can't express
a lot with a such a piano :-)

However, a smart 10-key "piano" opens new horizons: because now it's
reasonnably possible to "play" multiple pianos at the same time. I can
certainly see new forms of mastery and non-trivial expressiveness here (e.g a
setup of 4 "smart keyboards", each of them using a different sound).

------
skybrian
I think this might work a bit better if it were trained on a bunch of popular
songs from a large fake book and tried to guess the one you're trying to play?

As it is, you're never going to get a melody or accompaniment out of it.

~~~
TimTheTinker
> fake book

Anyone ever seen an original real book? Asking for a friend :-)

~~~
aspyct
Can't tell about original real books, but the fake books definitely are real
:)

[https://en.wikipedia.org/wiki/Fake_book](https://en.wikipedia.org/wiki/Fake_book)

~~~
TimTheTinker
I didn't mean this as a joke. I'm talking about the "Real Book" that was
produced by Berklee CoM students in the early 1970s.

------
diydsp
I think it's fun and has a lot of potential.

I think they could be set up in public places where pianos are to get people a
little more into it.

Future versions could include co-creating the patch and store/recall in
memory.

------
tabtab
I sounded like a drunk Jazz musician. (Not that drunk Jazz players are
necessarily a bad thing.)

Is there an ABBA mode?

I actually once noticed that melodies of popular tunes tend to hover around
the chord keys. The duration and probability of a given note favors chord
keys. I built a Compose-A-Tron based on this pattern, and it mostly worked,
but probably wouldn't generate any #1 hits. (Although, it could be improved
with tweaking; but so could ordinary draft "on paper" compositions.)

