

"What you say" – A robot and human musical performance [video] - jibjub
https://www.youtube.com/watch?v=O-bjTfYILPs&feature=youtu.be

======
tunesmith
Pretty cool!!! Hard to tell what is improvised and what is pre-planned.
There's a part later on where the human keyboardist and the robots are playing
a new melody line in unison, so that implies that the melody, rhythm, and
placement (form) of that part of the piece was completely pre-composed. Maybe
this is more like an in/out jazz piece where there's a set form of sections
where each section is a combination of pre-planned and improvised music.

~~~
mrobo
Hi! This is my work and I just saw that somebody posted my performance here so
I created an account to answer your question.

You are right that some parts are completely pre-composed. The bass lines and
sections where all the robots and me play in sync are composed. Most
everything else is generated algorithmically. The robots have a computational
knowledge of jazz and improvisation so they fill in the spaces between the
pre-composed sections with their own music. The smaller Shimi robots also
analyze the music and figure out how best to move so that their movements (or
dances) correlate with the music.

Unlike Pat Metheny's Orchestrion and other musical robots (honda) I focus on
the artificial intelligence and getting machines to generate their own music
instead of just being controlled by a person.

The other thing about my improvisation algorithms is that they require the
robot to have a self-awareness of its physical self meaning that it
understands its physical abilities and limitations. This allows it to make
musical decisions that are directly influenced by its physical function and
design. If another robot with different physical constraints were to use the
exact same note generating algorithms the outcome would be different.

~~~
tunesmith
That's really cool! Thanks for the response!

------
prawn
I googled and Shimon may be a DJ? Or is it software coincidentally with the
same name? Wasn't quite sure if it was just the robots dancing or something
more impressive and ad-lib going on.

~~~
mrobo
Shimon is the name of the bigger marimba playing robot and the Shimis are the
smaller robots. The smaller robots do indeed autonomously generate dance moves
to the music, but they are also improvising. Some parts are precomposed (bass
lines, drum parts, chord changes), but the lead robots during each section
play music that is machine generated.

------
gtani
Pat Metheny and Orchestrion:
[https://www.youtube.com/watch?v=faendyU73Wc](https://www.youtube.com/watch?v=faendyU73Wc)

