
Rethinking Design Tools in the Age of Machine Learning - mayava
https://medium.com/artists-and-machine-intelligence/rethinking-design-tools-in-the-age-of-machine-learning-369f3f07ab6c
======
pavlov
This is an area where the vision is very desirable, but user expectations also
run sky-high.

A company called The Grid collected about $5 million from users (and another
sum in the same ballpark from VCs) with the promise of an AI that designs
websites for you. Years later, they still have very little to show.

It seems like most of those tens of thousands of aspiring web creators who
paid $96 to be "founding members" of this AI revolution have quietly licked
their wounds and given up, because there doesn't even seem to be too much
active discussion about the product anymore. (The Reddit forum is dead, with a
few messages of the "Can we sue?" kind.)

~~~
mtdewcmu
Maybe I was reading into it, but I got the impression that the end product
here was something more of an artistic nature rather than a website.

If the end product is something that is basically a computer program, like a
website, then I expect it to fail. My conjecture is that code and a text
editor is already the optimal tool interface for creating computer programs.
You can't create a superior tool interface, because code and a text editor is
already optimal.

When it comes to creating art, there seems to be a great potential for better
tool UIs. A guitar and hands is not the optimal way to create music. We don't
seem to have found the optimal tool for creating music yet (if there is one).

~~~
andyjsong
>When it comes to creating art, there seems to be a great potential for better
tool UIs. A guitar and hands is not the optimal way to create music. We don't
seem to have found the optimal tool for creating music yet (if there is one).

My company is working on just that, creating the next generation of music
instruments to allow anyone to create music.

Here is our first prototype where we've changed the UI of the guitar, but also
left in traditional elements to allow beginners or experts to play music by
switching from what we call "Magic Mode" to traditional guitar:
[https://www.youtube.com/watch?v=vkM4sr2socQ](https://www.youtube.com/watch?v=vkM4sr2socQ)

~~~
mtdewcmu
Music is not my area of expertise, but

1\. It looks like there are limitless possibilities in that area of
technology.

2\. Magic Mode reminds me of Omnichord. It looked like with the Omnichord, you
could do certain things more easily than with an analog instrument, but other
things became harder or impossible. So it was a trade off.

3\. In addition to allowing anyone to create music, it would be especially
interesting if you could create new possibilities for expert players. For ex:
some kind of augmented drum could allow you to create and play more intricate
rhythms than would normally be possible, perhaps by having ML extrapolate and
combine patterns in ways that are too complex to even write (I used to play
drums). It seems like the ear can hear patterns that are beyond what can be
played or even created by a person.

~~~
andyjsong
For #3, it doesn't even have to be that fancy for expert players/song writers.

One of our investors is Matt Bellamy, the lead guitarist/vocalist of Muse.
Generally when he wants to write a song, he goes for the piano to figure out
the chord progression. It's much easier to compose on the piano because all
the notes are in front of you in sequential order (black and white keys).
After he figures out the chord progression, he'll switch to the guitar.

If he was to start composing on the guitar, there is a higher amount of
cognitive load that is based on the physics of vibrating strings and his
ability to physically shorten them by pressing your fingers on metal/nylon.

With our instrument he gets best of both worlds, a guitar interface, but every
button is a chord in any key/scale. Basically, you now have a chord
encyclopedia (like omnichord), but once he figures out the chord progression,
he can switch to traditional mode and do the crazy stuff that he does all on
one device.

~~~
mtdewcmu
And it's more portable than a piano. That seems like the killer feature in
that scenario, no? You can compose songs anywhere.

------
huula
Great article. Covers a lot of aspects on how ML can help designers be more
creative and productive!

At Huula, I'm a firm believer that ML can automate various parts of web
designs. We just released a new experiment CSSToucan[1] to auto color texts on
web pages with Recurrrent Neural Networks. It learns to color texts on web
pages without a single line of color theories in the code. All learned from
the data. Hope to see more and more ML powered design tools emerging!

[1]: [https://huu.la/ai/csstoucan](https://huu.la/ai/csstoucan)

------
mtdewcmu
It sounds like one of the core issues is dimensionality reduction. I would
suggest reducing everything to text. Text is one-dimensional.

In this case:

>> Like any other map, we can add textual labels — street signs if you will.
This allows us to navigate the design space verbally with commands like: “take
me to a maple leaf.” Once there, we could say something like: “take me a bit
closer to an oak leaf.”

The change here would be subtle. Rather than a two-way conversation between
tool and user, instead it's more like the user is telling a story and the tool
is making suggestions along the way:

start:

"Create a leaf"

tool inserts a suggestion:

"Create a [maple,oak,cherry,...] leaf"

user updates the story with the clarification:

"Create a maple leaf"

tool suggests:

"Create a [red,green,...] maple leaf"

user:

"Create a green maple leaf"

user starts typing more:

"Create a green maple leaf on a"

tool inserts suggestions:

"Create a maple leaf on a [branch,wreath,...]"

Hope this is helpful.

------
oron
Great well written article

