

Show HN: We built a voice to music AI app in 54 hours at a hackathon - seertaak
http://blog.beatmaticapp.com/

======
aculver
I recommend getting this in the App Store so musicians can start using this.
Even for a very simple version, charge at least $1.99.[1] Don't over engineer
it in terms of features within the app, but make sure the core functionality
works as advertised for a broad range of folks. There is a massive novelty
factor here which I think will create a viral growth of sales. This will allow
people who can't beatbox to experience the joy of beatboxing. :-)

If it continues to works as advertised as it evolves, I think you could quite
easily find yourself either in a licensing agreement (or small acquisition)
with Apple. Their Garageband app is packed full of "smart instruments" that
are in a very similar vein as this. Most of them are quite awesome. My son is
obsessed with playing them. They allow even kids to have fun making music and
experimenting with sound. Something like this would be a real improvement from
what they currently do for "smart" drums.

So, I think there is an initial awesomeness which is that you could have a
successful app, which in itself would be a real accomplishment these days.
This concept definitely has the potential. However, I really, really hope to
see this picked up and incorporated into a larger tool for creating music in a
fun way.

Best of luck, guys. Great job.

[1] Might be a bit specific in terms of advice, but not knowing your
background/experience building and marketing apps, I thought I'd throw it out
there. Coming in with a price tag that's at least $1.99 gives you flexibility
with promotions, etc. down the road. Furthermore, something like this
definitely provides at least that much value/entertainment.

~~~
thomasf1
Domo arrigato gozaimas... you seem to have some experience in marketing
apps... would be great to talk a bit about how you would approach it and if
what part (if any) you still would work on... we´re at dev@beatmaticapp.com

~~~
aculver
I'll definitely reach out. Looking forward to talking.

------
rootedbox
What is AI about it? There have already been a few audio to midi plugins.. How
is this different? Can it detect more than bass, and snare?

~~~
seertaak
It's different in that almost all audio to midi plugins only consist of onset
and tempo detection. Whereas Beatmatic also features classification of the
sounds, so it lays them onto tracks for you.

And yes, Beatmatic can detect any two (and in principle, 3+) "distinct"
sounds. We use unsupervised learning techniques (clustering), so your vocal
sounds don't need to sound like those used to optimize the software.

As for whether it's AI, the term is a bit subjective. I think it's not unfair
to use it as there's a fair amount of statistical techniques and tricks to
make it work, and the result is an app doing things you normally wouldn't
expect a computer to do.

~~~
rootedbox
Actually a lot of audio to midi are doing pitch on top of onset, and tempo..
some are also polyphonic; some are available as real time VSTs (are you on any
of the music dsp mailing lists? you really should be.. the prob you are doing
has been handled)...

The 2 minute hack to do what you guys are doing but being able to detect way
more that 3 distinct sounds would be to use ableton with the following audio
chain..

channel1(audio) -> looper -> vst audio to midi vst -> channel 2 (midi)

set your looper to trigger the on / off recording on channel 2.

------
nollidge
This is cool. I'm a beatboxer and think of rhythm in that "language" (the way
I always see musical notes on a piano, because that was my first instrument).
At this stage (at least based on the videos), it doesn't really offer any
efficiency benefit over just tapping out a rhythm or putting dots on a grid,
but it'll be cool to see where this goes...

------
thoughtpeddler
Heh, they weren't kidding when they said Berlin was leading a new tech
renaissance in Europe (I can see the Fernsehturm in your landing page photo).
One day hopefully Beatmatic can join SoundCloud and the rest!

Anyway, this is 100% Kickstarter material.

I would easily pay $5 just to show off this functionality to friends when
we're hanging out (but I agree that it should be kept around the $1.99 price
point). The novelty here is huge.

A lot of music apps are intimidating to most people but this is the most
intuitive way to take human musical creativity and make a structured,
manipulable track out of it. And trust me, most human beings can hum a
tune/beat out. It's instinctive so the market for this is huge.

I recommend Kickstarter rather than just shooting for the App Store because
it's going to be tough to go viral and get easily discovered on the App Store.
Kickstarter, in addition to the money, will get you a solid fan base before
you get on the App Store which is really important these days. It's just too
easy to get lost among all the other apps, no matter how novel your entry
might be.

So get on Kickstarter, raise a decent amount of money to get a very polished
app ready, but more importantly, use the time to get your concept viral BEFORE
debuting on the App Store. You won't believe how much publicity is generated
by the weekly updates that Kickstarter projects provide. Each week you give a
little nudge to all the supporters through your product development updates
and each week they're reminded to tell friends and family about their backing.
The press will of course catch on and you'll get that steam too.

Good luck to you guys. This is definitely at the intersection of liberal arts
and technology that Steve Jobs loved touting.

If you guys want to talk more about strategy from here on out, check my user
profile to get in touch. I'm definitely supporting this project.

~~~
joshschreuder
I'm not sure Apple's distribution really supports Kickstarter though. You
wouldn't be able to give away the app in that way.

~~~
thomasf1
Kickstarter would be great for this, especially building a community.

Maybe a way would be to do a 5$ Kickstarter for people to get early access to
the app, even before their friends can buy it from the App Store. This could
work through the Apple Enterprise Program, but it might not be allowed in the
small print...

That could generate some hype, and backers would have something exclusive for
a while till it´s available in the app store. It also would build a community
that surely gives great feedback and the first few iterations wouln´t need to
go through the app store approval process (which would mean faster iterations
/ feedback).

~~~
thoughtpeddler
Absolutely. There's nothing worse than debuting an app just to have it
destroyed by 1-star reviews commenting on stupid things you could have learned
about through a more open beta process that something like Kickstarter would
allow.

------
famoreira
Cool stuff guys! Can you share more details on the serve side implementation?

~~~
seertaak
Sure, on the server side we have jetty running an HttpServlet, which currently
just applies the computations straight on the request thread. Yes, you read
that right! It's designed to be run on remote servers, but we haven't gotten
quite that far yet and were more focused on making a demoable app. But it's
basically Java 1.7/Maven and your typical libs like Guava, Commons {Lang,
Math,...}, and a few other bits and bobs.

Basically a file (currently .wav, but it ought to be .mp3) is uploaded to a
the server, which processes it directly in thread and returns the results back
in JSON. Really it should be fanned of to a worker pool and update the client
with ajax but again, we're not quite there yet.

------
sic1
I really dig this work. Been wanting something just like this! I find myself
beatboxing here and there randomly, and sometimes i really brew up something
nice. Would be awesome to have a great way to catalog these "beat-deas". I can
envision this app as an easy means for me to pop out of my pocket real quick
when i have a good one. Would be nice to have the original beatbox sample
saved as well for a reference.

From there - getting this (beat pattern at the least) into something like
Ableton (or some importable format beat format (does that even exist?
(Inception!))) would be so awesome.

The biggest thing for me would be for it to be able to handle a pretty
complicated rhythm. When i get goin, its a lot faster than your examples (and
i'd imagine 2 notes hitting in the same place create a problem as well). I'd
imagine it gets pretty complicated pretty quick.

Maybe even a way to record a beat, have it do the conversion, then record over
it to make more complicated rhythms.

As you can see i'm excited about your product, and am probably getting way
ahead of myself on some of the comments. But either way, keep it up, and let
me know when i can try it myself. Thanks!

~~~
seertaak
Thanks, the ability to export to Ableton would be very cool. It's actually not
that hard; Ableton can import midi, so as long as you set up the actual sounds
yourself it's as simple as storing a .mdi file on Dropbox or Google Drive.
That's something we could feature. And exporting entire loops should be easy
as well.

As far as complicated rhythms go, I'd love to get some audio samples of you
beatboxing! One of the things we have planned is a petition for beatbox
samples -- we'll write a web app or rich client that makes it as easy and
quick as possible. That'll allow us to get some better data than the little
corpus I've been building up!

If you want to help out, you can either sign up for updates at
<http://beatmaticapp.com> or email us at dev at beatmaticapp com. It would be
much appreciated :)

------
thomasf1
We´d love to hear your thoughts on where to take this...

Either here or on Quora :) [http://www.quora.com/Startup-Advice-and-
Strategy/We%C2%B4ve-...](http://www.quora.com/Startup-Advice-and-
Strategy/We%C2%B4ve-build-some-AI-that-turns-voice-into-music-where-should-we-
take-that-product-wise)

------
tsycho
Awesome stuff! If you make an iPhone/Android app for this, it might even sell
well for amateur impromptu musicians. Alternately, once your backend is more
stable, you could make an API, and others can build mobile apps on top.
Contact me if you want to collaborate on the iPhone end.

~~~
thomasf1
Thanks for your offer, sure we´d like to hear from you. It is already a iPhone
app, although limited by time constraints one build in PhoneGap rather than
native. We probably will move the app into a native app, or extend PhoneGap
via plugins for more advanced sound functionalities like filters.

Our emails by the way are thomas / martin @ the app domain...

------
daralthus
Cool stuff I hope you go on with it! Also worth to check out "The Mouth" from
Native Instruments: <http://www.youtube.com/watch?v=zzyr66QhrOw>

~~~
seertaak
First off, thanks!

I've been thinking about playing around with that plugin for some time, I just
haven't got around to it yet. My understanding is that there is no voice to
music feature -- it's more about vocoder and harmonizer effects. It sounds
like a genuinely innovative and cool product, though, so I have nothing bad to
say about it.

The difference between the two products is that the emphasis of "The Mouth" is
on vocal performance, whereas the emphasis in Beatmatic is on composition. The
compositional aspect probably doesn't come across that clearly in the videos
because, well, all we have implemented is beat detection. But suppose the next
step in the app was to voice a bassline, or a melody instrument, or have some
cool pre-built synth loops to add to your beat, then we're talking about a
music app that can, in theory, offer the ability to write a song to anyone who
can sing "Happy Birthday" roughly on key and in time. As a musician who has
experienced the joy of writing songs a few times (my band: babylonbaby.de),
I'm excited about the idea of sharing that awesome experience with a much
larger slice of the consumer market.

------
deadlysyntax
This simplifies the process even further than typical midi editors do -
lowering the barrier to entry for creating music.

But as a professional drummer and tutor, a tool like this could seriously help
me, if you decide to expand the feature set beyond a novelty. For example,
this could be a way for me to take notes of drum beats that come to me when
I'm not at the kit, if it supported tuplets.

------
fudgy73
Thought this was pretty neat. I'd pay a buck for it and I don't really even
listen to music.

You should get to a level to where an average beatboxer can input and this
thing spits out something sweet. Game-ify it by having people try to match
something onscreen with their mouths or do something a la "draw something"
except I input the beat and you guess what song it is or vice versa. Fun!

------
logn
I think $1.99 could serve the mass market. But by and large musicians are used
to shelling out some serious cash for audio apps that have a lot of features.
A desktop or more full-featured version could easily sell around $30-50 I
think, especially if you find ways to integrate it into other tools, e.g.,
export as Apple Logic project (if that's even possible).

------
portlander52232
Wow... I've been working for a month on an app that does exactly this. Great
job, guys! I'd love to know about your algorithm.

~~~
seertaak
Sorry for treading on your toes then! :) Feel free to email us at
dev@beatmaticapp.com, and we can have a chat about implementation. There's a
lot of rough edges and it would be great to talk with someone who's dealing
with the same issues!

~~~
radarsat1
Damn i was working on something like this too :-P

Congrats, it sounds cool.

------
SoftwareMaven
My son is a Jazz nut, often playing for hours just for fun. The rhythm section
is always the most challenging for him. He usually plays to a CD that has a
bunch of different rhythms. I can see him and his friends having a blast with
this. Let me know when I can buy it!

~~~
thomasf1
Thanks... that really motivates :)... Let´s do some market research here if
you don´t mind: how much would you pay and what price do you think is fair?

------
_girishrao
Awesome stuff guys. Hackathon projects are fun. Also curious about server side
implementation!

------
domfilosa
I agree with many of the comments: Get this in the app store quick!! Then,
release alternative versions later.

I'd buy it just to show my friends and maybe jam a little like you did. So
rad. Triple "like". I hope you make a million dollars.

------
apercossi
This has got good commercial potential, not just as an app but also as a
Guitar Hero type game played by groups. Keep it simple and ensure it can
generate "immediate gratification" and you're onto something big.

------
joaoff
Excellent work, guys! As an amateur sax player, I can see myself using your
app to produce some funky beats on-the-fly. I wish you all the best and I hope
I can buy your app in the near future. :)

------
soulboy01
is the basic concept of this any different from the Vocal Beater app that's
currently available for iphone/ipad by Migamo? (mike gao mobile) perhaps you
should link up with him, and see if you guys could combine your research or
exchange ideas. i'm sure you guys could provide each other with some mutually
beneficial insight with what you guys are trying to accomplish. either way, I
look forward to seeing something like this available for android, since that
vocal beater app is only available for ios devices.

------
gilini
Great video, I'm already sold. Software + music is one of my passions, I bet
it's yours too, so I'm hoping you guys get traction with this app :)

~~~
thomasf1
Thanks! What direction would you take it first?... We´ve jotted down a few
"product" ideas in the blog...

------
experiment0
Hi Guys, Did you do this in the Startup Weekend event at Campus Party Berlin?
I was doing it too! Well done for your app it looked awesome!

~~~
thomasf1
Absolutely right... :) We really liked the challenge and not knowing if we
could pull it off... It was good fun...

~~~
experiment0
Awesome, we did Reco, the picture deals one. Was a lot of fun and met some
awesome people!

------
mkhalil
What OS/Programs where you running on those dell monitors in that video? Looks
like it's doing some sampling? Pretty curious.

~~~
thomasf1
It´s Eclipse // Java FX and Mathematica on Linux... The Macbook runs OSX with
XCode and some more stuff...

~~~
mkhalil
Cool thanks!

Bad ass application btw. Looking forward to it.

------
gtardini
Congrats! i was in berlin and watched your presentation at campus party. Good
job!

------
jurre
Awesome stuff! No need to be so apologetic about some stuff not working
optimal!

------
skant
This is just awesome guys.. Wish you all the best :)

------
Revex
Sweet guys. I'm looking forward to a nice write up.

------
davj
Kickstarter

------
emigal
Well done gents!

------
dpakrk
great hack ! cheers guys

------
hospital
really cool stuff.

------
ericraio
real cool app!

