
Thousands of bird sounds visualized using Google machine learning - ptrptr
https://aiexperiments.withgoogle.com/bird-sounds/view/
======
sharp11
As a birder, this looks like a failed experiment to me. Or I don't understand
what their goal was. The groupings make little sense in terms of what these
species sound like. I'm guessing that's an artifact of the way they sampled
the sounds, losing macro properties. Kind of like grouping the words
'paramour', 'enmity' and 'hamster' together bc they all contain /m/ sound.

~~~
glup
Another birder here, totally agreed. The audio samples are all really short
and downsampled.

------
jlg23
Unfortunately they only hint at the envisioned application in the video and
don't provide any further links, but the idea is amazing: Use sounds to
monitor bio-diversity. Imagine we'd not need cameras and lots of luck to
"catch" proof of an animals existence but a grid of interconnected
omnidirectional microphones. We'd get real time tracking of individual animals
in 3D and could have smartphones literally point the way to yet uncatalogued
or even undiscovered species.

~~~
edgyswingset
Somewhat related, but there was a very good talk at MLConf 2017 about using
sound to catch illegal logging in the Amazon. Similar premise: collect the
sound, analyze the patterns, and classify.

~~~
jlg23
Thanks for the pointer, found the video:
[https://www.youtube.com/watch?v=VlQyudLKJno](https://www.youtube.com/watch?v=VlQyudLKJno)

------
fairpx
It's good to see Google in the last couple of weeks launching a bunch of
[1]projects that are more in line with their mission of 'organising the
world's information'.

[1] [http://www.smithsonianmag.com/smart-news/google-
digitizes-30...](http://www.smithsonianmag.com/smart-news/google-
digitizes-3000-years-fashion-history-180963633/?no-ist)

------
daxfohl
From the video, it appears this is an AI-plotted hugely multidimensional space
t-sne'd onto two dimensions.

Would be interesting to do some kind of ML on how best to present hugely
multidimensional spaces onto two _interactive_ dimensions. Where one AI is
deciding how things are projected and how it can be manipulated, and another
AI is limited to some virtual "mouse, keyboard, 2D screen" to make inferences.
Such that it's optimized for faster, more correct inferences.

~~~
anotheryou
this has to end up in clustering, some color coding for the dimensions would
have been nice.

------
anotheryou
more like sorting bird sounds, not visualizing...

The tiny images are just spectograms/fft as far as I can tell.

edit: it's very fun though to click+drag, haha

------
bkasterm
There are some instances of the same bird in multiple locations (great horned
owl). Presumably multiple recordings of the same bird. My initial reaction to
them not being neighboring is to wonder about the quality of the result. Maybe
better feature engineering needed to make this biologically relevant. Any
other interpretations?

------
mortehu
If you haven't already, try zooming all the way out and drawing things using
the grid as a canvas.

~~~
mcbits
Neat, sounds just like my back yard in the morning!

------
jboggan
I really want to map these to a MIDI controller

~~~
mannytan
Stefano D'Alessio was able to make something pretty catchy
[https://m.youtube.com/watch?v=8hwDD7mYH_k](https://m.youtube.com/watch?v=8hwDD7mYH_k)

------
glup
It would be interesting to see how similarity in bird call behavior tracks (or
doesn't) the phylogenetic relationship between species. My hypothesis would be
that bird calls are influenced by other birds in the same ecosystem (imitation
or differentiation, and reflecting a high degree of cultural learning) rather
than the null hypothesis of genetic transmission.

------
bonoetmalo
Is there a better way to pan than click-dragging 100 simultaneous bird sounds?

------
thinkMOAR
Interesting, spammed a few bird lovers i know with it. Though they almost all
replied the recordings are not good enough.

Though personally (jk) i was slightly disappointed when i zoomed out i didn't
see a big bird (or other bird) likeness.

------
tren
Someone should make a Shazam/Soundhound app for bird calls, I'd definitely buy
it if it could narrow it down to a subset of possibilities.

~~~
newbear
Been thinking of this but the library of birds sounds is private to Cornell if
I'm not mistaken. The app , at least, should free and open source.

------
bravura
I poked around and also looked at a similar experiment, the Infinite Drum
Machine: [https://aiexperiments.withgoogle.com/drum-
machine](https://aiexperiments.withgoogle.com/drum-machine)

Does anyone know what they are doing t-SNE on? i.e. are they just doing t-SNE
on the raw waveforms? Or the MFCC spectrogram? Or what?

------
bane
It's interesting, at the very local level I think most humans wouldn't think
two adjacent bird sounds are all that similar. But if you drag along a long
line and listen to a series of different birds you can "hear" a definite
organized progression that seems to be organizing rhythm and major tones into
groups.

~~~
Theodores
On touch screen it is very BBC Radiophonic Workshop, a nice toy to get some
sounds from, to suit a play on Radio 4, that sort of thing. So you could make
lots of 'instruments' by applying AI to organising sounds into some space that
can be used as per this example and a touchscreen. It my make ornithologists
cringe but I think the merit in this work could be elsewhere, as a creative
musical instrument of sorts.

------
pishpash
Next: add animal languages to Google Translate?

------
KasianFranks
Feature request: Play All

------
voidmain
This is my cats' favorite machine learning application so far!

~~~
glup
Confusing my cat.

------
zo1
If you left-click and drag around on this (with short pauses), you can almost
hear something that sounds very close to R2-D2.

------
jaimex2
What? No Kookaburra?

It has the most unique call of them all.

------
simplehuman
I see "Oops, sorry for the tech trouble. For the best experience, view in "
for chrome on wayland...

------
MurrayHill1980
What task is made easier by this visualization?

------
coldcode
Wonder what it would do with a mockingbird?

~~~
Gigablah
Run unit tests?

------
mirimir
It doesn't work in Firefox. How rude.

~~~
natch
Worked in Firefox for me. However, it was a bit slow to load.

~~~
mirimir
I got:

> Oops, sorry for the tech trouble. For the best experience, view in Chrome
> browser.

But then, I block some stuff: ads, WebGL, WebRTC, and tracking.

