

New iPhone app identifies trees from photos of their leaves - gmac
http://leafsnap.com

======
apu
This is my project =)

Leafsnap is a project I've been working on for about 2 years and we finally
launched this week! It's a free app for the iPhone (with an iPad version
coming next week and an Android version later in the summer) that contains
very high-quality images and descriptions of the tree species of the Northeast
US. It also has an automatic-recognition feature that uses computer vision to
help users identify trees by taking a photo of a leaf.

This project is a joint collaboration between the computer science departments
at Columbia University and the University of Maryland, and botanists at the
Smithsonian Institution.

Please let me know what you think! Also, I'd be happy to answer any questions
about the app or the recognition technology behind it.

~~~
morganpyne
Congratulations and well done. I'm just downloading it now. My first thoughts
were if it was able to identify the 320+ species of native trees we have here
in NZ. My second thought was to wonder if it can be trained by the users to
learn new species? (I'm still waiting for the app to finish downloading so
have not tried it yet). It would be great to expand your knowledge base by
having user contributions, perhaps in a curated or weighted submission
process.

~~~
apu
Thanks! We currently have coverage of about 200 species from the northeast US.
We're focusing on expanding the coverage to the rest of the US (~500 more
species)...and then the world!

We definitely want to take advantage of user contributions in some way, but
we're still trying to figure out the best thing to do. Also, the app contains
several very high-quality images of each species taken in a particular way by
a talented group called FindingSpecies, and to keep the visual integrity of
the app, we would want very similar photos for all species that we add. This
might be difficult to outsource to the crowd.

~~~
defrost
It's not something that can be readily outsourced to "the crowd" in general
but it is something that can achieved using a hierarchy of interested/invested
self checking power users.

It's a similar problem to that faced by, say, open source projects that aim to
maintain a certain level of code quality and style matching in contributed
source code.

Addendum: the image collections of the Australian National Botanic Gardens and
the Australian National Herbarium might be of interest to you down the track:
<http://www.anbg.gov.au/anbg/photo-collection/index.html>

------
ataggart
Having surpassed Star Trek's communicator, we seem to be well on our way
towards the tricorder.

~~~
Groxx
Nah, I think we'll have that as soon as someone wires up a camera to
Mechanical Turk.

"Looks like cancer." "Definitely Athlete's Foot." "It's not a tumor!" "Please
state the nature of the medical emergency."

Repeat 1000x, pull out the winners, and diagnose!

~~~
bh42222
_Repeat 1000x, pull out the winners, and diagnose!_

1\. If consistent winners can be identified, I'd expect them to demand much
higher then average pay.

2\. Everyone of your turks might have to be licensed to practice medicine, or
the government might put at least some of you in jail.

I know of at least a few melanoma identification, using software, efforts
which died due to regulation and the shocking resistance of doctors.

But I wonder how free apps, good at diagnosing all kind of things, would do?
With cellphone CPUs and cameras getting better you could catch quite a few
things...

~~~
VladRussian
similar to already working scheme in the pathology - the hi-def microscope
scans are send to India, analyzed by real professionals there, and the results
are signed off by professional here

>I know of at least a few melanoma identification, using software, efforts
which died due to regulation and the shocking resistance of doctors.

Just make sure that "For entertainment only" is clearly visible on the product
label :)

>But I wonder how free apps, good at diagnosing all kind of things, would do?
With cellphone CPUs and cameras getting better you could catch quite a few
things...

and with portable ECG, EEG, ultrasound devices, DNA chips, etc easy available
and connectable ...

------
xutopia
That's funny I wanted something similar to determine it based on the trunk of
each tree. In Canada there are no leafs on trees half of the year and you can
make some interesting edible syrups with certain trees.

~~~
morganpyne
Being into woodworking (lutherie specifically), I would kill for something
that could identify species by samples of wood. I think this is a very
different task and extremely difficult to achieve though. Sometimes even 2
experts disagree on the pedigree of a given piece of wood - visual inspection
is simply not enough and nothing short of a detailed lab analysis can confirm
some identifications. Variation of the wood from tree to tree of exactly the
same species is large, at least you are dealing with relative consistency in
the leaves.

------
willwagner
Awesome app. I can't wait to use it on our next hike.

I really wish there were more apps like this (and Shazam). Just off of the top
of my head, i'd pay a reasonable amount of money to identify: bugs, fish,
seashells, animal footprints, mushrooms, and I'm sure a whole lot more.

Being part of a hiking family with two small kids, the questions are endless
and even if it were crowdsourced in some way (e.g. please identify this bug
for karma), it could be handy learning tool for kids, getting them more
excited about science at an early age.

------
eiji
Maybe a stupid question:

    
    
      Why isn't it open source?

------
zumbojo
A NYT article during development in 2009:

<https://www.nytimes.com/2009/05/10/business/10novel.html>

------
wallflower
Love the About page. Nice personal connection.

<http://leafsnap.com/about/>

Ah memories of my 10th grade biology class (bug or leaf collection). The
trusty Audubon Guide to North American Trees. It is really shocking that the
HGTTG is coming true...

------
kirpekar
Superb -- fun app while hiking with my kid. I grew up in another country, so a
lot of trees where I live are unknown to me.

Thanks

------
ksolanki
Very nice to see a good computer vision app make it to the market. This has
certainly brought the future closer to us :)

------
jenniart
I really adore your about page with all of the photos of your team with
leaves. Very fun and personal. :)

------
eekfuh
Does this use any type of location lookup? I wonder how beneficial that would
be, either look at the persons location if they use the cam or grab it out of
the exif data, so that you can better determine the tree by where it of could
be grown.

~~~
apu
Not right now, but we store the GPS with all images. Once we expand our
coverage to more of the world, we will probably start using the GPS to help
narrow down the list of possibilities.

However, the app does let you filter the list of species shown to "New York"
or "Washington DC" or "The Northeast".

------
mooneater
This is a fairly technical crowd, can you tell us more about how the algorithm
works?

~~~
apu
<http://news.ycombinator.com/item?id=2522523>

------
pshapiro
Wow, I was just thinking about how to accomplish this idea yesterday.

------
superchink
I can't seem to get to the site. Requests are timing out. HN effect?

~~~
apu
I wish -- but it was actually a Guardian article and an intomobile article
that knocked us out. We're back now!

[http://www.guardian.co.uk/environment/blog/2011/may/06/iphon...](http://www.guardian.co.uk/environment/blog/2011/may/06/iphone-
app-wildlife-species-recognition)

[http://www.intomobile.com/2011/05/06/leafsnap-iphone-app-
let...](http://www.intomobile.com/2011/05/06/leafsnap-iphone-app-lets-you-id-
trees-camera/)

------
phlux
It would be great if you could take a pic of a tree leaf, and i will capture
the geoloc of the pic as well and plot where that tree is on the map - and
start tracking where all types of trees live.

~~~
apu
This is exactly what we are doing. We're working with botanists at the
Smithsonian Institution in DC who are very interested in this kind of data.

~~~
andrewgleave
Great app. Next week I'm releasing a project for the Isle of Man which
crowdsources the recording of location and health of Elm trees (via iPhone and
Android app). The Island is one of the very few locations in the UK which
still have a healthy population but has recently suffered funding cuts meaning
Gov. surveys are no longer being done, and hence the need for a public
solution.

If the app could recognise more species of elm (English and American seem to
be there, but no Wych or Dutch for example), would be a valuable tool for
users trying to contribute to the project.

The source for the Django app and PhoneGap-based mobile app will be up on
GitHub next week, and the data is publicly available via our CouchOne instance
(<http://redrobot.couchone.com/_utils/database.html?openelm>) – the project
isn't live yet so there's only a small amount of data.

------
coldarchon
I'm just happy you didn't make one for insects. Just imagine an app that
doesn't recognize a running bug and the owner of the iPhone kills it after 3rd
try to get a better result ..

