
Identify Plants with an App - peter_d_sherman
http://www.plantsnap.net/
======
misja111
I use the flowerchecker app
[http://www.flowerchecker.com/](http://www.flowerchecker.com/) where you have
to pay a small fee per plant identification. It actually uses real human
biologists to identify the plant.

~~~
lsjdfkljdfwkwdf
You mean college students?

~~~
RawData
College students are in fact real humans....

~~~
sangnoir
> College students are in fact real humans....

That is _mostly_ true. The existence of animals with diplomas[1] is the
exception to the rule.

1\.
[https://en.wikipedia.org/wiki/List_of_animals_with_fraudulen...](https://en.wikipedia.org/wiki/List_of_animals_with_fraudulent_diplomas)

~~~
0x4a42
But having a diploma doesn't necessaraly implies the animal studied in
college. So your argument is broken.

------
dharma1
I did this for common plant diseases using Tensorflow on Android when learning
how to use TF last year. Found the accuracy to be not that great with real
world images that are not in the dataset (with a dataset of 50,000 images from
PlantVillage). I think there are just too many visual similarities between
various plants (especially when taken from various angles) that the network
doesn't focus on the right things. Did a short writeup here:

[http://helminen.co/plant-disease](http://helminen.co/plant-disease)

It could work for a subset of plants, or potentially with a much larger
training set - but I think NIR spectral/hyperspectral imaging would be the way
forward here with more differentiating data points.

~~~
lojack
Not an expert -- would this be a result of training data doing a much better
job isolating the subject vs a real world photo that may have other plants in
the scene? Not that this is a solution in all cases, but is it possible your
results would have improved with better real world pictures?

~~~
roystonvassey
Likely but the whole point of 'AI' is that it should be able to identify
flowers that don't look like what it's seen before. If not, it's just a huge
lookup table that's like a memory repository.

This is, of course, a critical challenge in data science and is definitely not
a trivial one to solve.

~~~
lojack
In some cases these are the goals, but dharma1 said the goal was to identify
plant disease. If you can improve your results by taking better pictures then
it becomes a tradeoff between training someone to take pictures and training
someone to identify plant disease.

I think we have a tendency to treat AI as a silver bullet when we should be
treating it as a tool we can use to help augment what we're already doing.

------
aphextron
This product already exists. It's called Google Image Search. For instance, I
was actually able to identify the second flower he takes a picture of in the
video as Camellia Japonica simply from a screenshot of the video. I'd be blown
away if they were able to assemble a better training set than Google. Kind of
disappointing, as from the headline I was expecting something based on NIR
spectroscopy, like the SciO molecular sensor
([https://www.consumerphysics.com/](https://www.consumerphysics.com/))

~~~
max_
In fact. There is a better product, Google Goggles -
[https://play.google.com/store/apps/details?id=com.google.and...](https://play.google.com/store/apps/details?id=com.google.android.apps.unveil&hl=en)

~~~
hrktb
I don't know if the mention was intentional or not, but Goggles is actually
dead as a product. They removed it from the iOS app, and the android app is
not supported, while staying in the store.

In this respect, as the GP says, image search is currently the best option

~~~
shaklee3
Goggles (the concept) isn't dead. They moved the functionality into Google
Assistant so you didn't​ need the app. Just hold down the home button so it
takes a picture of the camera view.

~~~
SamBam
Huh, I had no idea. An Easter egg, almost, as I've never seen this mentioned
by Google.

It works ok, send pretty beta right now. I pointed my camera at a business
card and pressed the home button. Google Assistant appeared and I asked it to
tell me about what was on my screen. It noticed one piece of the address, and
brought up a listing for that general area, but there was no way to further
inquire into any of the rest of the card.

~~~
shaklee3
Sorry, I should have been more clear. It's moved into the "now on tap"
feature, which now links to Google Assistant. You're right though, if you ask
what's in your screen it has the same effect.

------
videogramme
A great app, collaborative tools and community around plant identification is
the french pl@ntnet [http://identify.plantnet-
project.org/](http://identify.plantnet-project.org/) from Agropolis Fondation,
Tela Botanica, INRIA, CIRAD, CNRS, INRA, IRD and Montpellier University

------
valdiorn
> Fast forward to today - the tech is available and the team is assembled to
> make PlantSnap a reality. With a beta version completed, and 250,000 images
> in the database

250,000 images doesn't sound like a large enough training set to be effective
on anything but the most common plants.

~~~
ovi256
To help others understand why, the "Deep Learning" book, which summarizes
current state-of-the-art DL, advises having at least 5000 samples (images) per
class for OK performance (equal to non-DL approaches) and 50k to 100k for
state-of-the-art. A class here would be a plant species.

So even with 5k samples, the 250k image corpus would only have 50 species,
using this rule of thumb. A good engineer could pick up DL and build a system
that performs to this standard, because the tricks are all written down in the
literature.

If they do better, they either exploit unpublished methods, or researched
those methods themselves, with their researchers.

~~~
Houshalter
But there is a lot of transfer learning between classes. For example, the
network may learn to detect "broad leaves" or "red stems" for a common plant.
It will then be able to use those feature on less common plants, and require
much less training data for them. Transfer learning has been shown to work
with just a single image of a desired class.

It's also nice that life naturally fits into nested hierarchies because of
evolution. So if you can recognize what family it belongs to, then that
narrows down the possible sub families it can belong to. That in turn narrows
down the possible clades it can belong to, etc, which narrows down the exact
species. You couldn't find a more perfect use case for hierarchical softmax!

------
mharju
We did this
[http://www.luontoportti.com/suomi/en/](http://www.luontoportti.com/suomi/en/)
a couple of years ago. It is based on an elimination process – you describe
what you see and the application narrows down on the options. Works very well
in practice, and can be applied to all sorts of indentification purposes in
the wild, such as fish, birds and butterflies.

We also gave the image regocnition path a thought but it seemed to be quite a
tall order. Hopefully they come up with a novel approach on this!

~~~
0XAFFE
Is there something like this for Android? This app seem to be restricted to
iOS.

~~~
mharju
Unfortunately we did not get to the Android App. Maybe some day :)

~~~
mongol
I am almost certain I once installed it and paid for the birds. Even emailed
support suggesting a Swedish version.

------
mrgreenfur
When can we get this for identifying plants/animals/rocks/fungus/etc in the
wild? Would love this on a phone when hiking.

~~~
pingou
For fungus you could try
[https://play.google.com/store/apps/details?id=com.pingou.cha...](https://play.google.com/store/apps/details?id=com.pingou.champignouf)
[https://itunes.apple.com/us/app/champignouf/id1227854971](https://itunes.apple.com/us/app/champignouf/id1227854971)

I made it as an experiment, results vary and can be awfully wrong but it's
funny to see the confusions the neural network can make. Funny until somebody
eat a deadly mushroom because of me, I guess. I already put plenty of
warnings, so fingers crossed, but it's hard to be sure it won't be misused.

~~~
eagletusk
This is an interesting problem that almost all guide book authors have to
contend with, bringing knowledge of a topic may expose people to risk, but I
think it's maybe a false conclusion.

It's a given all guides have false information, it then begs the question are
guides worse than no guides? I think this can be easily answered more
information is better.

You can give people information but you can't understand it for them.

kudos for making the app :)

What do you think your app would identify this as? Carpet mushroom?
[http://imgur.com/a/Rqc64](http://imgur.com/a/Rqc64)

------
whistlerbrk
A similar app was released a while ago, I'd love to compare the two. For
reference: [http://leafsnap.com/](http://leafsnap.com/)

~~~
kalmari
thats the one i remember; worked well enough. not a big flora person but was
curious and wanted to try out the image rec.

------
nr23
I use this android app for recognizing plants by picture:
[http://identify.plantnet-project.org/](http://identify.plantnet-project.org/)

It's made by french researchers I think. It doesn't work perfectly but I did
identify a lot of plants I don't know with it.

You can snap multiple pictures of the same plant, for example, 2 pictures of
the leaves, 1 of a flower and 1 of the bark, and then use the combination to
search. You can also submit your observations to have them identified by
experts.

~~~
rorykoehler
It doesn't work very well in Singapore at all. I guess the training set didn't
include much from this part of the world.

~~~
notimetorelax
In the app it does say that

    
    
      Pl@ntNet only works for the plants of the French metropolitan flora.

------
prawn
Only slightly related, but I've always wondered about an app that told you
what crop was growing in the area you were driving through. Often they're
recognisable, but sometimes they're not. No business case for it, just to
satisfy curiosity.

------
gozzoo
This is very cool. I've been thinking about something similar, but only about
mushrooms. Some people in my country pick wild mushrooms and sometimes it's
very difficult to distinguish between the edible and the poisonous types. Such
application, if existed could save many lives. The same concept could be used
for other plants that people gather, like herbs and wild berries.

~~~
Houshalter
But if the algorithm screws up and gets someone killed you could be sued.

------
2845197541
Finally, I can identify mushrooms that will kill me and eat the ones that
won't.

~~~
oneeyedpigeon
I hope, for the author's sake, the app has robust Ts&Cs ...

~~~
kalmari
exactly. something like not resposible for what you do with this information
where the info is correct or not.

------
diyseguy
I've had good luck with plants.usda.gov identifying plants by geographical
location. Just use the advanced search, choose the location you found the
plant, e.g. state:county, and as much information as you can deduce, flower
color, growth form, etc. It'll usually provide pictures and interesting things
like whether Native Americans used it medicinally, etc.

------
alexandreshah
[http://www.gardenanswers.com](http://www.gardenanswers.com) app has the
ability to automatically identify plants. If you can't find your plant,
because it isn't blooming or is immature, or just isn't distinctive enough for
image search, you can ask a horticulturist for a small fee.

------
folli
Definitely an interesting concept. There's a bit more information on the
indiegogo website [1].

Their problems they mention seem to be quite standard for image recognition
(scale- and perspective variance), however I could imagine that this could be
quite disastrous for plant recognition, given the massive diversity of plants:
e.g. a leaf viewed from the side could equally well be a more narrow leaf from
a different species.

As they write on [1]: "Our challenge comes from adapting our image recognition
platform to recognize different shapes and sizes of the same plants, flowers
and trees. We know this is possible, and we are close to cracking it."

[1] [https://www.indiegogo.com/projects/plantsnap-identify-
plants...](https://www.indiegogo.com/projects/plantsnap-identify-plants-with-
an-app-nature#/)

~~~
yasth
Honestly they don't seem to be that close to cracking it, like at all.

They may be using camera phone images, but they are well framed images of very
distinctive features. Trying to recognize a literal tree in a forest is going
to be far harder, and recognizing a tree based on a dead leaf (as a photo
towards the bottom suggests) can be very very tricky. Also I have to admit to
being concerned that adding "any known plant" won't kill accuracy. The more
classification endpoints the less likely you are to get a decent result, and
already you have to deal with far greater scale differences than are usual
(basically it would be like identifying the breed of dog from anything from a
full picture to a photo of a single claw).

------
fredley
It's a nice idea, but I have a feeling it's never going to be better than
human enthusiasts/experts, who you can get to identify plants from photos for
free already[1].

I can immediately think of all kinds of challenges that are _really_ hard to
overcome: diseased leaves, different seasons, plus all the usual
glare/shading/background issues.

[1]:
[https://gardening.stackexchange.com/q/9355/3346](https://gardening.stackexchange.com/q/9355/3346)

------
hyperpallium
Botantical classification is interesting but difficult. Once trained, it's
fast enough, but can "deep learning" be anywhere near accurate enough?

I'd guess there'd be a focus on salient botantical features for
classification, and perhaps the human can be enlisted to circle them out.
There could be a "twenty questions"-type narrowing down, perhaps using images.

~~~
folli
Dichotomous keys are the standard in taxonomy for species identification since
centuries. It's basically a binary search tree, generated by hand.

[https://en.wikipedia.org/wiki/Single-
access_key](https://en.wikipedia.org/wiki/Single-access_key)

~~~
pvaldes
Dichotomous keys are mostly useless when you deal with photos. For some reason
people always find the most useless chamera position when taking photos to
unknown plants.

~~~
pvaldes
chamera -> camera

------
madshiva
I want do the same for mushroom, because they are hard to recognize. Google
images works quite fine.

~~~
pingou
I did it, but the results can be pretty disappointing, I'm still working on
improving them:

[https://play.google.com/store/apps/details?id=com.pingou.cha...](https://play.google.com/store/apps/details?id=com.pingou.champignouf&hl=en)

[https://itunes.apple.com/us/app/champignouf/id1227854971](https://itunes.apple.com/us/app/champignouf/id1227854971)

ps: any improvement suggestion is welcome!

~~~
madshiva
Nice, will give a try. Hope you didn't stole my spot where I get my mushroom
:P

Google Image is really working quite good. I made a lot of test. The only
problem is about there's TOS. I can easily write something but I don't want to
be flagged because I don't want use there API.

------
divanvisagie
A pre-order on an app, seriously?

~~~
hekker
This landing page might just be a way to discover if there is any traction on
the market without needing to build the actual product.

------
samirillian
I would imagine a series of questions that you answer about the plant would
help increase the accuracy. (Leaf patterns, etc.) Or, at least, whatever
answer it comes up with should confirm the defining characteristics.

------
midgetjones
Having recently become the owner of a garden, this looks very interesting.
I've been using the myGardenAnswers app, which purported to do the same thing.
It's been singularly useless so far, though.

~~~
pvaldes
> It's been singularly useless

I'm bribable for the job... :)

------
subutux
I've been using this (free) plantnet app [http://identify.plantnet-
project.org/](http://identify.plantnet-project.org/)

------
Herodotus38
Glad to see this as it is something I have been interested in for a while.
There are some online tools I have tried in the past but nothing that really
seemed to work well.

------
dates
or go get a book on plants and you will be able to identify them pretty easy.
The guidebooks ask you questions to narrow it down (how many petals does the
flower have? for example). you dont need any knowledge going into using one,
and you'll learn to observe and appreciate plants more as well through the
process of IDing! newcombs wildflower is the go-to for beginners on the east
coast but there are many.

------
otto_ortega
I love the idea of an app to identify plants, I hate the idea of having to pay
to use it... Definitely, I'm not in the target market.

------
personjerry
I wanted this app for a while, but it didn't seem feasible even with modern ML
techniques? Also what do I get out of a paid version?

------
zeveb
Weirdly, the site disables the normal scrollbar but adds a new one inset,
which doesn't respond to SPACE to scroll down.

------
cs2818
I wanted to do something similar for snakes once, but after some thought it
didn't seem like the safest idea.

------
rnga1dco
Just a quick note to the site creator- the twitter icon in the footer is
linked to instagram, and vice-versa.

------
surfrider
/r/whatsthisplant FTW

~~~
glenneroo
Now we need to write an app to post images automatically and distribute a
bitcoin bounty (set by OP) based on votes.

------
Mytruthlies
This is really cool. IF only I could find an app that does this with bugs.

~~~
Insanity
Why do you want an app with bugs?!

But on a more serious note, I would appreciate that as well, but it might be
harder to take a good picture of a bug by which it is identifiable?

I'm always impressed by these type of apps though and would love making one
myself

------
desireco42
I trained my kids, first for mushrooms, then for flowers and trees. They are
better then any app :). But definitely would love to have an app to confirm
our findings sometimes.

~~~
leppr
Train them to find truffle and you got a business.

~~~
desireco42
Ha ha, good one. It is good that they love nature and it is nice if there are
good companion apps. Some I saw, are not that great, so there is clearly need
for more apps.

------
darkst4r
I really want this app. That is all.

~~~
iamwil
would you pay for it? I also kinda want it, as I get curious when I'm out on a
hike, but I'm not sure I'd pay for it.

~~~
holydude
Why not if the price is right ? The biggest obstacle for me is the need to
register and create an account not the price itself.

------
sscarduzio
WTF 150K of his own money in 5 years? I don't think at a business level this
is wise under any circumstances for a bootstrapped project. Unless you have a
ton of money to waste and this is your caprice. Of course.

