
How a Japanese cucumber farmer is using deep learning and TensorFlow - karmel
https://cloud.google.com/blog/big-data/2016/08/how-a-japanese-cucumber-farmer-is-using-deep-learning-and-tensorflow
======
Animats
Cucumber grading has been automated for years. Look on YouTube for automated
cucumber grading systems.[1][2] There are many competing vendors. The
commercial machines process their video locally and don't need "the cloud".
They're also much faster.

Fruit and vegetable sorting using computer vision is routine. That's why
commercial produce is so consistent - it's been pre-sorted. The existing
technologies are so fast that it's routine to inspect and sort peas, and
possible to sort grains of rice.

Most of the cost and complexity in the commercial machines is in the material
handling. There are conveyors and devices to get the individual items lined
up. Cleaning and washing stations are usually part of the line. Commercial
machines have to be built sturdily so they survive continuous use, and have to
be cleanable with high-pressure soapy water. The little unit that does machine
vision is a minor part of the processing line.

AI techniques are routinely used. Support vector machines are popular. One
Indian system uses fuzzy logic. ANNs have been tried but are not yet popular.

The systems generally measure user-defined features - length, width,
curvature, color, flaws - which then go into a simple grading algorithm.
They're not trying to match human decisions.

[1] [https://www.youtube.com/watch?v=5DCpOx-
q_yo](https://www.youtube.com/watch?v=5DCpOx-q_yo) [2]
[http://www.aweta.nl/en/produce/cucumber.html](http://www.aweta.nl/en/produce/cucumber.html)

~~~
aab0
"Cucumber grading has been automated for years. Look on YouTube for automated
cucumber grading systems.[1][2] There are many competing vendors. The
commercial machines process their video locally and don't need "the cloud".
They're also much faster."

I think OP in a way demonstrates something characteristic of Japanese
agriculture:

"There are also some automatic sorters on the market, but they have
limitations in terms of performance and cost, and small farms don't tend to
use them."

Japanese agriculture is notoriously unproductive compared to other countries'
yields and inputs, and a major reason is the lack of scale and mom-and-pop
farms. If this farm were bigger or needed to save labor costs more, it could
afford the already existing solutions and would sort cucumbers into more
standard grades than their own ad hoc system. But they're not, so instead they
do it by hand.

~~~
yomly
>Japanese agriculture is notoriously unproductive compared to other countries'
yields

As an aside, my dad swore the produce was notedly better than what we have
locally when he was travelling there. He on multiple occasions also discussed
their dedication to quality.

Maybe they're onto something given this related discussion was on HN earlier
this year...

[https://news.ycombinator.com/item?id=10997485](https://news.ycombinator.com/item?id=10997485)

~~~
m_mueller
I'm a Swiss living in Japan and the produce here is hands down the best I've
ever eaten. Locally produced stuff in fertile areas in Switzerland comes
close, but in Japan it just seems to be so much more bred for taste than
looks. No problem to eat raw with nothing added.

------
ktamura
While it's a cool concept and agriculture is ripe for technology disruption,
the enterprise software product marketer in me finds it to be a calculated
marketing stunt, if not an disingenuous one.

TensorFlow is, in no way shape or form, ready to be used by anyone but very
technical people with a solid knowledge of both software engineering and
machine learning. In fact, it's probably unfair to single out TensorFlow
because every deep learning toolkit I have seen is still in the same
"extremely early adopter" phase. Yet, this article makes it seem like a line
of business user (farmer) with some background in technology (former embedded
systems engineer) can use deep learning to transform his/her business -when,
in reality, much expert help is needed to make it work. In other words, it
would have been a lot more plausible/genuine if the article read like:

1\. A progressive farmer got in touch with Google.

2\. Google dispatched their solutions architect to work with them.

3\. Hey, a cool, early prototype is working!

I am not trying to be a hater. Deep learning has huge potential, and Google,
among others, is doing a lot to make it accessible. That said, this type of
over-promising ahead of market reality is what gives cutting edge technology a
bad name.

~~~
devy
> in reality, much expert help is needed to make it work.

I totally agree. Having been taking classes and read ML/CNN articles online,
this Google Cloud blog post made this project seem to be significantly easier
than the actual effort that went into it. In particular, the blog simply
covered the secret sauce to make it work in one line:

> Makoto used the sample TensorFlow code Deep MNIST for Experts with > minor
> modifications to the convolution, pooling and last layers, > changing the
> network design to adapt to the pixel format of cucumber > images and the
> number of cucumber classes.

I doubt this is a one man operation in just a few months of time, considering
the computing power that's required to accomplish such feat (which it was said
to take 2-3 days for each complete training set). Let alone the fact that they
admit 7000 cucumber images are NOT enough to train the model to achieve the
typical 95% accuracy that the original MNIST model it was based off.

Although promising, this blog post reads more like a PR campaign for GCP's ML
offering (with the custom hardware devices to do the compute, perhaps TPUs?)

~~~
Eridrus
I just wish they'd move their CloudML thing out of alpha, I've given AWS too
much of my money for their shitty GPUs.

~~~
jonbarker
What will the pricing for CloudML be? It will be difficult to bootstrap
anything without a free tier of some sort.

~~~
Eridrus
I would be super surprised if it wasn't at least cost competitive with
AWS/Azure GPU instances.

They didn't approve my request for alpha access though, so no real insight on
my end.

~~~
jonbarker
What can you guess are their criteria for approval? I applied as well.

~~~
Eridrus
Probably how likely you are to spend a significant chunk of change on their
cloud offering

------
dangerlibrary
tl;dr: With a training dataset of over 7000 images, the system is incapable of
performing the cucumber sorting task on live data with greater than 70%
accuracy - likely do to over fitting on the "small" training set (where it
shows 95% accuracy). Also, it is currently incapable of taking into account a
number of the existing variables used for sorting.

If one were to leverage the power of a supercomputer trained with tens of
thousands more observations - all meticulously hand categorized - it would
probably get better.

Truly, we are only limited by our imaginations.

~~~
TN1ck
I think he can easily extend the system to use the missing variables. Scratch
marks, color, texture and prickles are easily enhanced by digital image
processing. Maybe use a different network for each feature and use this output
to feed into another network to get the correct classification-function.

~~~
dangerlibrary
If you read the article, the majority of the limitations I describe are the
result of his running the system on desktop hardware. It takes multiple days
to train the system using 7,000 80x80 resolution images. Hence the caveat
about the availability of a supercomputer.

~~~
llukas
Or he could stop training on CPU and use GPU like pretty much everybody in the
field...

------
clickok
At first the headline made me skeptical because I expected it to be one of
those clickbait articles where someone deploys excessive firepower solving a
trivial problem. However! It turns out that cucumber sorting is well-suited to
deep learning: a) there is enough interest to generate a reasonably-sized data
set, and b) It is one of those difficult "I'll know it when I see it"
classification/evaluation problems.

However, as soon as they described the actual solution I felt somewhat let
down.

1\. Reducing the input to 80x80 images is likely unnecessary; it removes some
of the most important features (number of thorns, blemishes) from contention.
If the issue is computational cost (and therefore training time) you might
consider using cloud GPUs. However, a network with multiple "towers" (like
Google's Inception model) could conceivably allow for evaluating the sorts of
aspects that require higher resolution relatively cheaply (one-or-two
convolutional layers with pooling and then a fully connected layer) along one
pathway, while a different tower (with more layers or more units per layer)
processes the resized images.

2\. If the network used was just a modification of the MNIST one, then it
seems like it ignores the ordinal nature of the problem. If cucumber quality
can be ordered, why not use a regression model instead (or maybe regression
along multiple criteria, like straightness, bumpiness, thickness, etc.)
instead of treating each class as if they were completely separate?

3\. Why not release the dataset? I have GPUs, the necessary background, and a
few hours to try out some models. I also have a heretofore unrecognized yen to
use my models to generate images of the ideal Japanese cucumber, so I'd be
willing to do it for free.

~~~
dmix
This is just a prototype - and an effective one at that because as you
demonstrated it's a good starting point to brainstorm about where this
technology could go if someone invested real time into developing it. So I
wouldn't over analyze this.

I personally think it's exciting to see average-person real world use cases
for these new deep learning tools. I wasn't going in expecting a fully-fleshed
out system. It's just one guy helping out his parents with a summer project.

------
mamon
Am I the only one that thinks this is grossly overengineered solution?
Wouldn't the simple machine with differently shaped and sized holes do equaly
good or better? Existing industrial machines are usually pretty good at
mechanical sorting (think of things like shelled sunflower seeds, etc.).

~~~
pmarsh
In the article they talk about many factors that a mechanical sorter will have
no ability to account for, color, prickliness, curvature.

------
dividuum
As someone who is using the Raspberry Pi professionally, I'm nervous when I
see that rainbow square in the top right corner in both their screen snapshot
and the video (0:16 in). It's a sign that the Pi doesn't get enough power
which might results in an unstable operation or even SD card corruption. I
hope there's still a human backup sorter should the system fail :-)

~~~
narrator
My Raspberry PI corrupts data if the power goes out unexpectedly. I don't
understand why it's used for any non-hobbyist application, at least if using
the internal SD card to do a lot of read/write. I got myself a fanless cherry
trail based system to replace it. Some guy did a hardware modification to fix
his: [http://therandomlab.blogspot.com/2014/04/how-i-solved-my-
sd-...](http://therandomlab.blogspot.com/2014/04/how-i-solved-my-sd-card-
corruption.html)

~~~
dividuum
I fully agree. That's why I've invested lots of work into my digital signage
os ([https://info-beamer.com/hosted](https://info-beamer.com/hosted)): It's
booting into a read-only squashfs that contains everything. The squashfs,
kernel and PI firmware itself are on the read-only FAT partition. All data
that is required as display content ends up on a second read/write partition.
If there is any data corruption or similar problem, my system can fully
restore that automatically. Here's a small blog post were I wrote about this:
[https://info-beamer.com/blog/building-a-reliable-
raspberry-p...](https://info-beamer.com/blog/building-a-reliable-raspberry-pi-
distribution). So far the system has never had any problem from power loss or
cutting power. Interesting hardware hack you linked there. I haven't seen that
one before. Thanks.

------
contingencies
If anyone is interested to work in the food tech / image processing area, we
are an early stage startup[0] building small scale automated food preparation
robotics and have an image-processing focused industry-experienced
Mechatronics PhD as an advisor. We are based in the most natural / pleasant
part of China, Yunnan.

[0] [http://8-food.com/](http://8-food.com/)

------
zerop
Similar one.. This AI bot from Spanish start-up removes plastic from
garbage...
[http://www.sadako.es/?page_id=33&lang=en](http://www.sadako.es/?page_id=33&lang=en)

------
dharma1
I don't understand why people say there is no way a non-Google engineer could
do this. It's not rocket science... This farmer obviously has an engineering
background, either as a job or a hobby.

The electronic/mechanical engineering of the sorter is actually much more
demanding than training tensorflow on a custom data set - which is a fairly
trivial task.

Anyway, cool work!

------
eva1984
While it is very cool...I cannot help thinking how many sorters are going to
lose their jobs because of this. Scary.

~~~
vthallam
But that's not how progress is made. This whole concept of machines taking
over has been happening and will happen more. Humans are good at adapting to
circumstances, learning new skills and doing more productive work. So guess
it's not that scary.

~~~
kafkaesq
_This whole concept of machines taking over has been happening and will happen
more._

But what seems different with the AI "revolution" is that we may be seeing a
hockey-stick rate to these changes that simply has no historical precedent. In
which the "progress" (read: forced obsolescence) that might hitherto have
happened over the course of 1-2 generations may soon be happening in the space
of 5-10 years, for large categories of job descriptions, planet-wide.

 _So guess it 's not that scary._

Unless you're the one whose job is threatened -- or as is apparently the case
of this "success story", eliminated already. Do you really think all these
displaced cucumber sorters can find work as Machine Learning engineers at some
hot startup?

Or in the near term, any kind of job at all, without significant difficulty?

~~~
35bge57dtjku
You're right. We should all go back to using horses instead of automobiles
while we're at it.

~~~
thevardanian
Don't be obtuse, and address the point. How are you going to retrain an entire
generation of people that will experience the effects of joblessness
immediately; essentially overnight.

Cars took at least a generation to become mainstream. This time around it
seems technology will destroy jobs almost at the flick of a switch.

Unless you're arguing that the adoption rate will not be significantly quick,
which I think most would disagree with.

~~~
35bge57dtjku
No one is, just like we never do. What have all the people who have been
losing their factory jobs for the past 2 decades done?

> that might hitherto have happened over the course of 1-2 generations may
> soon be happening in the space of 5-10 years

It's been happening in that shorter timespan for a while now, so really, what
have those people who have already lost their jobs done?

~~~
thevardanian
You have two options

1) Either technology is truly revolutionary, which means that it has a
revolutionary change on society by producing an entirely different dimension
of techniques and ideas that make everything in the past irrelevant.

or

2) The technology is incremental and will only make tasks more efficient, and
only has a marginal change on society by producing newer linear techniques.

If it's (1) then in the long term there is definitely a huge benefit, but in
the short term there's bound to be a great social upheaval. Ones that change
the shape of governments, and throw society in chaos. Just look at the
industrial revolution, or agriculture.If it's (2) then really it's merely a
gradual process that almost goes unnoticed, save for in retrospect. Which
makes it very straightforward to change and redirect people through education.

So pick either case. You can't have both revolutionary technology, and gradual
social change. Either the tech is truly revolutionary which in that case will
result in social revolution, or it's incremental and will result in merely
incremental changes.

~~~
aninhumer
Well sure, if you define "revolutionary technology" as something that causes a
social revolution, then something has to be both or neither, but that's a
tautology.

And besides, no one was arguing that the technology was "truly revolutionary",
so I'm not sure what point you're making?

------
TYPE_FASTER
Produce sorting machines are fascinating.

[https://www.youtube.com/watch?v=Lz88nsWL4kw](https://www.youtube.com/watch?v=Lz88nsWL4kw)

------
ars
Completely off topic, but why is having lots a prickles a positive thing in a
cucumber?

Would that just meant they have to be cut off before you can eat it?

------
quikoa
Any chance that the dataset could be released? It could be interesting to take
a look at.

~~~
BukhariH
The dataset has been released already:

[https://github.com/workpiles/CUCUMBER-9](https://github.com/workpiles/CUCUMBER-9)

~~~
quikoa
Thanks! I was hoping for the originals, the 32z32 is just very small.

------
blazespin
What I like best about this is he uploaded his dataset:
[https://github.com/workpiles/CUCUMBER-9](https://github.com/workpiles/CUCUMBER-9)

------
omarforgotpwd
What an eye opening blog post. I never thought about a production line /
manufacturing as a software process.

------
benten10
" ...First they came for the Cucumbers, and I did not speak out—

Because I was not a Cucumber ... "

-Song of those Tensorflow'd out of their jobs.

[Edit: Changed Cucumber sorters to cucumbers to clarify the sarcasm]

~~~
dangerlibrary
Honestly? Cucumber sorting sounds like a shitty job. Very few people are
picketing because humans aren't spending large swaths of their lives churning
butter.

But the tone of the article is ridiculous. For any reasonable definition of
"it works", the system doesn't work yet.

~~~
kafkaesq
_Cucumber sorting sounds like a shitty job._

No one's saying it's a great job. But the point for many of these people is
that it's _a job_. Which they won't easily be able to find a replacement for.

------
coldtea
> _How a Japanese cucumber farmer is using deep learning and TensorFlow_

Let me guess: shallowly and for marketing purposes...

------
kafkaesq
_TensorFlow democratizes the power of deep learning_

I won't say this kind of innovation is bad, per se, or should be stopped. But
the blitheness with which the article sidesteps the collateral damage that
will necessarily come of this brave new era of efficiency and progress -- as
if there was nothing more to this story than an unemployed engineer helping
out Mom and Pop -- really is quite breathtaking.

"Democratized unemployment, and for many older blue-collar workers in years to
come, a nearly inescapable sense of hopelessness and despair" is what we might
want to call it.

~~~
dmix
This is a technical post on a Google blog for a software library highlighting
a use-case written by a "Developer Advocate" for developers to read... this
isn't a NYTimes human interest piece about a new technology written by a
journalist for average humans to read.

Regardless of the non-relevant nature of the context, the HN crowd (the target
audience) is well aware of the side effects of technological progress.

Similarly, we don't expect every blog post by security researchers
highlighting the latest software exploit to include blurbs about the societal
implications of vulnerabilities on privacy. There's a time and place for that.

~~~
kafkaesq
Actually, the whole point of the blog post was that it was a _marketing_ piece
-- not a dry research article.

