
Breast cancer detection in mammography using deep learning approach - rusht
https://arxiv.org/abs/1912.11027
======
neuro_image3
There are several key points that get left out in AI radiology conversations
such as this one:

1) Mammograms are not interpreted in a vacuum. In fact mammograms are usually
the first in a long line of tests before a breast cancer or other diagnosis is
ultimately made. In fact, it's probably more accurate to refer to mammography
as a screening exam for which patients need a biopsy rather than a diagnostic
test for cancer (there are rare exceptions, but overall this point holds).

2) Speaking frankly as a radiologist myself, tests like mammograms aren't even
that good in terms of overall diagnosis. Thats why ultrasound, tomosynthesis
and MRI are often used as supporting evidence and/or alternative exams.

3) There is controversy over the overall utility of mammograms, particularly
in the screening context. Radiologists more than anyone would like the
sensitivity and specificity of these studies to be higher.

It strikes me that the people that push these "radiology is ripe for
disruption" or "AI outperforms radiologists" hyperbolic arguments are clearly
people that have never seen the inside of a clinic. I'm sure they love this
rhetoric though when pitching to VCs or sitting around the conference table
coming up with 'breakthrough ideas' to turn into power-points for the other
administrators.

~~~
cameronfraser
It could be useful as a tool to help a radiologist do their job better though.
I think many of these techniques described in ML papers will be used to enable
people to be better at their jobs rather than replace them. At least until
there is AGI at least.

~~~
neuro_image3
I wouldn't dispute this (if they finally put something together that isn't
horrendously cumbersome, time-consuming and hard to use) but this doesn't
justify the 'AI is about to replace radiology' crap I seam to see every time
some academic group publishes an AI/ML paper.

~~~
cameronfraser
I don't think this paper makes that claim, you might be thinking of media
interpretations of papers which is usually the one making bogus claims. It
says that it outperforms 5 out of 5 people, but that is in this specific
context. They aren't necessarily claims meant to be generalized that much.

~~~
neuro_image3
I see that type of hyperbolic claim several times in the comments.

I wasn't just commenting on the abstract presented. I was commenting on the
comments I see here, as well as comments I see related to similar papers all
the time.

I also interact with AI/ML researchers all the time. Most of them are
typically some combination of: 1\. Poorly informed about the appropriate
context and utility of medical imaging. 2\. Trying as hard as they can to push
AI/ML as the most important technology in medicine today. 3\. Pursuing a very
task-specific project which they claim is massively generalizable in some
(incorrect) way.

~~~
StreamBright
Usually they do not understand that your workflow if not revolving around
pattern recognition on pictures. The best use of ML in pattern recognition for
radiology is ordering images. You would get the the images sorted by the
likeliness of having something unusual.

------
RcouF1uZ4gsC
Of the major specialties, it seems that radiology is the most in danger of
significant disruption. First of all, it can be done remotely, so there is
risk if the regulation is lightened that foreign radiologists will be allowed
to read studies at much less cost. The other issue is that this is something
that deep learning can rapidly progress in given there are already a plethora
of labeled data sets. For example, every mammogram that is taken has already
been labeled normal or abnormal.

~~~
Gatsky
I don't know anywhere that has a surfeit of radiologists just sitting around
chewing gum. The types of tools described in the paper will enter the market
as radiologist assisting devices, if for no other reason that the regulatory
burden for this is dramatically less than a total radiologist replacement.
Their introduction will make radiologist's jobs easier, and improve scan
throughput and reduce waiting times for results. This will free up
radiologists to deal with more complex cases. But I doubt it will materially
reduce the number of radiology jobs, at least in the next generation.

~~~
jayd16
Is this different than any other illness that became easily testable? Wouldn't
this eventually move into something a technician would perform, like a blood
test?

~~~
braindeath
Technicians don’t interpret blood tests, and radiologists don’t perform most
imaging procedures, so I don’t understand the comparison.

~~~
jayd16
Well my thinking is more that the results would be read by your GP and not a
specialist.

------
mikece
I imagine this is just scratching the surface and before long we'll be doing
full-body, 3D scans every few years and everything from cancers (all of them)
to heart disease, to gastero-intestinal issues, to things even as mundane as
acne and dandruff will be diagnosed by algorithms pulling on a cumulative
database of images of healthy and diseased body parts. The real hope is to be
able to see into the brain and pick up things like CTE and Alzheimers __years
__before symptoms manifest.

~~~
Gatsky
This is the wrong idea. Detecting pathology early and then treating it is
going to be complicated and expensive, and available to only a few people for
a long time. We don't actually want some kind of constant AI body surveillance
(for CTE?? So an NFL player can take just enough knocks before they retire?).

What we want is prevention, not early detection. Bear in mind that prevention
is a proven outcome, because some people never get Alzheimers, and some people
never get cancer. Detecting cancer or Alzheimers early and then intervening to
cure the condition is actually not something that is definitely possible.

~~~
dwelch
While I think I know what you’re getting at, there are of course examples of
“detecting pathology early and then treating it” preventing morbidity,
mortality, and saving money. This is the basis of all (good) public health
cancer screening practices. Think of a screening colonoscopy identifying a
local pre-invasive adenocarcinoma and cutting it out (cure).

~~~
Gatsky
There are specific examples for high prevalence conditions as you cite, but
the original comment was not talking about that, it was talking about a
broader approach.

Even taking your example, getting to the point of removing a pre-invasive
adenocarcinoma is actually very complicated. You need a motivated patient, you
need a healthcare team of specialist doctors or nurse practitioners,
pathologists, radiologists, anaesthesiologists, nurses, an endoscopy suite, a
recovery area, you need a system to follow up and track these cases etc. The
full embedded cost of this undertaking is huge, I don't think it is scalable,
and at the same time, will never capture everyone. Randomised studies usually
just barely show that these screening approaches are better than not
screening, once false positives are accounted for. Furthermore, you have
chosen an example where there is a useful pre-screening test (fecal blood
tests), a relatively non-invasive diagnostic test (colonoscopy) and a
relatively painless intervention (removing a polyp with the scope). This is
not broadly applicable. A pancreatic lesion for example is very hard to
diagnose for sure, and the intervention is a massive and life changing
operation.

------
statesdj
Instead of trying to squeeze blood from the mammography stone that has failed
to improve longevity in breast cancer patients despite enormous investment
over many years, AI/ML need to take a broader perspective and look at
modalities like circulating tumor DNA and 3-D ultrasound

~~~
avocado4
Why those two in particular?

Especially 3D US - so far it's mostly been used for "cosmetic" purposes of
getting a 3D picture of your baby. There hasn't been much clinical evidence to
its usefulness from what I can tell.

------
yellow_postit
As the authors rightly call out in the abstract _obtaining large amounts of
annotated data poses a challenge for training deep learning models for this
purpose_ But doesn't appear that the data they've collected and annotated is
made available from my read of the paper, I get that this is from a company
(DeepHealth) but it seems like an opportunity for NIH to push for more broadly
available data sets.

Anyone have a good reference point for the reader selection of 5 specialists
with 5.6yrs avg experience? That population seems small. Another opportunity
for licensing bodies or national institutions to grow a publicly available
dataset -- including annotations from a wider selection of imaging
specialists.

~~~
1e-9
The radiologists in this study had read 6,969 mammograms on average over the
preceding year. That's about 15X the certification requirement and 4X the
average for U.S. doctors. Reading volume is one of the main predictors of
performance[1], which suggests that these doctors were probably above-average
readers. It would have been nice to see more readers involved, but reader
studies are a major effort. Even with the small sample size of readers, these
results were statistically significant as well as clinically significant.

[1]
[https://www.ncbi.nlm.nih.gov/pubmed/21343539](https://www.ncbi.nlm.nih.gov/pubmed/21343539)

------
travisoneill1
I see a lot of machine learning work on medical imagery, which is great, but
it seems like this is solving a problem that the human brain is already pretty
good at (image recognition). I wish I saw more work being done on finding
patterns in medical data in numeric formats which the human brain is terrible
at. Is there much of that going on?

~~~
jcims
You're absolutely correct but from what I can see there's going to need to be
a lot more work done in collecting and standardizing that data before it's
available in sufficient quantity and quality to do anything with. I think a
more aggressive approach to normalizing externalities (primarily regulated
diet/nutrition) would help as well.

There's another completely (to me) unintuitive angle as well. Andrew Lo and
some folks from MIT Sloan have published a paper about using clinical trial
data to predict which medicines will be approved by the FDA in order to help
reduce investment risk and unlock dollars. He does a pretty good talk about it
here -
[https://www.youtube.com/watch?v=AzELyaVf0v8](https://www.youtube.com/watch?v=AzELyaVf0v8)

He's on a recent episode of Linear Digressions discussing this as well.
[http://lineardigressions.com/episodes/2019/12/8/using-
data-s...](http://lineardigressions.com/episodes/2019/12/8/using-data-science-
to-make-hard-prioritization-decisions-behind-the-scenes-of-an-analysis-to-
predict-drug-approvals)

------
jaxr
Got me thinking... How long till AWS DeepPhysician? They don't seem to have a
clear cut on the limit of the scope of their services. Joke apart, what would
be the implications and responsibilities of big tech entering the medical
field?

------
1e-9
Excellent results with good generalization. The study appears to be well
designed and executed. This was a significant effort. Clearly, there are
commercial intentions.

------
sergers
Computer assisted screening for mammo has been around for years... like icad I
am sure some of the vendors in this space are using some form of deep learning
already

------
joe_the_user
Everyone swooning with optimism over this result should the machine learning
reddit comments on it first.

[https://old.reddit.com/r/MachineLearning/comments/ehpllt/dee...](https://old.reddit.com/r/MachineLearning/comments/ehpllt/deep_learning_model_for_breast_cancer_detection/)

And also linked blog[edit]:

[https://lukeoakdenrayner.wordpress.com/2017/12/06/do-
machine...](https://lukeoakdenrayner.wordpress.com/2017/12/06/do-machines-
actually-beat-doctors-roc-curves-and-performance-metrics/)

TL;DR; There are sooo many subtlties to stuff like this that this things
really shouldn't be taken at face value. This is far from replacing doctors in
anything.

~~~
1e-9
1) I see no credible criticism on reddit.

2) I see nothing in the Luke Oakden-Rayner blog that calls this study into
question. This study actually _avoids_ the pitfalls that he mentions.

3) This paper said nothing about replacing doctors.

------
mscasts
This is simply amazing. Great job to anyone involved in that project!

------
rogerdickey
Dr Sausage has had this technology for decades

------
kevinalexbrown
If you find this kind work interesting, our AI group at Siemens Healthineers
is hiring interns to carry out projects like this. We typically target machine
learning or medical imaging PhD students, but are open to a variety of
backgrounds. Please feel free to reach out via email.

~~~
selimthegrim
Physics PhDs?

------
256lie
The Clincal Center for Data Science at Massachusetts General Hospital (one of
the top hospitals in the world) is hiring for a variety of positions. We have
access to tons of medical data (imaging, NLP, time series), clinical domain
expertise, and one of the largest GPU computing clusters.
[https://www.ccds.io/careers/](https://www.ccds.io/careers/)

~~~
abrichr
I'm curious, how is the compensation, e.g. compared to the lowest levels at
[https://www.levels.fyi/](https://www.levels.fyi/) ?

------
oarabbus_
It will be a great day for individuals everywhere when we automate away 75% of
MD/DO jobs. It won't be a great day for the AAMC, and I also look forward to
seeing how they respond.

~~~
natalyarostova
The cotton gin increased demand for labor to use the tool. You have to think
very carefully and subtly to even have a chance at predicting second order
effects from more complex forms of automation.

~~~
oarabbus_
I worked for a surgical robotics company as well as other medical device
companies, and procedure lengths and surgical wait times both decreased in
facilities with the device. Based on my experience the danger is in areas like
"will we over-prescribe imaging tests even more than now?" but as to whether
we'd need less doctors to treat the same number of patients, the answer is
yes.

~~~
natalyarostova
If imaging tests become higher quality and cheaper would the concept of over-
prescription of them still make sense? (Asking sincerely, I don’t work in
medicine)

~~~
oarabbus_
It is still dangerous, as x-rays are carcinogenic, and also the chance of
false positives can adversely affect patient health.

------
brooklyndude
And as my MD will tell you, voodoo! No one can ever replace me. No "robot."
:-)

