

Your doctor doesn't want you to see this -- diagnosis using AI - cmonsen
http://symcat.com

======
parfe
How does this use "AI"? Right now it seems like a less robust version of
WebMD's multiple symptom checker.

<http://symptoms.webmd.com/coresc/landing#/introView>

For example, I can click Arm -> Armpit -> Lump or Bulge and see the possible
conditions.

Armpit doesn't even exist in your system. Which would be fine if I already
knew how to self diagnose a swollen lymph node. Of course, if I didn't, I
would still need a doctor to tell me what search term to use on your site.

I can understand if the system is early in development, but if it doesn't do
anything for the layman why would my doctor care if I saw it?

~~~
cmonsen
We've started with the most common symptoms, but are building it out more.
We'll add the armpit, thanks for the suggestions!

The AI component is the fact that we train our algorithm based on clinical
data from real patients (started with CDC data -- 500k at the moment and
counting as people use the site).

I've had bad experiences with the WebMD symptom checker like most people I've
spoken to. Try typing in "chills" and you get "Lyme disease, acne, Bubonic
plague" (no joke).

WebMD makes money on advertising, so they unfortunately direct you to the
pages that are going to keep you on the site for longer. You may find what you
have using their symptom checker, but it is really optimized to keep you
clicking.

~~~
Ralith
Expert systems have a history of being used very effectively in this domain.
Have you considered incorporating one?

~~~
cmonsen
We'd love to incorporate a DXPlain or QMR as a starting point, but the true
value of this is in learning from additional use/data.

------
karamazov
This title is some of the worst linkbait I've ever seen on HN. It reminds me
of so many low-quality ads on blogs going "one mom discovered this trick that
doctors don't want you to see"; please consider not advertising your site this
way.

~~~
jasonkolb
Agreed. One of the things I love about HN is its general resistance to run-of-
the-mill marketing techniques like this headline.

In fact, it makes me wonder if the submitter of this article is an affiliate
for the site this story links to. His only activity is made up of 3
submissions that seem to be related to this site
<http://news.ycombinator.com/submitted?id=cmonsen>

It's an interesting site but it's sad to see HN being gamed like this.

~~~
Geee
He is apparently the founder of the startup.

------
DanBC
I entered "lump on testicle".

I was surprised at the results. I was disappointed that there wasn't some
gentle friendly persuasive advice about going to see a real doctor quickly.
Yes, I know there's a disclaimer on the front page, but I feel this is a
missed opportunity.

You're going to tap into that hard to reach market of people who _might_ have
a serious problem but who are reluctant to see someone. You have a great
opportunity to educate them; to let them know that there's probably nothing to
worry about but that early intervention is crucial; and that this is what
happens at these types of examinations.

But: It's a really cool tool, and I look forward to seeing how it develops!

~~~
cmonsen
Yeah, we have about 1000 symptoms, but that is an example of one we haven't
incorporated yet. We'll add it, thanks for your help!

That's exactly where we're going. We want to direct patients to the _right_
doctor/level of care and are incorporating triage guidelines towards that end.

Thanks!

~~~
cmonsen
Ok, fixed that. The results still need some work, but it should be much better
than what you were getting before.

------
iterationx
I used WebMDs version of this last year, and then the page started flashing
"Seek medical help immediately", my appendix was taken out that night.

~~~
cmonsen
Glad to hear it worked out for you! Unfortunately, for every person that are
helped by medical information, there are 5 who are caused unnecessary anxiety.
We want to give patients access to data that lets them know when they should
be concerned and when they should not be concerned. Microsoft has a great
study on this (try Googling "cyberchondria microsoft research")

We're still building out our recommendations, so you can expect ours to also
alert people to see a doctor when appendicitis is likely.

------
pbhjpbhj
The UK NHS has a pretty good expert system, like I assume this is supposed to
be.

I entered (but do not have) "bloody feces" as a test case, it gave me 4
options for symtoms that they think I meant, none even anywhere close. Hint:
if they're black and stinky you need immediate medical help, if the blood is
red and sparse it's probably piles - THIS IS NOT MEDICAL ADVICE.

Is this an alpha test?

~~~
cmonsen
Re: NHS Direct, I've seen it and agree that it's a cool tool. It's pretty much
restricted to triage and doesn't give you a great sense of what you have,
though. Also, only helps in the UK. :(

There's a good paper on it that explains most people feel that it just directs
them to a primary care physician. We'd like to offer value to those people as
well and match them to exactly the physician who can help them (ie OKCupid for
patients-doctors).

------
rdouble
This has the same problem as hypochondriac internet forums.

I put in "tired" and got a list of: depression, diabetes, hypothyroidism,
atrial fibrillation,anxiety ,multiple sclerosis, congestive heart failure and
gastroesophageal reflux disease.

If I click on "increased risk," I get: valley fever, muscular dystrophy,
hemolytic anemia, intracranial abscess, thalassemia, gallbladder cancer,
polycythemia vera, and autonomic nervous system disorder.

Nowhere does it ask me if I happened to get enough sleep...

~~~
cmonsen
The data comes from ED and outpatient doc visits so it "assumes" a certain
level of concern from the user at the outset. However, the algorithms can
learn so that less severe diagnoses ("lack of sleep", "caffeine withdrawal",
etc) will rise to the top as more people indicate that is the problem.

~~~
tansey
Couldn't you just add a prior probability that someone has any problem? Going
to the ER is much more difficult than typing something into a web app, so you
should use your expert knowledge to adjust that.

It would also seem like a better approach would be to have a prior that's much
lower for the internet and is never updated based on user searches, then
continue hand-entering real records for quality control purposes. Otherwise,
sooner or later some internet community (4chan, SA, etc) will get carried away
and start manipulating your system to make everyone think they're dying of
elephantiasis.

~~~
cmonsen
Agreed, and that's a very reasonable place to start for conditions that are
not challenging diagnoses but rather explanations for common symptoms (eg too
little sleep, bumped my knee, etc).

WRT abuse, we're going to require the log-in and user consent before
collecting any data and allow people to maintain, in effect, their own medical
record. We can monitor for abuse (eg someone who seems to visit a hospital
thousands of times / day).

------
kiloaper
At a bare minimum I would except the site to default to HTTPS (with no
warnings). Got an "Untrusted Connection" when trying to use HTTPS. That's not
a good sign if you want people to enter personal medical information.

~~~
cmonsen
Thanks for the suggestion. User privacy will be critical and we are making
that a priority.

------
_k
I've been thinking about systems like this because I met way too many
pretentious docs and on top of that lots of docs misdiagnose and the older
they get, the less motivated they are to stay up to date on new drugs.

One word of advice though. Hating on docs might be a good idea PR-wise, but
only if you're going after patients who hate on their docs, but that's not
your market, your market is the docs. This industry is always going to be
regulated and that's a good thing. So, don't try to fight them, join them and
help them make money. It doesn't take a brain surgeon to know this could be a
money maker for them.

You also need all the data you can get. Data on symptoms, diseases, drugs,
side effects, off label usage, ... You need the FDA, you need the docs and you
need the patients. While docs are all about sharing data, they only do so
within their community. That's the part that needs to open up. And stay open.
The patients are the easy part, they know sharing data is going to help them.

Docs aren't against systems like what you're trying to build. Systems like
this are already being used when you get PET scans. There're hundreds, if not
thousands of spots on the photos and an algorithm helps them narrow down the
results. It's a huge time saver and it obviously helps the patient. The
money's in making sense of all the data and offering solutions.

You will also be dealing with companies like GE. An example. The docs are your
friend, so let's say they measure your bone density and they're willing to
give you the results, what do you think GE is going to do about that? You're
in their market and they will fight you with package deals! You'll be up
against Google as well because they will offer the docs a free version of your
product, ad supported.

Your system needs more data. I tried it but the diagnosis was wrong, stayed
wrong and eventually no diagnosis was given. You need photos. You need all the
help and data you can get.

~~~
cmonsen
The evidence suggests that 15% of medical diagnoses are incorrect. Being 2
months away from finishing my MD, I have a sense that this is not because
doctors are failing to do their job, but rather, because medical diagnosis is
hard.

Most people believe that their doctor gives them a diagnosis and are 100%
certain of it, but in reality medicine is a highly probabilistic domain.
There's a great article about a family's story with misdiagnosis in their
child
([http://www.slate.com/articles/double_x/doublex/2011/07/how_f...](http://www.slate.com/articles/double_x/doublex/2011/07/how_facebook_saved_my_sons_life.html)).
The diagnosis was initially strep, but it soon became evident that the
diagnosis was the much more rare Kawasaki disease. Symcat is an attempt to let
patients know about both (ie "This is probably strep, but you should continue
to monitor because Kawasaki disease can also cause this and the only way you
know is if your symptoms persist").

We would love to get more docs on board and we've been working with physicians
at Hopkins to make this as useful to patients as possible.

Completely agree that this lives on the quality of the data. We would like for
users to contribute their data (a la patientslikeme). We also are exploring
partnerships with health care providers and data stores for getting additional
data.

------
masterponomo
I think "House" should end with him and his team being replaced by a computer.
It will need to be a robot (like the one on Lost in Space, ideally), not a web
page, but symcat has the right idea. It would also be cool if the robot was a
master game-player, finally putting House in his place.

~~~
cmonsen
I love House. I'm not sure I'm ready to replace him just yet.

~~~
masterponomo
You do know this is the final season of House? Would you rather see them
continue to turn Chase into a Mini Me version of House (getting scruffy,
painful injury, ignoring protocol) or witness a battle for relevance against a
robot with smarts like the Watson computer? I'll take the robot battle.

~~~
cmonsen
I didn't know that. I have to get out (watch TV?) more. Robot battle it is,
then.

------
slavak
I find it amusing that when specifying a symptom of "Mass in scrotum" the
website sees the need to ask me if I'm male or female.

~~~
cmonsen
Yeah, we're working on that. :)

~~~
pamelafox
Ditto with everything related to menstruation-- although actually I have a
sister/brother who's transitioning from female to male and is still
menstruating. So, uh, there's some complexity there.

------
brohee
Needs SSL badly. I don't want my proxy administrator to know that it burns
when I pee.

------
nmcfarl
You are right - she doesn’t. My wife, a 4th year med student blanket opinion
of self-diagnostic websites for the general public is they are trying to kill
their customers. And more and more I agree.

If you are ill and you don’t know what’s going on, see a doc (or a nurse
practitioner etc). Uninsured (yeah - that’s me till 2 months ago.) - then it
will cost a packet. Which is good as you’ll be encouraged to figure out what’s
normal for your body - which is what needs to happen anyhow.

Remote diagnostics is a hard game even if a doc is doing it. A rules engine
just isn’t up to it. It’s not just a game of checklists. This thing might be
better (It’s down right now) - but nothing is as good as being in a room with
the patient, having some EQ, asking questions, and being able to view symptoms
with your own 2 eyes.

~~~
cmonsen
Sorry that it's down. Getting a lot of HN traffic.

I tend to agree with your wife (I'm a 4th year med student at Hopkins), which
is why I'm building my own.

I think it's easy for websites to just say "if you are concerned, see a
doctor" but I also believe that to be the problem. Obviously, if it were
costless (time, money, opportunity cost) people would go see a doctor, but
they in reality are weighing those costs. We'd like to make it easy for people
to weight the risk vs. the benefits of seeing a doctor.

Moreover, for too many people "seeing a doctor" means going straight to the
ER. There they wait for 4 hours (nat'l average) and are frequently told to
follow-up with a doc anyway. We want to let others know that there are other
options (eg urgent care centers) and let them know when such care is
clinically indicated.

~~~
tomwalker
As a doctor I think it is a great idea and I think there are many sources of
automation that haven't been implemented in health care yet.

Learning medicine is all about learning diagnostic algorithms

------
cmonsen
We're Hopkins med students with a new startup. Would love to hear your
feedback.

~~~
chime
<http://symcat.com/conditions/lumbago> does not seem correct to me. I typed
back pain and got linked to "I may have lumbago" which talks about mastectomy.

~~~
cmonsen
Should be fixed now. Thanks for letting me know!

------
davidjhall
I just had my appendix taken out a few weeks ago and I tested this out with my
symptoms, but it said that I had a 0% chance of appendicitis. I went to the
appendicitis page and added two of the recommended ones (fever and bloated
abdomen) but it only raised it to 1%!

Granted, appendicitis is pretty rare but I would've expected something higher.

On the plus side, it would've coached me towards Diverticulitis with a "Call
911 and go to the emergency room now" which would've led me to the right
place/right conclusion (at one point when I was in pain, I'd thought I'd sleep
on it but decided against it -- lucky for me, I didn't and went right to the
hospital ... when I got there, they said it was about to burst.)

------
modulusprime
This should be targeted toward two groups: doctors (site) and EHR vendors
(API). I also think you should get some expertise involved sooner rather than
later. Do a serious search and find some docs, including all the major
specialties, that are both good at their jobs and sympathetic to your goals.
No armchair MD's, use clinicians.

I have a side gig helping docs transition to EHRs. I started by helping an
immunologist family member back in 2003. She's on her second EHR now. So far,
her transitions have been the first and second most difficult of the bunch.

Right now it looks like SymCAT has the same problem as every EHR I've seen:
the immunology side of allergy/immunology must be considered explicitly in
your design, or the system will never work well. In immunology, you get weird
sets of symptoms, you get multiple problems presenting, you get unusual
systemic interactions and feedback loops.

For example, I just cruised through a few of the signs a less articulate
person (not a medical pro) with common variable immunodeficiency might
mention, along with a history of asthma. When I also _include_ my "itchy
scalp" (because you offered it in the "related" section) a bunch of potential
diagnoses are _excluded_. None of them were correct, but a few were on the
right track.

You're asking people to come up with their own symptoms and have a clue about
which terminology will work. In practice, a big portion of the difference
between a good doc and a mediocre/bad doc is the ability to construct a usable
history from poor reporting. Via interview. Patients often do not have the
tools to make connections (or ignore them) between symptoms.

That said, good luck. It's a great problem. If you get this right, it will be
huge. Again, I'd encourage a public side for marketing and a different kind of
input, but put your money on licensing.

~~~
cmonsen
Thanks for the suggestions! We are fortunate to have some great advisors
including practicing physicians, medical informatics specialists, and health
care entrepreneurs. We're part of Blueprint Health, a TechStars-affiliated
health startup accelerator.

You point to a challenge for any patient-facing patient decision aid --
getting the language right. We are building on the work of the U of Utah and
building a robust consumer health vocabulary based on query data.

~~~
modulusprime
Great! Sorry for adversarial tone; working in/around a lousy template
implementation today.

<https://twitter.com/aheadresearch> doesn't seem to exist.

~~~
cmonsen
Hoo boy, thanks for catching that! Was wondering why we weren't getting any
follows. Fixed now.

~~~
modulusprime
Still not fixed as far as I can tell.

------
aarondf
I'm sure that there is a great service behind this link, but I completely
ignored it at first because of the "your doctor doesn't want you to see this"
message. Seems like the acai (sp?) berry ads that are scattered all over the
internet.

~~~
cmonsen
Haha. My co-founder and I went back and forth about the right message for our
HN title. I completely agree with you but he won the argument. Does anyone
know of a good A/B testing site for HN titles? ;)

~~~
gcp
I also ignored this at first because of the title.

It's a good title for gossip newspapers. It's not good for HN.

------
gcp
I think its worthwhile to add some more self-consistency tests to this thing.

You can enter symptoms in the first question, and then move on to the second
question. At this point, regardless of the timing selected, some of the
symptoms drop off.

In my case, at the end, "What are my chances of having..." reported 0% for the
correct diagnosis. This is not good.

What is worse: the condition is in your database and described perfectly
there, including the details that would allow for a correct diagnosis.

Did you try starting from each condition, and measuring how often you get the
diagnosis right? Correcting for the prevalence of each condition as needed -
in my case its a common one so there didn't seem to by any reason not to get
it right.

Also, it would be nice if the conditions page would tell you how often this
condition happens (if needed by asking age group/sex).

~~~
cmonsen
We did a standard 80/20 training/testing trial. We found that the correct
diagnosis was in the top 5 of the generated possibilities 50% of the time.

What will be helpful is to update the algorithm from user input. I'd be
interested in hearing about your particular case. We use symptom/disease
prevalence in the calculations, so it is usually great at getting common
conditions. Maybe you can leave a comment using the feedback form on the
website and we can look into why it missed your diagnosis?

~~~
Ergomane
I entered the HIV triad.

It didn't ask me if I was MSM and/or had unsafe sex previously. As such, there
was no HIV seroconversion illness mentioned in the diagnosis list.

The recommendation (to see a professional) was correct.

------
adaml_623
I think it needs a lot of work. The interface is nice but it's missing a lot
of symptoms and rules. I think it is repeating expert system work that has
probably already been done.

In it's final result it should make a mention of urgency in seeking healthcare
and what to do in the meantime.

~~~
cmonsen
We'll be adding symptoms based on user input analysis and building out the
symptom synonym engine.

True, there has been a lot of work in medical expert systems. DXPlain is an
old one that tends to come up a lot in our conversations. We have several
Hopkins medical informatics specialists advising us and I would say that our
algorithm is novel.

Moreover, past attempts have attempted to codify a medical expert's knowledge
(eg let's ask this smart doctor we know to determine disease prevalence). We
calculated it from clinical data and can continue to learn as more people use
it.

------
davehdo
Whats the algorithm here? This is pretty neat--the future of diagnosis.

design needs some work

------
nswanberg
Very cool, and good luck.

Who is the intended audience for the symptoms and conditions directories? Are
the numbers next to the entries probabilities of having that symptom or
condition for the general population?

And not that it's particularly important, but this entry doesn't seem correct:
<http://symcat.com/conditions/arsenic-poisoning> (it discusses bacteria rather
than the element).

~~~
cmonsen
Thanks!

Honestly, Google. We need to make sure that it is easily indexed and
understood by crawlers. Some med students have found it useful though.

Will fix it, thanks!

------
EdiX
The description of lumbago (low back pain) is mastectomy:
<http://symcat.com/conditions/lumbago>

~~~
cmonsen
Fixed. Thanks!

~~~
bazzargh
The fix seems to have been to copy the definition of Chronic Back Pain. But
those have different diagnoses in your system (I'm seeing Lumbago as 14% and
Chronic Back Pain as 10% for the same symptoms)

------
epenn
I would recommend allowing a person to add multiple symptoms up front on the
landing page instead of just one. Requiring a postback and then having it to
calculate the probabilities under "I may have" after entering only one symptom
when the person already knows they plan on entering 2 or 3 more is a waste of
resources on the system end as well as a significant delay on the user's end.

~~~
cmonsen
That's a fair suggestion. The question is if "rewarding" people with a list of
diagnoses from just 1 symptom is worth the cost of resources. My fear is that
in the process of having people enter an arbitrary number of symptoms and then
having to click a "next" button, they will lose their interest, but we'll test
that.

------
russgraney
This is awesome! Looks like phone triage might be enjoying its last days. What
inspired you to start this project?

~~~
cmonsen
We started working on this during our ER rotation and were surprised how many
people were showing up concerned about things that were ultimately minor
complaints (eg cough, sore throat). Many of them had no primary care doctor or
just didn't know there were alternatives to the ER. Very little info on the
Internet helps these people so we're trying to fill that gap.

------
gravitronic
This is cool, but to be fair, the Rogerian psychotherapist version of this has
existed since 1965.

------
mrleinad
Combine this with Siri, and you have version 1.0 of the doctor for the Voyager
(Star Trek)

------
cjy
There are some bugs. I looked up some mouth symptoms I had and the entry on
Tooth Abscess talks only about depression:
<http://symcat.com/conditions/tooth-abscess>

~~~
cmonsen
Should be fixed now. Thanks!

------
tansey
Machine learning guy here. I'm curious: which AI model you chose on the back-
end?

------
abh_gh
Thought I would mention in Mycin (<http://en.wikipedia.org/wiki/Mycin>) in the
interest of completeness.

------
frobozz
This is even worse than the GP I had in my teens. At least he recognised that
my headache came from a primary headache condition, and wasn't Hay Fever.

------
kiloaper
For some reason I can't get to work in Firefox 10 or IE9 on Win7. Please tell
me this is not another webkit only site.

~~~
cmonsen
We've tested on both of them. What's your error message? This may be a
traffic-related problem...

~~~
kiloaper
I wasn't getting any error message but now I am. Must be traffic. Clearly
there's a lot of interest.

"Application Error

An error occurred in the application and your page could not be served. Please
try again in a few moments.

If you are the application owner, check your logs for details."

------
masterponomo
I tried the site with some minor nuisances in my sit-upon. It thinks I have
Crohn's Disease. This IS like Dr. House.

------
benshyong
this is pretty cool. The "find a medical practice" tool is a great use of data
on care quality. kudos for making this data easily accessible to the general
public!

~~~
cmonsen
Thanks. We're adding more medical practices this weekend.

------
tasooey
good work folks. keep it up

