
Willy Wonka and the Medical Software Factory - thomasjudge
https://www.nytimes.com/2018/12/20/business/epic-systems-campus-verona-wisconsin.html
======
henrikschroder
> Workers who fully commit to Epic — who survive the long hours and grisly
> sights — are treated to a remarkable perk. After five years, they get a
> sabbatical, including round-trip airfare for two to travel somewhere they’ve
> never been for a month, plus a per diem for meals and lodging. (They get
> another sabbatical after 10 years, 15, and so on.)

Obligatory snark: In Europe we call this paid vacation, and we get it every
year.

But on a serious note, it's so funny how experiences temper expectations. Only
in a place where most people's vacation time is used for familial obligations
or not used at all, does a month of traveling sound like the ultimate luxury.
And they're willing to slave away for _five years_ to get this?

When me and friends briefly served a stint as middle managers in corporate
America, we had a vacation policy of "always yes". And people would still
start their time off discussions by laying out arguments as to why they really
needed time off. No, stop, you need zero arguments to get a yes from me as
long as you have earned the time off according to company policy. It's my
fucking job to make things work despite your absence, not yours.

~~~
johnnyb9
The fact that it's a paid for trip is why it is worth mentioning, not
necessarily the length. Five weeks of vacation, while not very common, is
common enough that it's not really worth writing about otherwise. Edit: Agreed
that 5 weeks of _contiguous_ vacation is rare

~~~
arkades
> Five weeks of vacation, while not very common, is common enough that it's
> not really worth writing about otherwise.

Where do you work that you think this is common?

My wife and I are both in reasonably well compensated professional careers,
and while we get to take more vacations than most americans, a solid 5 week
block is basically impossible. And I don't think that's atypical: the Bureau
of Labor Stats certainly doesn't support that idea
([https://www.bls.gov/opub/ted/2018/private-industry-
workers-r...](https://www.bls.gov/opub/ted/2018/private-industry-workers-
received-average-of-15-paid-vacation-days-after-5-years-of-service-
in-2017.htm))

~~~
henrikschroder
In Sweden, five weeks of paid vacation per year is the legal minimum, and you
also have the right to take four weeks contiguous in the summer if you want.
But you don't have the right to choose _when_ exactly if you exercise that
right. Different industries do it differently, some shut down and kick
everyone out in July, others try to get their employees to spread their
vacation over summer so that there's at least someone in the office.

The difference in mentality and expectations is absolutely crazy.

------
maxyme
I interviewed at Epic Systems on-site and this article hits on basically
everything I saw and felt. It's a very interesting, unique and weird place.
But you also get a kind of weird vibe from it, it seems like a large facade.

A few things the article missed: The offices are shared by up to 3 people in
offices designed to have 1 or 2. About half the company is akin to consultants
where they fly out to a medical facility every Monday and fly back every
Thursday. The vast majority of hires are new grads and turnover is very high.
Oh and the article touched on this but the selling point for switching to Epic
software is that Epic has nearly every person in the US's medical history in
their databases (since their usage is so widespread) so if you sign up to use
Epic you get the medical history with it.

If you ever interview there I recommend trying to schedule the flight in on a
Thursday, your flight will have a transfer at MSP Airport and the next flight
will be all Epic employees whom you can ask questions and they can answer
honestly. Every employee I talked to on that flight said they planned on
quitting soon and didn't like working there. This was several years ago.

~~~
solarkraft
> Epic has nearly every person in the US's medical history in their databases
> (since their usage is so widespread) so if you sign up to use Epic you get
> the medical history with it

Why is this legal?

~~~
benmaraschino
While Epic produces the database/electronic health record system, the hospital
or health system ultimately owns the data. AFAIK patient data don’t sit on
Epic’s own servers outside of some research/pilot initiatives.

(Source: my personal experience working extensively with EHRs, including Epic
products.)

------
yhoneycomb
Epic is a steaming pile of shit. I hate the medical software industry so bad.
I don’t understand why the current EMRs are so bad. I’m in medical school now
but I hope to make a better EMR some day.

I’m upvoting this in hopes that someone with more knowledge than me can
explain why the EMRs are all terrible.

~~~
Insanity
You might enjoy this video then:
[https://www.youtube.com/watch?v=xB_tSFJsjsw](https://www.youtube.com/watch?v=xB_tSFJsjsw)

Now on to your question. A lot of the EHR systems have grown organically and
in the end they are made by people with software experience but not per se
medical experience.

Disclaimer first: I work on a competitor for Epic (Well, they only run in 3-4
hospitals in my country and we run in 30, so they're not really a threat). And
I'm not saying we are doing it better than Epic, but these are a few of my
thoughts (loosely collected).

1) Most of us don't have medical training. Our campus is shared by a
university / hospital, so we do have access to a lot of doctors and when they
work with us, their sides of the program tend to run quite well. But, not all
doctors want to spend time on this (understandable) so one system might be
well tailored for doctor A, but not to Doctor B.

2) The systems are _old_. They grow organically, but Epic is a few decades old
and our system started in 1992, so it's by no means 'young' either. The
Medical field evolves and so does the software field - we are playing catchup
on both most of the time.

3) We can't afford downtime. We can't just decide "upgrade tonight and go
offline for a bit". So we have to engineer around this. We release a new
system of our software every week (one week testing by a dedicated team with
medical knowledge, then production). But since we can't do a bulk update and
force people to restart, we need to deal with different versions which impacts
the code quality.

4) We run on multiple hospitals. Hospitals have their own way of working, but
to tailor to each hospital directly becomes impossible at some point. In the
end you get a system that doctor Y from hospital A really likes, but doctor B
from hospital A doesn't. Here, sadly, money will factor in. If you want the
software more tailored you will have to pay more. And this is not just to the
doctors to decide.

5) These systems are _huge_. (SVN reports a linecount of just over 37 million
at the moment). Because they do a lot, to give an idea: * Patient scheduling
(find optimal time for a patient to get X therapies) * Decision support for
doctors (suggested medication, ..) * AI System to monitor patients in the
hospital * Scheduling for the devices (MRI, CT, ..) * Communication system
between specialists (GDPR compliant nowadays, yay) * Import/Export to share
with all other hospitals * Billing system * Social insurance integration *
Government integration * Medication system * ER System * ....

This is a pretty loose collection of some things that come to mind. :)

~~~
Eridrus
> We can't just decide "upgrade tonight and go offline for a bit". So we have
> to engineer around this... But since we can't do a bulk update and force
> people to restart, we need to deal with different versions which impacts the
> code quality.

This is pretty typical of almost any customer-facing software written these
days. Every mobile app has this problem. Web apps have this problem to a
lesser extent.

------
kgrin
For a take on Epic-the-software (as opposed to Epic-the-company), Atul Gawande
had a great piece in the New Yorker recently:
[https://www.newyorker.com/magazine/2018/11/12/why-doctors-
ha...](https://www.newyorker.com/magazine/2018/11/12/why-doctors-hate-their-
computers)

It gets into some (though hardly all) of the issues of why EHR software is the
way it is, and why (some) doctors hate it. (Among my friends who are doctors,
some hate Epic, and some absolutely love it - depends on specialty, age,
institution, how it's configured, etc.)

Among some interesting issues:

\- As others in this thread have noted, the buyer (administrator concerned
with maximizing billing) is not the user. That's the easy one that's common to
a lot of enterprises.

\- Epic makes it easier for medical directors to track population health and
impose standard protocols of care. Individual practitioners don't always like
that! I am not expert enough in those fields to say who's right, and I suspect
there's not always an obviously correct answer.

\- A lot of doctors dislike the underlying mechanic - being forced to actually
write down everything they're doing and why - on top of sub-par UIs. The goals
of the system conflict with the goals of the practitioner.

\- Interoperability sucks, but it's true that standards aren't really there,
and it's hard to get consensus... plus everything you put in prod needs to be
back-compatible approximately forever. A lot of institutions got burned by
maintaining internal software built over decades that you can't turn off
because lives depend on it, and Epic/etc. are part of trying to avoid
repeating that mistake.

There's more. I only have a toehold in that world now, but love to chat about
it. Email in profile.

~~~
nradov
The interoperability standards are actually pretty much there, at least for
the more common types of patient records. The situation has improved
tremendously in just the past few years. Epic and their major competitors are
doing a decent job of supporting the lastest HL7 CDA and FHIR standards,
including specific implementation guides.

The bigger problem now is that many provider organizations haven't upgraded to
current EHR release versions, or haven't opened up their systems to access by
other providers.

------
kurthr
I have to mention that having seen the software used that the UI is awful.
Perhaps this is mandated by the hospital customers or made difficult by HIPAA.

However, when the database for patient records of two Epic systems in the
center of silicon valley do not interoperate that is just embarrassing. When
the designated procedure for a patient transfering to a hospital is, "print
out the entire record, fax it to the hospital, hand re-enter all data, shred
both copies of patient data" that is asking for horrible transcription errors
and missing data. Sorry, no donut.

~~~
goda90
Is your experience with Epic from over a decade ago? Transferring patient
data, especially between two Epic systems, is much easier than that. As far as
the UI goes, over the last four years Epic has had a big push to hire experts
and make processes to help develop and test good user experience, as well as
trying to make it look better. But that takes time in a big, old, complicated
system, and you still have hospitals who haven't upgraded from 3+ year old
versions. The latest version already looks immensely better in lots of areas,
but you also can only do so much to make an easy to use program that also gets
dense medical data in front of the right pairs of eyes so that patients don't
die.

------
bawana
the problem with EMR is that it fosters copy-paste. you read the same
misinformation over and over. If copy-paste is disabled, content will
degenerate to fiction.

in our data driven medical system, human narrative has lost value, just as the
physical examination is the equivalent of judging a book by its cover. Today
we rely on invasive and computationally intensive tests to tell us what's
going on. Ultimately i see AI as offering a 'tree of choices' at each step in
the diagnostic and therapeutic decision tree where we doctors only need to
click on the most right (least wrong?) choice as we pare the tree to the
situation at hand.

Drug recommender systems are the crude first step i see being implemented. but
these are choices made in isolation without integrating the numerous other
data points available for a particular patient.

currently, the health care industry is the largest employer in many towns. the
social disruption by further 'unemploying' these people will convert gainfully
employed individuals into the depressed patients of the institutions they used
to serve. so some wisdom is needed here to temper the zeal of our blind
pursuit of the electronic medical record.

------
jdeibele
When my mom was taken to the emergency room last year, it was common for the
doctors to be followed by somebody who would enter things into the computer
for them.

It seemed like a reasonable way to handle entering medical information without
having to wait for it to be transcribed. It seems like doctors making rounds
would not have that.

------
skwb
I'll just leave this here to try to help explain the complexities on medical
technology. These systems are all the complexity of medicine (and medicine is
hard) with the difficulties of a large enterprise system.

[https://www.wired.com/2015/03/how-technology-led-a-
hospital-...](https://www.wired.com/2015/03/how-technology-led-a-hospital-to-
give-a-patient-38-times-his-dosage/)

------
akhilcacharya
I'm a big fan of this line:

> Epic is constantly scanning the undergraduate ranks for new hires. Rather
> than sticking to engineer incubator schools like M.I.T. or Stanford, it
> scouts institutions like Carleton College in Minnesota and Clemson
> University in South Carolina. Candidates take online tests to gauge their
> problem-solving skills and, if they pass muster, are whisked to Madison for
> an on-campus interview and tour of the area.

------
andrewl
The comments on the article are fascinating.

