
How a tiny hospital used AI to lower costs and improve patient outcomes - jtsymonds
https://www.healthcareitnews.com/news/flagler-hospital-uses-ai-create-clinical-pathways-enhance-care-and-slash-costs
======
joshgel
As a physician who does lots of bioinformatics, this seems like a very
optimistic article which is likely just overfitting the existing data.

Patients don't come to the hospital saying "I have pneumonia". The come for a
cough or a fever and it takes these tests to figure out the diagnosis. A small
hospital like this probably doesn't have the data to really build a robust
model that will help all its patients. Even major academic centers have
relatively limited data for the number of permutations of health
possibilities. Sure they have lots of pneumonia data, but how many cases of
pneumonia do they have in cystic fibrosis patients who also have COPD?

I certainly think AI can help cut costs, reduce over-testing and reduce length
of stay, but decreasing length of stay by 2 days would be a shocking advance.
(The average simple pneumonia LOS of stay for a medicare patient without co-
morbidities* was 3.3 days and with major co-morbidities* was 5.7 days in
2017). [0]

*as defined by medicare.

[0] [https://www.cms.gov/Medicare/Medicare-Fee-for-Service-
Paymen...](https://www.cms.gov/Medicare/Medicare-Fee-for-Service-
Payment/AcuteInpatientPPS/FY2017-IPPS-Final-Rule-Home-Page-Items/FY2017-IPPS-
Final-Rule-Tables.html)

~~~
boxspam
> is likely just overfitting the existing data.

I would be more careful with your assessment. Or you are overfitting to your
limited 5-minute-understanding of what this does. Telling mathematicians and
statisticians they have overfitted is akin to a grave professional insult (you
are basically accusing them of foolish behavior at best, fraud/unethical
behavior at worst. As a bioinformatician, you may be interested in these
articles, which go a little deeper on the technology:
[https://www.nature.com/articles/srep01236](https://www.nature.com/articles/srep01236)
[https://www.sciencedirect.com/science/article/pii/S240547121...](https://www.sciencedirect.com/science/article/pii/S2405471216301831)
[http://journals.plos.org/plosone/article?id=10.1371/journal....](http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0126383)

> probably doesn't have the data to really build a robust model

It is both possible to build robust models on small datasets (and that data is
shared, or more abundant than you expect).

> Even major academic centers have relatively limited data for the number of
> permutations of health possibilities.

This is one of the problems they (the IT supplier) solved. They are able to
build regulatory-proof models from 100.000's of permutations. Another, more
recent, advance is in counterfactual analysis (what would have happened if
this pneumonia patient also had COPD?).

~~~
joshgel
Listen I'll be thrilled if I'm wrong, but the article says they 'expect to
save' $xxxx.xx. Not that they are saving that much. Come back to me when they
actually have reduced average length of stay for pneumonia patients by two
days over a long time period. Just because they can find a most efficient
pathway for patients, doesn't mean it will be followed, or that it is even
possible to be followed clinically.

My hospital currently uses Allscripts, we have care pathways, and when I want
to order something for a patient of mine (because I think the patient needs
it) that isn't in the care pathway, I just order it. It seems that what they
are trying to do is 'Nudge', in the Thaler sense, providers to do what they
suggest is the most efficient pathway. Would love to see a randomized trial of
giving some providers this nudge and others the current pathway to see which
is most effective. But what I often see is that providers have learned
behaviors from years of training where they 'need' tests X, Y, and Z for
disease Q, so without extensive education, they still tend to order those
things.

edit: just saw that ballenf beat me to this punchline already.

~~~
boxspam
They actually reduced the average length of stay for knee replacement from 3.3
to 2.4 days. They verifiably saved costs of a middle-sized hospital of 10m$ a
year. If we assume professional integrity, then they won't intentionally
inflate projected savings, so we have no reason to doubt their expectations.
(Just like I don't have any reason to doubt that you don't give inflated life
expectancy projections to your patients, or receive kickbacks from needless
prescriptions). It is, on a professional level, like telling an engineer who
projects a bridge to be safe, that they likely build an unsafe bridge, and
you'll only believe it when at least a 1000 people have crossed it (trust me,
I am a lorry driver who loves driving over bridges).

The hospital in the article also used Allscripts.

> The next step for Flagler was to review the findings with the Physician IT
> Group (called the PIT Crew) and to make the necessary changes to AllScripts.
> Physician buy-in is critical...

> There are two interesting anecdotes from this process that bear repeating.
> The first is that once doctors became aware of the work that was being done,
> requests for membership in the PIT Crew skyrocketed and attendance at the
> bi-weekly meetings doubled. Doctors want access to data.

> The second is that one of the more accomplished physicians remarked that the
> care process model for pneumonia was far lighter than what he would have
> used, but upon looking at the outcomes, readily agreed that it delivered the
> same or better care in almost every case – and that what he was doing was
> essentially unnecessary, or wasteful. Presented with the evidence, he
> committed himself to rethinking his approach.

Edit:

> but decreasing length of stay by 2 days would be a shocking advance.

If you look at the industry average, maybe. But these 2 days were for this
specific hospital (being a community hospital they get many different
patients, and you can't have experts for every area). Perhaps removing those 2
days brought them closer to industry average, which seems like a very
reasonable advance.

------
Justin_K
HIT news is paid articles by the vendors. It's an industry rag and all their
stories read the same, with the vendor name getting dropped about paragraph 3.

------
ballenf
> It _expects to save_ $1,356.35 per pneumonia patient in direct variable
> costs (35 percent savings) versus the status quo, while reducing length of
> stay by two days. The new sepsis pathway has also been deployed.

So the headline was BS. This is the projected savings estimate used by the
vendor, who totally coincidentally got a bump in their contract from 4 case
studies a year to 12 on the basis of this totally objective guess.

------
trhway
Reminded that episode from Greys Anatomy where they, using similar approach
[manually, not AI though], discovered that pulling kidney surgery drainage
couple days earlier (on the 3rd day instead of 5th) minimizes complications
like fistula.

The AI produced recommendations in the article doesn't seem to have any
rationale other than statistical correlation ("topological data analysis").
Patients who had that care path did better - i hope they controlled for the
possibility that those patients may have been the "better" ones to start with.

>“After looking at the data, a cardiologist on the Pit Crew said, ‘Oh my, the
Goldilocks group of doctors did less than I would have done but achieved as
good or better outcomes. Therefore, when we add anything outside the CarePath,
except regulatory requirements, we are adding cost without any benefit to the
patient.’ For him, that was a ‘light bulb’ moment.”

I hope there was another cardiologist near by when this one was having his
light bulb moment.

Overall i suppose it is a nice (by the metrics of healthcare IT ) sales push
piece. I'm kind of wondering why such basic analytical practice hasn't been a
routine for decades already ( may be because anybody able to do basic
statistics would make much better money on wall street than being a lowly
number cruncher at a dusty office in the hospital basement ... right to the
moment of course when the thing gets renamed to AI and starts to be shiny and
glorious and the hospitals start to pay good money for it).

>Flagler expects to save $1,356.35 per pneumonia patient

better than if i switch to Geyco.

~~~
merpnderp
Unless the people were dying, the proof is in the vastly reduced readmittance
rate.

------
bayesian_horse
Sounds like someone is applying statistics with lower statistical thresholds
and calling the method "AI" instead.

~~~
larrydag
That's what I thought too. It sounds like basic data analysis. Nothing wrong
with that because it brought good results. Why tag the project with AI?

~~~
reaperducer
Funding.

------
maxxxxx
I am glad that they are lowering costs but is this really about AI? Usually
someone can sit down and quickly find ways to reduce costs but the problem is
usually the power to actually implement them.

It's like dealing with consultants. The company's employees already know
what's wrong but don't have the power to change. Then consultants come in with
CEO backing and suddenly change gets made. Consultants (or AI) get the credit
but it's really about being to implement.

~~~
boxspam
This is more of a puffy PR piece for the IT supplier.

The methods they use are very advanced mathematics / graph theory. If we take
the modern usage of the word "AI", then this qualifies. These methods are
beyond what anyone can do in Excel. Usually data is so big and complex that a
manual analysis takes months. It does not scale to sit down and find ways to
reduce costs. Using unsupervised pattern detection does scale.

About the consultants vs. engineers and the power to actually implement
systems: Many IT systems that call themselves "AI", are not. They require tons
of labeled data, a human making assumptions, lack justification and
transparency, are not embedded properly in the business (lack UI,
documentation, buy-in, data processing), and don't continuously learn.

You need an entire ecosystem of processes and software tools to have the power
to change (identifying problems is easy, of course engineers already know
what's wrong with their daily work, if they could implement a solution, then a
proper engineer would, but they really need a consultant and CEO backing to
actually get something actionable).

------
tomohawk
> "expects to save"

Shortened stays, less expense, and fewer tests sound nice, but and independent
verification after some period of time seems imperative.

It also makes me wonder about some of the outlier patients. Those tests may be
very beneficial for them. However, I'm sure doctors may feel pressure to not
order them due to the cost savings.

~~~
merpnderp
Likely hard to fake the vastly reduced number of readmittances.

~~~
riahi
As mentioned elsewhere in the comments, small/tiny hospitals rarely keep
anything complicated and ship everything vaguely complex to
tertiary/quaternary center.

------
SQL2219
I don't understand how their readmission rates are so low at 2.9%, I was just
checking some stats on that at the link below, typical readmission rates are
15-20%:

[https://data.medicare.gov/Hospital-Compare/Hospital-
Readmiss...](https://data.medicare.gov/Hospital-Compare/Hospital-Readmission-
Rates/92ps-fthr/data)

~~~
Fomite
"Tiny hospitals" have low readmission rates, because the complicated patients
with lots of readmissions get transferred to tertiary care centers.

This also came up in things like Consumer Reports and their analysis of
hospital acquired infection data. Small community hospitals with uncomplicated
patients end up looking _really_ good.

------
wolfi1
when such terms as "unsupervised learning" are coined I doubt that there was
something achieved, common sense wouldn't have achieved, too. As an AI-expert
once said to me: unsupervised learning does not work

