
Medical devices: A ticking time-bomb  - bootload
http://www.economist.com/blogs/babbage/2012/05/medical-devices
======
pak
"the average error was a staggering 24 minutes"-- The average is a terrible
measure of centrality for data that is unbounded at one end, because one
outlier (the 42 year off clock) skews the measure disproportionately; the
median would be much more appropriate to report. That's unusually
sensationalist number crunching for The Economist.

I'll bet that out of the clocks that doctors actually use to decide on
treatment (tallying _every_ device is a bit unfair; if a blood pressure
monitor on some cart in a closet has the wrong time of day who cares) 80% of
them are within a few minutes of the right time. And at the end of the day,
doctors are supposed to be able to reconcile conflicting information, it's
part of their training. This is a little bit like tallying up every textbook
lying around a high school and getting concerned that the average book is
three editions out of date and has 142.8 errors--calm down, most of those are
not in active use and have some faith in teachers to check their material.

~~~
chrisbroadfoot
Average is a fine measure. You've inferred that they used a mean average. The
article doesn't say what kind of average they took.

Let's assume they did take a mean. If the mean error for 1700 devices was 24
minutes, the total error is 40,800 minutes. That's 28 days.

Therefore, they didn't include the outlier of 42 years in their average. It's
most likely a median (or mode) average.

~~~
blahedo
It's pretty unlikely to be a median; if 20% of machines are 30 minutes off and
24 minutes is a median, then 30% of machines are between 24-30 minutes off,
which would seem to be a strangely narrow band.

More likely, the 24 minutes is a mean average after excluding outliers. One of
the standard ways to compute outliers is to measure the inter-quartile range
---the "distance" between the 25th percentile value and the 75th percentile
value---and call everything that is more than 1.5*IQR past the 75th percentile
an outlier. You can do this without computing a mean first, and you can use it
to exclude values before computing a modified mean.

------
m104
NTP is great and all, but...

When you're timing something that needs accuracy, you use a timer not a clock.
If you need to use a clock, you use one clock not all the clocks. Yes, I work
in the medical industry. Yes, we generally use timers not clocks. Yes, our
clocks are all wrong too.

If you really do need a solution to the "time" problem (accurate time all the
time), you need to issue everyone a smartphone.

~~~
tomwalker
im a doctor also and the need for highly accurate timing is exceedingly rare.

A lot of medical timing is guess work - why are antibiotics commonly
prescribed for 5, 7, 10 or 14 days? Whynot 6?

Automatic filing of EMR information would be important and the legal aspect
would unfortunately demanc the greatest amount of accuracy.

Surely connected devices could use one centrally calculated timesource?

~~~
stewartbutler
> "Surely connected devices could use one centrally calculated timesource?"

... like NTP?

Sorry, but I'm not sure what point you are trying to make here. Anything with
a computer, even a microcontroller, and basic network connectivity can use
NTP, it is an extremely simple protocol.

Anything that doesn't connect to a network could be kept in sync via a
wireless pulse/tick, though then you have security issues unless you
cryptographically sign the clock signal. Though I'm not sure why anybody would
try and screw up a clock signal like that other than general asshattedness.

That aside, I'm unsure why they are using local time for logging data anyhow.
If the folks managing the hospital were rational, shouldn't there be a
regulation to use GMT and 24-hour clock? My home town is right on the
Eastern/Central timezone split, and I know it causes all kinds of problems in
day-to-day life. I just hate that kind of ambiguity.

~~~
kijin
> _If the folks managing the hospital were rational, shouldn't there be a
> regulation to use GMT and 24-hour clock?_

People are _not_ rational, that's the problem! Just look at any web app that
can't get timezones right.

In addition, most Americans swear by EST, CST, PST, etc. and have no idea how
many hours behind GMT they are. Record everything in GMT and soon there will
be a similar article documenting how medical professionals routinely
misinterpret GMT timestamps. Especially in March and November.

~~~
stewartbutler
When lives are at stake, people tend to be more amenable to changes like GMT
and 24-hour clock. If the military can do it, a civilian organization with
sufficient motivation (insurance terms, regulation, etc) should be able to
make the same shift.

However, I was referring more toward the recording system than the staff. If
the recording systems were all using unix time stamp with a GMT system clock
synchronized over NTP, there would be no ambiguity as to when a record was
added to the system. End user systems could be responsible for keeping track
of what time zone the user is in, and could convert as necessary. The
important thing is that the "One True Datum" have the appropriate stamp.

~~~
kijin
> _End user systems could be responsible for keeping track of what time zone
> the user is in, and could convert as necessary._

That would require routine updates to the timezone database, which is a bit
more complicated than NTP. (Yes, DST rules change all the time. The U.S.
changed its rules just a few years ago.)

Hopefully, the rise of cheap and powerful ARM SoCs will make it impossible for
device manufacturers to make excuses about not running a proper OS on their
machines anymore. I would love to see more Linux and BSD on medical devices.

------
nitrogen
I have two questions:

    
    
      Since the EMR is updated constantly, data from devices
      whose clocks are way off would simply never be recorded.
      Others might bury current information in older files.
      Worse, they may insert the data into the EMR when the
      patient they concern has left the given clinical
      environment and another has come in.
    

1\. Why does the central EMR trust timestamps from the devices?

    
    
      Daylight saving time corrections twice a year require
      tedious manual tweaks that the MD PnP Program, an
      initiative supporting networking standards for medical
      devices, estimates cost American hospitals over $17m
      annually. At Massachusetts General, a patient-monitoring
      system deletes an hours' worth of data when rolling back
      from 2am to 1am every autumn, while drug pumps are kept
      permanently on standard time, so they are (at least) one
      hour off for half the year.
    

2\. Why are they using time zones at all?

~~~
famousactress
Hi. Written EMRs and built medical devices (10-12 years exp between them). In
many cases messages from devices and external systems are processed
asynchronously, so there isn't much choice but to trust remote timestamps.. I
may be getting data minutes or hours later. Just a distributed system
challenge.

The time zone issue is a human one. Historically clinicians schedule doses at
"Every 3 hours" or "Once per day, 8am". Humans figure out what that means, but
a dose scheduling system that shifts 8am an hour after daylight savings
(because it interpreted the schedule as "every 24 hours" is going to run
aground with nurses when it tells them they're late). However, every 3 hours
is pretty obviously an interval and likely shouldn't shift by 30% of the
interval!

This isn't a distinction that clinicians are used to making, but as someone
who had to write some of the first dose-scheduling systems focused on clinical
accuracy, we ran into this challenge and did a lot of thinking about which
schedules really meant 'intervals' of specific units of time and which ones
meant 'due at these calendar times every day'.

It's an interesting example of a problem that introducing technology
introduces/exposes while showing up to fix other things.

~~~
nitrogen
_Hi. Written EMRs and built medical devices (10-12 years exp between them). In
many cases messages from devices and external systems are processed
asynchronously, so there isn't much choice but to trust remote timestamps.. I
may be getting data minutes or hours later. Just a distributed system
challenge._

So I'm guessing you might solve that by asking the device what its internal
clock is at the time of synchronization, and adjust the logs by the offset
between device time and EMR time (with manual device time changes included in
the log)?

~~~
famousactress
Sure. If there was a 'time of synchronization'. There's generally not. The
device often doesn't know when/if/who is consuming it's data. Likewise the EMR
systems don't generally have any access to the device other than receiving a
stream of messages some non-deterministic after they were produced.

------
subwindow
I wrote software for medical devices for 3 years. Saying 'just use NTP' is all
well and good, except that getting Internet access inside of a hospital is
usually impossible. So of course we offered a configurable setting so that
they could use an internal or firewall-allowed server. I think one hospital
took advantage of that setting. Everybody else was just wrong.

~~~
zdw
NTP settings can be distributed by DHCP, which in a managed networked
environment should be reasonably trustable.

Unfortunately Windows ignores this, and also sets the system clock to a
timezone-specific time, not UTC as nearly all other OS's do.

------
tokenadult
Previous submission, with a few comments:

<http://news.ycombinator.com/item?id=4013021>

I'm glad that there is some more discussion of this issue, as medical devices
are some of the main high technology products in my part of the United States,
and my son (also a Hacker News participant) worked as an intern at one of the
medical device companies here last year. Medical device businesses are much
more regulated as an industry than are most Web-based SaaS businesses.

------
mmagin
I just hope this doesn't result in poorer security between outside networks
(including the internet) and medical devices. I'd argue there should be an
"air gap" between the two, but I'm probably more paranoid than most.

Certainly it's fine to use NTP, but I'd want to use some kind of GPS-receiver
NTP appliances at each site (possibly redundantly).

------
imcqueen
why is the EMR automated at all if it isn't accurate?

~~~
famousactress
accuracy isn't boolean.

