
Babylon Health admits GP app suffered a data breach - g_p
https://www.bbc.co.uk/news/technology-52986629
======
azalemeth
Some context: Babylon is a (US owned) health conglomorate that has come under
a lot of flack lately in the UK for:

(a) promoting a "Virtual assistant doctor" \-- glorified series of 20
questions -- that did not spot medical 'red flags' like chest pain,
breathlessness at rest, etc; providing dangerous and inappropriate advice

(b) fleecing the NHS on expensive contracts and aggressively marketing "phone
first" or "digital first" consultations

(c) "partnering" with NHS Digital and NHS X to build their commercial products
upon medical records (this is related to the 'Care.data' programme);

(d) generally speaking, muscling into "core" NHS territory (if you use their
app, you used to be automatically de-registered from your real-life GP --
meaning that they got some money for providing GP services instead)

(e) having several high-up executives who have made significant donations to
the Tory party, or who are Tory members.

I'm somehow not surprised they've had a data leak.

~~~
djmobley
Babylon is not US owned, nor is it a conglomerate.

Everyone I’ve spoken to who uses the Babylon powered NHS GP at Hand service in
the UK seems happy enough with it.

They claim a 96% satisfaction score.

~~~
epanchin
It’s hard to get a GP appointment in many areas in the UK, anything more
convenient is likely to make people happy.

Do they know their video chats were recorded and retained?

~~~
martimarkov
I should have been aware if I read the T&C but I didn’t. Kinda annoyed at
Babylon and deffo not happy with them as a consumer but I do take fault for
not reading the T&C. And I only used it because I couldn’t get a GP
appointment in time and didn’t want to waste the time of the doctors in A&E

------
Nextgrid
I think this represents the danger of collecting and/or generating data even
when in good faith. Any data is a liability and it's unfortunately just a
matter of time before a mistake exposes it.

I am not sure whether recording the sessions and storing them for longer than
necessary was a good trade-off compared to the risk of having such data
exposed down the line. This breach would've been less damaging if video
recordings didn't exist and only symptoms or notes from the consultation were
stored.

This is also why I'm so opposed to data collection such as analytics or
telemetry; even if all the players involved are acting in good faith there's
the risk of a technical error exposing data that wouldn't have been there in
the first place if it wasn't for the analytics.

~~~
deforciant
Totally agree. Currently working on a public transport system and constantly
pushing not to collect names, surnames and other details when it's not
absolutely needed (in some cases it needs to know when discounts are
involved). So far so good, I hope I wouldn't even need to raise this points..
Company is a good one, no plans to ever sell or monetize data but what if we
ever lose the data :)

~~~
g_p
Thanks for engineering responsibly! I think it's our duty to do this where we
can.

As you say, data can be lost or stolen. But companies also change hands, and
it is notoriously hard to prevent it being used for other reasons after
acquisition (particularly in the US). Perhaps you can even look at if you
actually need names and other details when handling discounts? Could you
validate eligibility or do whatever is required, then assign a verified token
to it? If it's more complex, a blinded signature might let you attest to a
given identity being eligible for a discount, without you being able to look
back and check which signature it was. I'm all for finding ways to not store
data that isn't strictly necessary.

Everyone calls data the new oil, but I'm over that, and now see it as the new
asbestos. It's expensive to have it, expensive to keep it, and expensive to
get rid of it (if you do it right)

------
g_p
Sounds like an access control failure here, leading to users being able to
access other users' recorded consultations. Babylon claims that this is due to
a new feature letting people switch between audio and video in the call, but
this seems a somewhat strange claim.

It sounds more to me like an access control/authentication failure - firstly
on listing recordings that belong to another user, secondly on then giving
access to those recordings when requested. And arguably also a failure in
storing the recordings in an effective "plaintext" meaning they could be
retrieved by a user that shouldn't have access - given this is about as
sensitive health-related PII as you can get, I'd argue these recordings ought
to be encrypted with a per-recording key available from a separate key
management system?

I guess this gives rise to a bigger question around whether online GP
consultations should be recorded and preserved like this, given this kind of
situation was/should be clearly in the threat model, and will be from now on?

Edit to add: From Babylon's website, "All data is encrypted and stored
securely to guarantee your privacy" \- once again, "non end-to-end encryption"
fails to save the day due to a failure of another component of the system, and
becomes snake-oil marketing copy.

~~~
save_ferris
> It sounds more to me like an access control/authentication failure - firstly
> on listing recordings that belong to another user, secondly on then giving
> access to those recordings when requested.

Yep, I agree. Perusing their careers page, I can see they use ruby and java.
It’s so easy to misconfigure resource authorization in a rails app, which is
why it’s so important to have solid testing around those components of an app.

> I guess this gives rise to a bigger question around whether online GP
> consultations should be recorded and preserved like this

I also completely agree here. EMR systems are well established at this point
and doctors have somehow gotten this far without needing full recordings of
most consultations. It wouldn’t surprise me if they were developing an
abridgment system that automatically creates notes for doctors or something,
but like you said, storing that training data is such a huge risk.

------
FearNotDaniel
It's easy for people to pile on to Babylon Health and suggest they are both
evil and incompetent in their approach to patient data. And it's easy for BBC
journalists with a strong pro-NHS, anti-private-healthcare agenda to spin it
into a clickbaity "suffered a data breach" headline which, though technically
true, implies something much worse than what actually happened here.

Sounds like somebody accidentally shipped a bug that didn't lock down
permissions properly. Not something any of us wants to happen, that's why we
have QA procedures, code reviews, and hopefully somebody independently
auditing any new code that touches personally-sensitive data to ensure
security standards remain watertight. Obviously that part of the process
failed this time round and the bug slipped through the net.

Bugs happen. Nobody releases 100% bug-free code 100% of the time. Not even
NASA. Some firms, due to the nature of their data and the risks involved, have
a greater responsibility to run processes that minimize the likelihood of bugs
but _also_ to deal with them quickly when they are spotted in the field.

This was fixed two hours after it was reported. A handful of users had
temporary access to other users' data that they shouldn't have had. And then
they didn't. There is no evidence of any enormous data dump on the dark web
containing yottabytes of personally identifiable patient secrets, though that
is no doubt what a journalist wants you to imagine/fear when you read the
words "data breach".

Regardless of your political/economic view of Babylon's business model and its
potential negative effect on the public healthcare system (which may well be
valid criticisms), it sounds like from a purely engineering perspective they
should get some credit here for addressing the issue so soon after it became
apparent.

------
gen220
When these sorts of health breaches make the news, I’m reminded of my favorite
dataset! The archive of
[https://ocrportal.hhs.gov/ocr/breach/breach_report.jsf](https://ocrportal.hhs.gov/ocr/breach/breach_report.jsf).
Hope it’s as interesting to you all as it has been for me :)

This case won’t make it into the database since it doesn’t impact US-based
consumers, but in general these kinds of breaches are (tragically) very
common, and remarkably well-documented. Ultimately, this page is very
convincing evidence for the often-touted idea that your SSN is effectively in
the public domain.

The most common errors include sending report-containing emails to the wrong
addresses, and leaving networks open with default access credentials.

------
epanchin
Why are sessions recorded?

~~~
g_p
This seems to be a pretty big question - how long are they retained for as
well?

From their own FAQ (on their NHS GP at Home service) at
[https://support.gpathand.nhs.uk/hc/en-
us/articles/1150037092...](https://support.gpathand.nhs.uk/hc/en-
us/articles/115003709229-Who-has-access-to-my-consultation-recordings-) :

"All recordings are encrypted and accessible only to you and restricted
members of the Babylon GP at hand senior clinical management team managing
your care, for the purpose of ensuring exceptional quality of service.ted and
accessible only to you and restricted members of the GP at hand senior
clinical management team managing your care, for the purpose of ensuring
exceptional quality of service."

Asides from the fact it's got a copy-paste typo on it (perhaps not an
important topic), no real info on how long recordings are retained for.

Edit: From [https://www.gpathand.nhs.uk/legal/privacy-
policy](https://www.gpathand.nhs.uk/legal/privacy-policy), it says recordings
are retained until 10 years after death (with a few caveats), so "forever"...
But only accessible via the app for 14 days, then you would need to ask for
access.

They also say at [https://support.gpathand.nhs.uk/hc/en-
us/articles/1150036923...](https://support.gpathand.nhs.uk/hc/en-
us/articles/115003692305-How-safe-is-the-data-that-babylon-holds-) that

"All medical records are centrally held and meet or exceed NHS Standards. We
work to processes and procedures that meet ISO27001 standards and have been
independently audited and confirmed by BSI. The application has also been
independently security tested."

They could probably argue these recordings aren't "medical records", but a
recording of what was discussed is tantamount to a medical record. Just goes
to prove to any doubters left that "tickbox" security doesn't work, regardless
of ISO standards... And that a pentest is only as good as the pentesters, and
the version of the system they tested.

~~~
ekzy
Re-watching a consultation is useful for both the patient and the consultant.
As far as I know (I did some consulting work for them), the data is encrypted
and not accessed for any other purpose.

~~~
epanchin
I wouldn’t allow my doctors surgery to record my in person GP sessions, why
would I allow an app to do the same?

------
cde-v
Bet their engineering department is crippled by bureaucracy and headed by a
business person not a tech person.

~~~
g_p
Looking through the vacancies, they have a category of jobs under "Enterprise
IT", which includes CI/CD Engineer. Doesn't bode well, but no hard-and-fast
details of who they report to.

They're also hiring full-stack engineers and senior software engineers under
their "Clinical & Business Platform" group, as well as a (Sr) Full Stack
Engineer under their "AI Cognitive" part.

If these listings are reflective of their structure, you are probably right!
Is there any good data or research on why heading up tech/engineering with a
non-tech business person is bad? It's a pet bug-bear of mine!

~~~
rvz
The reviews also paints a different picture and look just as brutal as the
company culture: [0]

[0] [https://www.glassdoor.co.uk/Reviews/Babylon-Health-
Reviews-E...](https://www.glassdoor.co.uk/Reviews/Babylon-Health-
Reviews-E1338681.htm?sort.sortType=RD&sort.ascending=false)

~~~
rubatuga
You know, I've read through a handful of "health" startup companies on
GlassDoor, especially those that had posted on HN, and I would say more than
half have similar issues as Babylon. Sentiment such as "a great cause" but
"horrible management", and "tons of smart people" but "bad leadership". I
wonder why it's so common at healthcare startups.

~~~
g_p
I wonder if there's a factor in the make-up of the founding team needed to
raise money and gain traction through credibility, that leads to this?

The leadership and management issues being so common makes me wonder if the
resulting founding teams lack the experience of running technical teams and
cultivating a good environment and morale. I suspect it's inherently NP-hard
to manage a tech team whose work you can't yourself do. I imagine little of
the founding C-suite of "health' startups has experienced the "length of a
piece of string" engineering deadline when trying to solve complex issues etc,
and perhaps that feeds into culture?

~~~
LyndsySimon
I admit I hadn’t read your comment when I posted a sibling to it.

I agree with you. I think successful healthcare tech companies go through a
growth stage first where the key is a solid basis in the healthcare field.
Engineering experience isn’t as important in the beginning... right up to the
point where everything comes to a screeching halt as tech debt reaches the
point where it rapidly expands to consume pretty much all of the engineering
team’s time.

