Hacker News new | past | comments | ask | show | jobs | submit login
Chaos Computer Club saves the German healthcare system 400M Euros (ccc.de)
160 points by ulnarkressty on Oct 15, 2022 | hide | past | favorite | 48 comments



For anyone who doesn't read German, here is a summary: The company in charge of the connectors for the health care system "Telematik" built devices with certificates that expire after 5 years. Instead of updating thousands of machines with new certificates, the company claims these need to be replaced for a total cost of 400M Euro. The CCC showed how the firmware can be changed to accept new certificates, making the hardware replacement unnecessary, and offers to assist hospitals and doctor's offices with the software patch. However, the company "gematik" who delivers these connection endpoints first has to sign the new certificates, which they so far haven't agreed to.

What's worse about this story is that the company apparently planned the hardware replacement in a way that it would have to be replaced again in 2027, and they still were awarded the contracts for this multi-million dollar project.


I wonder how many golf trips with steak dinners and female "companions" the bureaucrats awarding the contracts got treated to...

The German government has been lauded as this great bastion, "Look, their economy is still running!", but it's also has its corrupt elements. Besides, the whole roaring economy thing is related to making the rest of Europe peg its currency to the German one by way of the Euro, making the rest of the continent suffer...


We've already been fined by the EU for refusing to work against corruption. It's a big thing here and voters only care for a few months after the news reports, if that much.


How can I vote to replace corruption?


Don’t vote for the most egregious offenders. I’d say on a party base that’s CDU and AfD, followed by SPD (with our CumEx-"I can’t remember"-chancellor Scholz). Short term, nothing would happen. But if people en large refused to allow corruption, party politics would change.


Why downvote this valid comment?


This makes it sound like Germany had chosen to introduce the Euro whereas [1]:

>President François Mitterrand argued for the single currency because he hoped to bolster French influence in an EU that would otherwise fall under the sway of a unified Germany

However, you can argue that the continent suffers because Germany had reduced its labor costs [2], which unnecessarily moved production processes.

[1] https://www.economist.com/leaders/2012/11/17/the-time-bomb-a...

[2] https://en.wikipedia.org/wiki/Hartz_concept


I‘m assuming the German government wouldn’t be able to compel them to turn over their private key, but they could certainly make it very clear that they would jeopardise any future contracts if they refused to cooperate.


It is a little more problematic than that:

The Gematik GmbH is in part managed (owned?) by the German ministry of health [0] and some health organizations like the Association of Statutory Health Insurance Agencies.

That says something about the 'risks' the Gematik takes (Hint: none).

Similar story: Earlier this year some health card readers, certified by the Gematik had a bug. They wouldn't read certain cards that supposedly were electrostatically charged. The solution was a grounding device connected to the USB port of the card reader. [1] This thingy cost the doctor's practices another 100 bucks even though this clearly is a design flaw by the manufacturer.

They can do pretty much what they want at this point and the physicians and hospitals just have to cough up the dough.

[0] https://de.m.wikipedia.org/wiki/Gematik

[1] https://www.borncity.com/blog/2022/01/16/problem-mit-statisc...


there is somebody in the chain of command who benefits of that deal and will do what she can to keep it as inefficient as it is.


I'm assuming you haven't heard about corr^w lobby.


What's corr^w?


Ctrl-W in certain contexts (especially shells) deletes the previous word. Shells vary in their behavior, but in this situation partOfAWor^w is essentially the same as using strike through. It implies the writer started writing a word and changed their mind on which word to use.

In this case, they're implying lobbying and corruption are one and the same, as far as I can tell.


> Finally, the CCC appeals to the manufacturers of the connectors to look for honest ways of earning a living.

This is fucking brilliant XD


In almost every country there is a number of well connected companies who make a living exclusively by specializing at navigating through the complicated layers of bureaucracy and swindling the taxpayer by selling overpriced and unnecessary proprietary products and services to a government who by cluelessness or corruption, has no idea or doesn't care about sensible technical solutions and just agrees with whatever those preferential suppliers offer.

It's hilarious when you look at the web pages of such companies and see that their only customers in the last decades are various government agencies and state enterprises which are their bread an butter as they would never be competitive on the free market. Working at such companies is even more toxic. Extremely outdated tech, poorly qualified staff, huge bureaucracy yet poor management and poor understanding of the work going on, and so many people doing nothing all day than keeping seats warm and answering a couple of emails per week.

This is especially true in Germany. I remember the story from a former colleague who worked at the German information security government office (BSI) which had a cipher calculator on their website and it turned out to have a flaw in the calculation, and since the guy who implemented it didn't work there anymore and nobody on the staff knew where the code was, so instead of fixing the calculator or removing it from the website, they instead put a warning on the webpage that this calculator is wrong and shouldn't be used and called it a day lol. Hilarious but also sad for the German taxpayer.


One big german company who has had a couple of scandals attached to their name and that has been 'disappearing' progressively from markets operated in similar ways (and the market cap going down progressively).

They end up cornered in markets where the expertise needed and moats are high and where they are slightly not too behind or ahead of the competition.

But anyway, the way to procure such "routers for secure networks" would be to add a contractual requirement of support for 20+ yrs.


I work at a company like this. From what I gather, it seems to have started out as a fine product-oriented business, but turned into what it is today due to the incentives involved. We sell our product primarily through public sector invitations to tender, where what matters is hitting a bunch of feature checkboxes on a list, with zero regard for the actual quality of the product.

You get exactly what you pay for.


>what matters is hitting a bunch of feature checkboxes on a list, with zero regard for the actual quality of the product.

This is true for most government contracts in most countries.

It gets worse with government contracts involving tech products and services as most governments don't have any skilled and knowledgeable tech people working for them, even in Germany, because why would they work for the toxic and underpaying public sector when they could earn more and get more respect in the private sector. This way, you only retain clueless people who only heard about the internet from a book, don't care about learning or improving things and are just there for a cushy job, ticking boxes and filing paperwork till retirement. If you want to hire skilled tech workers you gotta pay them well and give them freedom, which is not usually what government jobs are all about.

From what I've heard, Denmark is a great example of a well run, transparent government, which also employs skilled tech workers to develop high quality government software in house, instead of farming it all off to slimy companies who are gonna ship low quality trash at insane markups.


When it comes to anything with computers it’s difficult to imagine ways to end up more incompetent than literally all German public (or pseudo-public, like here) agencies. You’d certainly get something more competent by collecting random strangers off the street.

It’s virtually impossible to overstate how bad the situation in this area truly is in Germany.


The fact that the agency is public or private has no innate meaning in terms of efficiency. The people in the organization and the processes make it more or less efficient and this is true independently from where the funding is coming from.

Can public organizations be made more efficient? Of course, as any organization can.

Finally, in this case is it not a private firm that is promoting inefficiency via dishonest market practices?

The real issue here is that the public agency has to buy services and products from private firms. Why can't the German state produce cost efficient routers on its own?


Private companies have a stronger incentive to make choices that accrete value to the firm.

On average, this means they’re more rigorous in their vendor selection processes.

Therefore private entities are less likely to make bad purchases.


> Therefore private entities are less likely to make bad purchases.

This claim gets thrown around so much, I can't stand it. The bigger a company gets, the more bureaucracy you have to deal it. Many big companies/market leaders are basically no better than government agencies when it comes to efficiency.

Also, I hate this idea of privatization for the sake of "efficiency". Imho some things (like healthcare) have so positive external effects on the whole economy, the should not be trimmed for profit, but for the best results instead. And, as it turns out, letting companies compete for this kinds of public infrastructure with a "winner takes it all" principle often does not quite turn out to be the most efficient choice


That stops working when the internal bureaucracy of the company reaches a certain size and internal politics become too isolated from the realities of external competition. (Other reasons for isolation are also possible, such as when the competition on the merits of the product is secondary to competition on the ability to navigate regulation or the social circles of the few prospective customers.)

As far as I’ve seen, the general dynamic of having to spend the budget you’ve been given, even in the most wasteful way possible, or facing a budget reduction next term is universal whether you’re a bureaucrat in a socialist government, a bureaucrat in a large department of a democratic government, or a manager in a large company like IBM or Oracle.


This is nonsense. In Spain the FNMT issued certificates work fine and have simplified much daily business.


Germany not Spain....

>>difficult to imagine ways to end up more incompetent than literally all German public

I think that pretty much sums it up perfectly.


It’s unconscionably incompetent how a financially-stressed federal government appears to be unable to immediately seize an opportunity like this, along with doling out harsh repercussions for the executives who have decided (and still do) to cash in on this scheme.

This has become a repeating pattern to such a degree that claiming plain incompetence cannot plausibly explain it any more. Maybe it’s not outright malice but corruption and fraud indeed.


Deepl:

> Special routers are required in German doctors' offices to connect to the "telematics" health data network. After only five years of operation, there is no alternative to replacing the devices - at least according to the manufacturers. This exchange is expected to burden the already struggling healthcare system with additional costs of around 400 million euros. The Chaos Computer Club (CCC) shows that the expensive hardware exchange is anything but necessary, and donates a solution to the problem free of charge.


>This exchange is expected to burden the already struggling healthcare system with additional costs of around 400 million euros.

I'm always amazed at the gross inefficiencies of the German bureaucratic machine (including most public and government institutions), despite the traditional world renowned stereotype of "German efficiency", even though at this point I know I shouldn't be surprised anymore.

The CCC is a national treasure.


Call me jaded, but my experience with public contracts told me that, most likely, the hardware will still be replaced or, best case, the company providing the firmware will get a follow up contract around 100 million to do the update.

Also expect close to no media coverage about that, or any political consequences. Heck, it took a thorough, highly public late night show investigation into the head of a government agency for cyber security and his close, and private links to former KGB agents and owners of cyber security firms, shady lobbying associations and whatnot for to be soon replaced. As if his known close ties to the private sector, and specific companies that ended up in the concil advising the government on cyber security, wasn't enough. One has to wonder so, how such a position doesn't require a security clearance. If I had close private contacts to known former KGB guys (known because the guy in question got an award in public for long service for the KGB and the state) I wouod have never passed these checks. Or if I had not mentioned them I would have lost my job immediately. Fun fact, it was the former conservative led government, and more specifically conservative politicians in cjarge of the ministry of interior, that put the guy in his position. And politians from the same party maintained the contact with Russian authorities.


It’s probably difficult to appreciate for Americans how wildly intransparent German governments and their agencies are across the board. The default for everything is to be closed and private. No accountability mindset at all.


Isnt the stereotype 'deutsche Gruendlichkeit' which is not efficiancy but thoroughness, which just mean that a few thousand pages need to be filled out, signed and countersigned for the 400 mill order.


Living in both Germany and Austria, I've learned that the excessive paper based bureaucracy is mostly there so that people in charge have their butts always covered. "It's not my fault, I did everything by the book, here look, I've got the paperwork to prove it".

Tip for anyone living or wanting to move there: keep digital copies of all important letters, bills, documents, contracts and paperwork you get. It's not unheard of for a company or government institution to make mistakes and you ending up on the hook with huge fines or bills to pay, so having copies of all possible paperwork from the beginning of time could save your ass one day.


This sort of abdication of responsibility permeates through german companies as well and it pisses me off to no end.


To play devils advocate:

€400M sounds a lot but how many of these devices are there? If there's one in every medical practice that could be 100-200,000. [EDIT: this article https://www.healthcareitnews.com/news/emea/error-which-cause... suggests there are 130,000 clinics, that would be €3K per clinic]

Having a technician visit each and do a firmware update - could well cost over $5K or more, as long as introducing downtime at the surgery, the changes would need to be done by people who are trained and this is a device that is involved in personal medical data - they need to be managed and monitored.

Delivering a new piece of hardware with the new certificates that could be dropped in could well be cheaper (how ever bad for the environment) than updating them within the legal requirements that may be in place for tech that handles medical data.

There may be good technical and legal reasons why the certificates can't be updated remotely or are set to expire, but if I were the companies involved I would take in some devices, 'refurbish' them with new certificates and send them out to medical practices for drop in replacement, rather than sending out new devices.


>There may be good technical and legal reasons why the certificates can't be updated remotely or are set to expire, but if I were the companies involved I would take in some devices, 'refurbish' them with new certificates and send them out to medical practices for drop in replacement, rather than sending out new devices.

It's about that the Devices DON'T accept new certificates over a certain date, like when your iphone just accept certificates who are valid up to 2022, then you need a new iphone, that should be illegal, and the firm should have to pay the technician/fw-update.


Isn't that because the certificate the boxes to use to validate the remote certificates have an expiration date (as they probably should). An iPhone gets updated certificates every time iOS is updated.


> An iPhone gets updated certificates every time iOS is updated.

No you get updated certs from cert-authorities (the one's trusted by apple/google/mozilla etc), the ones who "signed" the received certs from website X. Otherwise you would have to download gigabytes of certificates.

https://www.youtube.com/watch?v=86cQJ0MMses

>>TLS Handshake Explained - Computerphile


Love the CCC! Visited the congress in Leipzig in 2017 and it's been such a great and fun experience, absolutely recommended for everyone. Finally a tech event that isn't all about money and corporate sponsors.


The corruption in Germany is unbelievable. Given Schröder, Merkel, Dieselgate, this thing ... Germany os basically a rich Russia.


The difference is that in russia corruption is "in theory" illegal, in germany no one (if you are a politician) gets hurt.

You forgot the mask-scandal ;)


Can car manufacturers sell cars with a sealed tank, and you would have to buy a new car if it runs out of fuel?


There's the concept of an implied warrant of merchantability. If a thing has an ordinary purpose, and the thing sold fails in that ordinary purpose, the buyer has recourse.

Since a reasonable person buys a car as a long-lasting mode of transport, the sealed fuel tank would probably not meet expectations.

However, if the propulsion system were, say, nuclear fission, and the system lasted 5-10 years and then needed to be replaced, that might be a satisfactory product from this legal viewpoint. The NTSB and Department of Energy would have some things to say about it, though.


Correct headline would be "Chaos Computer Club shows the German healthcare system how 400M Euros might be saved". Given how incompetent politics has been in the past, I wouldn't be surprised if this is ignored with some bullshit reason and the money is wasted.


Here's a podcast where they talk about it:

https://logbuch-netzpolitik.de/lnp433-auf-yolo-konfigurieren

(In German)


Hackers save the day. Once again.

Remember, kids, don't buy non-rootable devices of which you have no control! )


Alternative title:router reseller scams the German government for 400m


Wait, does that mean they've upgraded from fax already?


tl;dr

To transfer patient data between doctors and (state) insurance companies, doctor‘s offices need to have a hardware VPN device. The system was implemented by the company „gematic“. A small number of companies produce these devices.

The certificates on these devices expire after five years. Now, instead of simply updating the certificates, the companies want the state and the doctors to buy new devices which costs around 400 million.

The CCC firstly explained that this is bullshit and a total waste of money and secondly showed that it is easy to update the devices. They could do it themselves but only need the private key from gematic.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: