Hacker News new | past | comments | ask | show | jobs | submit login
A mysterious bug in the firmware of Google's Titan M chip (alexbakker.me)
238 points by alexbakker 3 months ago | hide | past | web | favorite | 88 comments

Wow, I normally am all about how google handles security issues (they get dragged through the mud for some Project Zero stuff), but this def did not get handled well.

Super unclear communication, starting with "you're just using it wrong", more than six month turn around, and even then at the end no clear explanation of what went wrong with someone who was collaborating with you? That's amateur hour security.

I'd pay to see Bryan Cantrill's reaction [0][1] to this: A seemingly mysterious firmware bug of a secure element / trusted-execution-environment but there's no knowing if there are more bugs (or, shudder backdoors).

Since the source code isn't available for scrutiny (though Google has promised firmware transparency [2]), it is kind of difficult to tell what really went wrong in the current reported case and what else possibly could go wrong given the use-cases for it are far-reaching and sensitive: Google has advocated StrongBox as a trustable companion that could be used to attest user actions on medical devices [3], for instance; or for use as an Identity verificafion for documents such as Driving Licenses and Passports.

[0] https://www.youtube-nocookie.com/embed/30jNsCVLpAE

[1] https://www.youtube-nocookie.com/embed/fE2KDzZaxvE

[2] https://www.youtube.com/watch?v=0uG_RKiDmQY?t=33m

[3] https://android-developers.googleblog.com/2018/10/android-pr...

My reaction might not be worth the price of admission, because it's exactly as you'd expect: this is great, diligent, responsible work -- and if the Titan M firmware had been open, it seems likely that the author would have been able to get to root cause (or much closer to it). This in turn would have in turned tightened Google's response time, led to a better fix faster, etc.

That said, there are several reasons for optimism.

1. OpenTitan.[1] On the one hand, this is not about opening up extant Titan implementations so much as developing next-gen Titan in the open -- but it is nonetheless a laudable and important effort and it is increasingly reasonable to expect that the hardware roots-of-trust of the future will be entirely open.

2. Open firmware more generally. The Open Source Firmware Conference[2] this past fall was truly inspiring in terms of the broad interest from the industry: while there is much work to be done, there is more reason than ever to believe that it's attainable.

3. Rust. It's hard to speculate without knowing what the root cause of this issue actually was, but to the degree that memory corruption was at root here, the emergence of Rust for firmware is an incredibly important development. Speaking personally, if there was any doubt in my own mind about the appropriateness of Rust at this lowest level of software, it has been erased by our own experiences at Oxide over the past few months: Rust is unequivocally the right language for firmware, and it will yield higher quality artifacts.[3]

[1] https://opentitan.org/

[2] https://osfc.io/

[3] http://cliffle.com/blog/prefer-rust/

More troubling to me than the closed source firmware is that the bug in TFA seems like something that the most basic of a test suite should be catching.

It’s reminiscent of Apple’s “goto fail” lack of certificate checking - another easily testable case that simply wasn’t.

The test authors don’t even need to be on the same team/manager. They can just write black box tests to the spec, like the author of this post did.

I’m not even some big TDD guy. It just seems to me that in these core security-critical libraries/functions that should be pretty side-effect-free that you should have some basic “receive x, produce y” functional tests to make sure the API is doing what it claims to do on the tin.

the most basic of a test suite should be catching.

A most basic test suite is not likely to wait some arbitrary amount of time (2 seconds, as the author found by trial and error) between calls to the HSM.

The way I read the article was that it fails immediately, it just fails in a different way after two seconds.

Perhaps catching the instantly reproducible failure before release would have lead them to the root cause of both.

The images in the 'Digging deeper' section suggest otherwise. They appear to show a successful run followed by one that fails because the 'encrypted' value is garbage. Where am I missing the instantly reproducible failure?

That's where fuzzing comes in.

In what way? 'Temporal' fuzzing to an eon-like range of two seconds seems, naively at least, entirely impractical.

Edit: a somewhat different way of putting this concretely - what is a practical stochastic testing regime that can reasonably be expected to find this bug?

Well designed systems have a mockable clock and timer subsystem.

They could easily test "Do X, wait 100 years, do Y".

You find all kinds of wired bugs when you do that - things that poll, for example Cron daily, will have to be run 36500 times. Certificates expire. Counters overflow. Date systems can't convert the date to and from strings. Logfiles get too big. Etc.

I guess if the answer to 'how do you fuzz time' is 'easily wait 100 years', I have no further questions.

You know you can set the clock / time date of Systems without ntp? :)

The premise was that basic automated testing by some other team at Google (to whom the HSM is a black box) would catch this. I don't see how that's obvious. Then you're all 'but fuzzing!' and I'm like 'wat' and now you're asking me if I know you can set the clock on the HSM? I don't think I know that. It's a black box.

wouldn't you pay to see Brian Cantrill's reaction to anything? :)

Cause I would, if he had a "last week in tech with Brian Cantrill" show I'd subscribe in a minute.


I think Google's handling of security issues in Android has been badly broken for quite a while now. Here's a timeline from another security bug in the same (security critical!) subsystem:

  - 2014-02-24 AM - Discovery.
  - 2014-02-24 PM - Vendor notification.
  - 2014-02-24 PM - Vendor acknowledgement and confirmation.
  - 2014-02-26 - Attempt to setup coordinated disclosure (no response).
  - 2014-04-07 - Public disclosure.
  - 2014-10-17 - Response from Android security team offering line in Android security - acknowledgements.
  - 2014-11-03 - Verified fixed in Android Lollipop.
(The impact of this bug was keystore material leakage between apps. No CVE was assigned!)

That was a long time ago, and of course two anecdotes aren't data. But it makes it all the more interesting to read Project Zero's frustration with poor disclosure practices by others.

Project Zero and Android are a world apart.

Could it be that the first excerpt (https://github.com/beemdevelopment/Aegis/blob/def7c676148f26... ) should not have included the line with .setIsStrongBoxBacked(true) yet?

Edit: funny: 0dd0adde looks a lot like deadd00d (like "dead dude") with a reversed byte order. If this is a coincidence?

Edit2: https://www.bing.com/search?q=deadd00d

Very interesting observation!

"Comments on a number of StackOverflow questions have pointed out that a fault address of deadd00d indicates a deliberate VM abort."

That ending up in the ciphertext multiple times seems to point to some memory corruption issue. It's also a good argument for using magic numbers that stand out.

You're absolutely right about the excerpt. Fixed, thanks.

Another case of Google not following their own rules. But they're happy to hang others out to dry when they exceed 90 days.

In what way did they violate their own rules? Google didn't prevent the researcher from disclosing and the researcher could have disclosed - the timeline describes requests, not demands. For reference, Project Zero's disclosure FAQ:


There are several cases in which deadlines were extended way beyond 90 days. And in the post itself, the researcher points out they could (and, in hindsight, feel they should) have imposed a hard 90 day deadline.

Disclosure doesn’t hang anyone out to dry. Any advance notice to a vendor before publication is a courtesy.

You are not obligated to keep any secrets about your own research into a product that has been publicly released for everyone to do research on.

Can you explain where you see that? In the post itself it says that Google offered coordinated disclosure at 90day mark?

Yep I fully agree, you can't hold others to a standard you're not willing to hold yourself to. I specifically remember Microsoft begging for more time on a bug.

Also if I understood it correctly, it seems as though some devices may require a factory reset to apply the new firmware? If so, for a lot of devices this still isn't fixed.

The big question is indeed how many devices got themselves into this 'bad state'. Your guess is as good as mine.

Author here. I should have made this clear in the blog post, but I'd be interested in seeing boot logs from Pixel 3 (or newer) devices. If the firmware update failed on more devices than just mine, it would be good to know about that. If you'd like to help, first make sure you're at least on the December 2019 security update. To capture the log plug your phone into a computer, run "adb reboot ; sleep 1 ; adb logcat | grep -i citadel" and turn the phone back on. Wait for it to boot, unlock the SIM card and unlock the screen. This should yield the version information of the firmware of Titan M.

Mine got updated it seems

The most important takeaway is to stick to the 90-day disclosure policy. The deadline is only credible when it's enforced, and on a number of occasions Google have stated they believe so themselves. [1]

1. https://www.google.com/search?q="deadline+exceeded"+"automat...

The fact they still haven't gone around to open-sourcing the Titan M firmware is, honestly, what worries me the most about this. It's probably going through some heavy internal auditing now, which is not the best approach, all things considered.

If I were an Android user: I'd feel good with how Google managed this one.

Regarding NSA vs Google (seems like after I commented on a corona virus thread a few times, I was rate limited - editing existing comments still works though):


"NSA infiltrates links to Yahoo, Google data centers worldwide"


"Googlers say “F* you” to NSA, company encrypts internal network"


@monocasa: Got a credible source?

Google is part of PRISM and was (and probs still is) collaborating directly with the government.

Except for the 2013 "Oh shit" slide from the Snowden dump. At least part of Google wasn't on board.

That was MUSCULAR, not PRISM. It was to exceed the access that even PRISM gave them.

PRISM doesn't give the government additional access to anything. It simply ingests data that the FBI has already collected by wiretapping individuals under investigation.

Edit (responding too fast):

> It by default gives government access without anyone at Google or anywhere else granting that access at time of use.

Where did you get that idea? All the documents show that it ingests data the FBI already has, for individuals the companies already manually approved after potentially fighting about it with a judge. You simply made up an illegal system out of whole cloth that wouldn't last a minute in court if anybody challenged it, unlike the phone metadata program, which went through two courts to conclude its illegality.

Edit 2:

> Page five lists the companies and page six lists the per company agreement date. Unless you're trying to argue that Google didn't respond to wiretapping requests from the FBI at all before 2009.

The FBI has to set up a system for canonicalizing and routing data from each different company. Those dates list when the FBI did that for each company. Since almost nobody (including suspected terrorists, apparently) uses Apple's email service, their system was the lowest priority to support.

This is well documented, both in Snowden's documents and in documents the government later declassified. Once again, if PRISM were as you described it, it would be flagrantly illegal and shut down long before the phone metadata program.

Edit 3:

iMessage was launched near the end of 2011, and FBI's DITU handles content collection via wiretaps. When are you going to address the fact that the program from your fever dreams is insanely illegal and that it doesn't match any of the documents? If you would like me to respond normally, upvote my comments, so I don't get rate-limited.

It's automated systems for those requests. It by default gives government access without anyone at Google or anywhere else granting that access at time of use.

It does record the request though which is why NSA tried to exceed the bounds of that with MUSCULAR.

Edit to respond to your edit: Page five lists the companies and page six lists the per company agreement date. Unless you're trying to argue that Google didn't respond to wiretapping requests from the FBI at all before 2009.

Edit 2 since apparently this is how we're doing this:

> Since almost nobody (including suspected terrorists, apparently) uses Apple's email service, their system was the lowest priority to support.

There's a fuck ton of metadata that iMessage reports back up; PRISM isn't just about email. And yes, iPhones are the most common smartphone in the world. I guarantee you that Apple isn't last because they were a low priority, that's absolutely absurd.

Edit 3:

Your argument that "it would be illegal and shutdown like the other illegal programs documented here if it were actually illegal" has to be one of the hottest takes I've heard.

And the PRISM collection was part of what the Supreme Court dismissed not because it isn't illegal, but because you can't prove that affected the claimant personally without a breach to national security, so they can't prove they have standing, so the case had to be dismissed. https://www.aclu.org/files/assets/amnesty_v_clapper_scotus_o...

> they can't prove they have standing,

The plaintiffs in Clapper v Amnesty would have standing if the program worked as you described. No documents have ever been released saying the program works as you described, including the documents Snowden leaked after that case. If such docu6were released, the case would be relitigated. Here is an article describing how it actually works, linking to multiple sources: https://www.cnet.com/news/no-evidence-of-nsas-direct-access-...

> "it would be illegal and shutdown like the other illegal programs documented here if it were actually illegal"

How did one illegal program turn into multiple "illegal programs"? How do you come up with this stuff?

> The plaintiffs in Clapper v Amnesty would have standing if the program worked as you described.

No, because the way the system works is that information makes it's way to the NSA on the presence of certain search terms and is prefiltered before it ends up in their hands. The ruling by the supreme court in the case of PRISM is that amnesty international can't prove that they were among the search terms ever searched for, so they can't prove that they standing. Only if there was a leak of the actual keys slated for collection (or if the NSA agreed to release that, which would never happen), then they could relitigate.

This is in contrast to the bulk call data, where, because the NSA was collecting from everyone who made calls, standing could be confirmed.

> How did one illegal program turn into multiple "illegal programs"?

I'm bundling it up with the other programs Snowden leaked.

> No, because the way the system works

That's not how the system works. The system allows collection of data to/from specific non-Americans outside the US. Amnesty International didn't know that it was for specific individuals at the time they filed their suit, but Snowden's leaks and later the DNI confirmed it.

> I'm bundling it up with the other programs Snowden leaked

Once again, only one of them (phone metadata collection) was illegal. The other programs he leaked, including PRISM, are so legal that nobody with any sense would attempt to challenge them.

No, Google isn't part of PRISM. The FBI is part of PRISM. Google responds to court-ordered data requests, including from the FBI.

The NSA's slides disagree with you. Here is the slide explaining how the data flows. https://imgur.com/setOJIm

PRISM is simply a data integration system that gets data from the FBI's Data Intercept Technology Unit, which is the group that handles Internet communication wiretaps on specific individuals under investigation.

> No, Google isn't part of PRISM.

The slides literally list Google as a data provider as part of the PRISM program.

If you paint with that broad brush, the users whose data PRISM consumes are also part of the PRISM program. That's a fairly useless definition. What people are interested when talking about PRISM is whom the NSA integrates with.

End users didn't build automated systems to comply with government requests and integrate with the PRISM program.

End users also aren't listed in NSA documentation as collaborators.

Google also isn't listed in NSA documentation as collaborators.

Google also didn't integrate with the PRISM program.


Edit (responding too fast):

> Literally page five lists the companies

Page 5 doesn't say they are "collaborators."

> and page six lists the per company agreement dates

Page 6 doesn't say there was an "agreement" with those companies. It simply lists the dates that the FBI made data they have from these companies available for ingestion.

Stop pretending words exist in the documents that don't. That's conspiracy theory nonsense by the exact same method as Pizzagate.


The title of page six is:

"Dates When PRISM Collection Began for Each Provider".

That is not the same as "company agreement", by any stretch of the imagination.

(gaslighting? seriously? you're clearly the person who is trying really, really, hard to twist the facts here)

I don't like Google more than anyone else. The truth though, that's important stuff worth spending time defending.

They're automated systems in support of the 702 program. The companies in question had to have done work to assist, by the nature of how the program works.

Okay. I see what you mean. Take care.

Nothing in those slides substantiates your claim.

The slides literally list Google as a data provider as part of the PRISM program.

No, they list Google as one of the PRISM data sources. It doesn't say if Google was breached or if Google volunteered.



The PRISM slides list per company agreement dates spanning over five years. Microsoft started first in 2007, Google was added in 2009, with Apple being one of the last in 2012. If it was a system that was simply ingesting FBI data, why would it wait until 2012, unless you're suggesting that Apple didn't respond to FBI requests at all until late 2012?

Just because the NSA had a program to exceed even what had been negotiated via PRISM, doesn't mean that PRISM didn't involve collaboration.

@alexbakker: could you please make code in the snippets wrap at least when viewed on a phone? I gain nothing from them being non-wrapping, while having to scroll back and forth to read. In each editor I've used, code soft-wraps at screen edge, so I don't even understand where this trend came from.

(Or rather, I guess it's inherited from ye olde ‘pre’ block, even though snippets are full of their own markup nowadays.)

As a counterpoint, I always disable wrapping in my editor, as I find it much less readable wrapped.

Well, at least you're likely editing on the desktop. On a phone, reading code listings is PDF all over again.

Maybe this mysterious “bug” is a backdoor

Maybe you should refrain from spreading rumors?

Hanlon's razor applies.

I remind you, Google was one of the companies caught collaborating with NSA as per NSA leak by Edward Snowden

The Snowden leak indicated Google's internal network has been physically compromised. That's the opposite of collaboration.

Google is still a big target for the NSA and other espionage organizations.

Two separate programs. They collaborated with PRISM, and MUSCULAR hacked Google to provide even more access than was negotiated.

PRISM was just a fancy NSA source name for “the FBI serves warrants and collects data on our behalf.”

PRISM involved automated systems created by the various companies to comply with FISA requests that could originate from the NSA but would be served by the FBI.

No. The FBI issues Section 702 data requests for individual users to the companies, whose lawyers manually review the requests and may dispute them before a FISC judge. Only after the company approves the request do they start sending data to the FBI's DITU. PRISM consumes the data from DITU's servers.

The system that sends new communications with the monitored individual to the FBI is definitely automated, but configuring an account to be surveilled is a manual process controlled by the company, not the FBI, and certainly not the NSA. The reason you cannot provide documents that say otherwise is that they don't exist. The reason those documents don't exist is that the program that you've described is a conspiracy theory fiction.


Can you give an example of a PRISM request being appealed?

Also, that article is entirely around a quote from Clapper's office that the NSA "does not unilaterally obtain information from the servers of U.S. electronic communication service providers", which we found out literally weeks after that article was openly a lie, at least because of the MUSCULAR program. Like Clapper has openly perjured himself on the specifics of some of these programs.

More recent article about Clapper's perjury on these matters: https://www.usatoday.com/story/opinion/2018/01/19/james-clap...

> Also, that article is entirely around a quote from Clapper's office

No, that quote is merely an update to the article. The bulk of the article quotes people who worked on the Secrion 702 data integrations at the tech comanies, quotes from a former government official who made the requests, quotes from a New York Times article that interviewed other people who worked on the system, quotes from the heads of the companies involved, quotes from former lawyers of the companies, and quotes from investigative joirnalists who specialize in national security. It also mentions that multiple government officials, including Senator Wyden who has long been concerned with government surveillance and has been read into all the programs, confirmed phone metadata collection but none confirmed Greenwald's ridiculous misreading of the PRISM slides. It also gives a layman's description of how Section 702 works, which is what enables this collection. Notably, Section 702 does not enable the government to do what you claim PRISM does. Finally, the article also answers your previous question by giving an example of a company fighting one of these requests. Your characterization of the article is mind-bogglingly inaccurate.

MUSCULAR cannot take arbitrary data off the companies' servers. At best, it can intercept their communications off unencrypted international WAN links. According to Snowden's leaks, this was used to collect email metadata for connection chaining (not email contents), a program that Snowden's documents said that Obama shut down.

> More recent article about Clapper's perjury

That's an opinion piece about a single incident. That incident involved a series of questions asking Clapper if the NSA built dossiers on Americans. It doesn't. Eventually, the questions loosened to whether the NSA collects any data on Americans. They do, but by that time, Clapper had been repeatedly saying No for some time and had not realized that any data also includes what he referred to as metadata. Notably, the phone metadata could only be queried in some fixed set of ways according to Snowden's documents, and tying that data to an individual required a separate request, so the phone metadata couldn't be reasonably thought of as constituting dossier information.

But the bigger issue is that you have absolutely no documents saying he lied about PRISM, and you once again exaggerated by saying he perjured on specifics of multiple programs.

You seem to claim a lot knowledge on the topic, can you tell how you got to know that? What is your occupation?

I got to know that by (1) reading the documents that Snowden actually leaked instead of just the interpretations of those documents from computer illiterates like Greenwald, (2) knowing people who worked on the major email services at the time, and (3) reading the laws behind these programs.

> The reason those documents don't exist is that the program that you've described is a conspiracy theory fiction.

PRISM was once told to be a conspiracy theory fiction

> PRISM was once told to be a conspiracy theory fiction

PRISM as it actually is was never a conspiracy theory. PRISM as Greenwald described it was and remains a conspiracy theory.

Indeed. And there is evidence PRISM is not.

If there's evidence for the other program you're describing, it too can move out of the realm of conspiracy theory fiction. But until there is...

They were both compromised, and being coaxed into collaboration.

It was on the powerpoint in the leak with the list of "industry partners"

As stated by Monocasa above, Google was double penetrated by MUSCULAR and PRISM:

PRISM for semi legal metadata access

And MUSCULAR for whatever they did not get with above

The NSA literally dug a hole in the ground via Level 3 to get into that network.

That isn't collaboration.

Was/is there any US corp not cooperating?

Yes, Qwest (the telecom, formerly US West) resisted and fought tooth and nail to not cooperate.

Their CEO was prosecuted to hell and back for daring to do this, and the company was forced to sell to a competitor. Nobody even remembers his name anymore, few people even remember Qwest.

His name is Joseph Nacchio. If this story is true (and it has been around for many years), he is a hero for standing up for transparent governance, and the privacy of Qwest's customers.

I don't know anything about it, but I think there may still be related litigation going on.

Edit: maybe not. Wikipedia says he was convicted in 2007. However, I was given an NDA regarding something that sounded related at some point after that, so...

Naccchio and Qwest aquired US West.

You skipped the part where he acquired a competitor under false accounting, contributing to monopolization, as dumping his stock with irregular sales while to profit from value NSA contracts before the public knew they were canceled. Should a CEO make a fortune selling stock while the company loses 90% of its market cap?

Yeah it's funny how he was the only corrupt CEO in all of corporate America during that time. So weird how nobody else besides him and Martha Stewart ever got prosecuted for insider trading despite it happening ALL THE TIME EVERY DAY EVERYWHERE.


> Former Qwest CEO Joseph Nacchio, alleged in appeal documents that the NSA requested that Qwest participate in its wiretapping program more than six months before September 11, 2001. Nacchio recalls the meeting as occurring on February 27, 2001. Nacchio further claims that the NSA cancelled a lucrative contract with Qwest as a result of Qwest's refusal to participate in the wiretapping program. Nacchio surrendered April 14, 2009 to a federal prison camp in Schuylkill, Pennsylvania to begin serving a six-year sentence for an insider trading conviction. The United States Supreme Court denied bail pending appeal the same day.

> NSA cancelled a lucrative contract

He was happy to get paid hundreds of millions of dollars for spying on his customers, but didn't want to spy on his customers? What did he think those contracts were for?

> What did he think those contracts were for?

Maybe internet transit or MPLS services, which were the company’s core product?

I won’t argue for a second that he was a good guy, but CEOs rarely get involved in the details of customer contracts. Especially in giant telcos.

Feels like there's a lesson in this story.

I don't know what the lesson is after reading his wikipedia page. His refusal to cooperate with the NSA seems like a non-sequitur as a response to the whole insider trading and fraud thing. I mean, if the prosecution was revenge, well, ok, but I can't connect the dots to how that makes him not guilty.

Never attribute to malice that which is adequately explained by stupidity (it is not stupidity, but more not thinking it through).

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact