Hacker News new | past | comments | ask | show | jobs | submit login
Full key extraction of Nvidia TSEC (githubusercontent.com)
629 points by vitplister 4 days ago | hide | past | favorite | 167 comments

As a boring software engineer nowadays, these hackers "for fun and profit" make me proud of our profession. They're like a Robin Hood version of Alan Turing & co. working on cracking the Enigma encryption. No matter how tight the black box is, there is always a gap somewhere.

I've done some reversing when I was younger, cracked some software and hardware locks, there's nothing as exhilarating as breaking through something that looked impossible. Well done!

Just recently bumped into a podcast on the xbox hacker scene (https://darknetdiaries.com/episode/45/) and was really fascinated by the dedication, ingenuity and talent of these people. I am also amazed that you don't hear often enough stories of them receiving job offers from the companies they targeted/"hacked".

I used to follow he Xbox 360 hacking scene and the number of things these guys did would blow your mind away. Most notable when Microsoft increased the size of their games from the standard 7.5gb to something like 8gb. There were no 8gb DVD’s on market where you could burn the games. Initially hackers truncated the games and it worked for a bit with unnecessary buffer data removed but Microsoft got wise to this and found out how to detect it and banned a bunch of people using truncated games. What happened next was amazing. Hackers found a way to flash the DVD burner drive of certain models to actually burn 8gb on. 7.5gb disc! The outside edge of the disc is actually not entirely used the disc writing software would leave it alone I believe reason being it is inconsistent in quality at the very edge and you may get a bad write if you use it. Well hackers didn’t care if you may get a bad write you may also get a good one. They hacked certain DVD drives I believe the one I ended up buying was a lite-on drive with certain firmware. I then flashed its firmware and was now able to write on the outer edge of the disc doing the previously impossible. That is only one scene form the Xbox hacking days I fondly remember. Genius if you ask me.

I always thought the best cat and mouse example was the Xbox 360 drive firmware angle tests.

The drive would report via some “secure” firmware if the disc passed detection or not. So the hackers made a firmware that reported good on a failure, ways to flash the drives over SATA, etc.

But either Microsoft was very clever or the hackers made a mistake… the drive would report the angle of the disc during certain movements. It would do some operation and report it went from 20degrees to 223 degrees. Well, the hackers and MS disagreed on an angle integer rollover.

The original drive would report 0-359degrees, but the hacked drive rolled over different and reported 0-360degrees or vice versa, I don’t remember. So iirc, MS listened for awhile, if a drive ever reported 360 degrees or whatever the wrong indication was, MS added it to a list.

One day, the drop the hammer banned the lot of them. It took the hackers awhile to figure out how they were getting caught. In the meantime, I now had an Offline-Only 360.

It’s like overclocking “what if I told the cpu to… just go faster?”

For real. The talent in the game industry is insane. Recently, I got my mind blown by the Unreal Nanite team:


The core of it is a respectably sophisticated LOD building algorithm, as you might expect, but the sheer amount of engineering horsepower they put into driving it into production just boggles the mind.

Oh cool talk! I was impressed by that system when it was announced and I had no idea they had released an explainer!

If you haven't seen it, there's a great talk on YouTube on the Xbox One's security. Surprisingly, it's from a senior Microsoft engineer.

Guarding Against Physical Attacks: The Xbox One Story — Tony Chen, Microsoft - https://www.youtube.com/watch?v=U7VwtOrwceo

Discussion: https://news.ycombinator.com/item?id=21325421

What's great is that it's mostly done for the street cred. To show off how cool you are, you need to work in the public, and present your results for all to see, for free.

"This is how cool I am. I have cracked it before everybody else."

They're positioned on the complete opposite of the modern corporate, capitalist Internet, keeping computers open and still cool. For that, I salute them.

While I'm sure street cred plays a factor there is also good money in being able to break the protections to enable piracy.

I don't think this is true anymore, having had experience in this space.

In the "bad old days," you could make a moderate (nothing like Silicon Valley engineer money) sum by selling exploits to modchip manufacturers, as they'd then use this to drive their hardware sales - pretty simple model. The last one of these I remember being particularly popular was the PS3 "True Blue" dongle.

These days, exploits aren't particularly useful to drive hardware sales as they're mostly hardware free. So there's not a ton of monetary value - yes, you could try to sell a "custom firmware" for a few months, but once the exploit is reversed, it's game over for your income stream.

Cheating is probably the only major revenue stream left in console exploitation, and as far as I know it's not popular enough to drive high prices for console exploits. Compared to phone exploits (wanted by nation-level actors and shady security firms for mostly evil purposes), ECU exploits (easier to protect and worth more per install), and PC exploit bug bounties, I think console hacking is pretty low on the lucrative scale, which is why so much more of it is done in the open.

There seems to be some suspicion that there is a cottage industry forming around undetectable cheats for streamers... Not sure how true this is.

This is absolutely true in the PC space, cheating is a growing business.

There was a thread on HN about this the other day - at the most advanced end, bus mastering DMA devices are used to dump game memory for direct inspection, or to recover ephemeral / session negotiated keys used to secure client<->server traffic, and then dump or inject network traffic on a separate machine. PCIe FPGA cards are the most popular tool for this, but there are other approaches given anything with DMA mastering can be employed to sneak data out without the OS or user land knowing much about it.

There's also a big middle ground which is just a software cat and mouse game between detectability and effect - just like antivirus, anti-cheat is an uphill battle on machines where users can run whatever code they'd like.

Many of these cheating services are subscription based so they're pretty lucrative for the authors.

But, I'm not aware of as much (or really, any) of this going on in the console space. There aren't that many competitive console streamers to start with, and console eSports events generally use tournament-provided hardware. So, the possible revenue stream doesn't really reach the massive undertaking that would be required to break modern console security on anything but the Switch.

This wouldn't surprise me in the least, in shooting games. People don't like watching someone getting their ass kicked.

Also, to succeed as a streamer you have to stream ~40 hours a week or more, and there's something called "aim fatigue". After an hour or so without breaks, your aim goes downhill. Anyone who maintains amazing aim for hours of continuous play is cheating. That's why you see experienced, successful streamers taking breaks, or interspersing "hang out time" or a non-aim-based game, etc.

Stream framerates / compression can make it difficult to tell what's going on, and using a controller means it's nearly impossible to see whether their controller movement matches on-screen movement. But controller aim assist is so strong in many games these days that if you have experience with a controller you can easily dominate all but the top mouse and keyboard players.

Shooting-based games just aren't fun these days. Between the cheaters and the streamers you get your ass handed to you pretty regularly, except when matchmaking throws you an easy game to keep you from rage-quitting.

They haven't been fun for a long time. I realized ages ago performance was mostly ping based. You had to have skill if your ping was OK, but if you didn't have a good ping, no amount of skill would help you.

> But controller aim assist is so strong in many games these days that if you have experience with a controller you can easily dominate all but the top mouse and keyboard players.

Man, that's so weird to see.

I've always been told by fellow gamers that a keyboard and mouse is the competitive option, while controllers are for the less experienced.

Are you sure? How exactly would this be monetized?

By publishing the exploit they lose any market advantage.

Darknet Diaries is one of my favorite podcasts. The episode Money Maker [1], about a guy counterfeiting money, is just a fantastic story.

[1] https://darknetdiaries.com/episode/102/

How coincidental, I was reading this two hours ago after finding it in my documents while sorting trough them.

I can really recommend this, if only as a cautionary tale against password reuse.

I love hearing these stories too!

Here's a video of someone finally cracking into the Sega Saturn well after the console was current commodity: https://www.youtube.com/watch?v=jOyfZex7B3E&t=202s

He eventually released a commercial product, the Satiator, and I'm happy to say that it works great and is very well supported by the creator himself and the community!

For those unfamiliar, the Satiator is an adapter of sorts that lets you load Saturn ISOs onto an SD card and play them via the Saturn's MPEG adapter slot.

Unlike many solutions on various consoles that bypass the optical drive, no hardware modifications are required. Your Saturn stays intact; it's truly plug-and-play.

This sort of thing is important. Consoles (specifically moving parts, like the optical drives) and physical media from the 90s are failing. Surviving consoles and games can be quite expensive; even thousands of dollars. Emulation is imperfect and introduces lag. Satiator and other flashcarts let us play these games on original hardware and bypass these issues.

Not game related, but this is what I like about the MagicLantern hack for Canon cameras. It's just some data on your card that gets loaded at boot. If you use a card without the data on it, the camera boots/peforms as a regular stock camera. No hacking of the software on the camera itself.

I've always been curious how MagicLantern achieves this.

Wouldn't the original firmware need code to boot from SD? Maybe it automatically checks for new firmware on the card and then gets exploited from there? Does anyone know?

I always imagined it like a BIOS setting allowing you to choose boot device order. The camera looks to see if something is available on the card first, if not then it just boots to camera. It's been so long since I've set up my camera to use ML, but I know I had to upgrade the firmware to a compatible version. Then IIRC, you load the ML data on the card and tell the camera to upgrade firmware again. So it maybe something that it thinks it is a firmware upgrade but just hijacks that process to boot a full thing instead. ???

The concept of undervolting the chip, causing bitflips, to do a differential fault analysis[0] seems like a stroke of genius. I had no idea AES could be broken in such a fashion, of interfering with just the last 1-2 rounds of the cipher.

I wonder if it will be mitigated by requiring a larger minimum voltage?

[0] https://en.wikipedia.org/wiki/Differential_fault_analysis

Environmental sensors are common in security critical hardware, but very hard to get right.

The problem is that this doesn't work for high performance CPUs, so it can only really be done for things like TSEC or Apple's SEP. You need the CPU to have a large margin where it operates correctly, so the sensor doesn't have to be extremely fast and accurate (which is nigh impossible with on-chip sensors with no external reference). And even then it has to be able to detect things like microsecond-long dips in the supply voltage, or extreme temperatures, or a single clock pulse that's too short. It's really hard.

Several years ago, I got the Wii U PowerPC boot ROM keys out by doing a self contained glitching attack like this. In that case it was a reset glitch, where I pulsed the reset line for less than the required 256 clock cycles (from the ARM core which controls it). At very specific pulse widths (that varied from device to device), that got the CPU into an inconsistent execution state that eventually fell out of the Boot ROM and into code I'd placed in RAM. That one could've easily been mitigated by a reset stretcher circuit, but it's another fun example of a "self-glitch" attack with no additional hardware.

Voltage glitching for secret/rom extraction has a long and storied history.

It's the basis of how DirecTV cards were compromised in the late 90s/early 00s.

Or a good cover for the leakers

Leaker conspiracy theories are almost never true in this field. E.g. I happen to know for a fact that the HDCP master key that "leaked" a few years back was computed from a pile of device keys (using an attack that had been known almost since HDCP's inception), not leaked from Intel.

It's not a cover story in all cases.

The best video about it that I've seen is from 33c3 https://youtu.be/lhbSD1Jba0Q

He bought a carful of cable boxes and glitched them into submission using a NOP slide and custom code to extract the firmware, IIRC over I2C.

It was an awesome talk and really explained it well!!

That was an amazing talk when it first came out. I didn't fully understand the process but I wish the speaker would do more talks. His clear Canadian accent and method of explanation is wonderful!

Has anyone ever hacked DishNetwork/Echostar/Nagravision cards? 10+ years ago I was fascinated to learn about how the new at the time "Nagra3" cards worked so well. I never did find out how those things worked as they seemed impossible to break. I also vaguely remember the story of how Directv owners (owners of FOX networks) secretly commissioned some hackers in Israel to crack their competitors systems while at the same time securing their system further. They eventually got sued but paid a pittance in restitution while their competitors suffered serious financial issues. Really unethical behavior.

> Directv owners (owners of FOX networks) secretly commissioned some hackers

It wasn't much of a secret.

Newscorp (the FOX assholes) didn't control DTV until 2003. Prior to that, however, DTV utilized the services of NDS for security.

NDS was owned by Newscorp, and run by ex-mossad hackers. For some strange reason DTV was constantly owned again and again while NDS was responsible for security at DTV.

In 2002, DTV rolled out a new access card without the involvement of NDS, and by golly, it was secure (and remained secure for years).

In 2003, Newscorp said fuck that, and bought a controlling interest in DTV.

I've seen the technique reproduced enough that I'm pretty convinced it's not a cover.

Ha, that mentions "DFA was also applied on AES", citing a 2005 reference. Is the OP article being understatedly humble? ;)

> I wonder if it will be mitigated by requiring a larger minimum voltage?

How would you detect low voltage without a reference voltage?

You can put a diode and see if it turns or. Or manufacture a transistor with the specific gate voltage and see if that turns on. You would have to account for things like low inductance (to protect against short, sharp dips), etc. Obviously this is a very naive solution, but the point is you don't need a high enough reference voltage.

Resistor voltage divider on the power rail, plus a voltage reference that runs well below the minimum voltage of the part. Both go to a comparator, which outputs to whatever you want (like hold the part in power on reset)

p-n junctions have a threshold forward voltage.

which is temperature dependent

Which will not tell you anything without knowing the body bias voltage on CMOS, which is set relative to power rails.

How do you think "reference voltages" work?

(Also, temperature sensors. Sheesh people.)

Either that or pausing crypto operations without the required voltage, but that reduces your fault tolerance.

Holy cow. Reading these recaps I feel a lot the Impostor Syndrome. I think I'm competent on what I do, but when I see those guys it's hard not to feel really dumb.

Congrats to them !

Computer Science is a wide field. You are not an impostor for not understanding all components of the field.

For example, an oncologist is not expected to have the same understanding of bacteria as a microbiologist and they aren't an impostor for not knowing.

But I feel you, whenever I read the more technical posts on HN I feel the same way.

I agree with both of you, but still kinda feel like this is wizardry in action...

Because it comes from that special perspective of being able to xray through the entire stack, all the way down the physics of the electronics driving the thing while keeping the context of all the other layers in mind to see how different parts can be manipulated to get the results you want. I get the most enjoyment out of my work when doing this with higher level layers, but being able to do it at the hardware end must feel exhilarating, because that's really how it all works.

Drilling down into the stack is a skill, and a skill that is very useful for things like debugging low level issues (sometimes what's causing your app to crash is a kernel bug or even a CPU bug). It's something you have to learn, but it's not wizardry. It all comes down to moving electrons at the end of the day.

I find that it helps if you start with things like 8-bit microcontrollers where you're writing C code or even assembly language, yet it's simple enough to wrap your head around easily (make sure you hit the metal; a lot of people doing this start off with Arduino and get stuck with the high level libraries, but then you're not really learning anything about how it works). Then if you know a bit about digital logic, it's not hard to learn how such an 8-bit CPU might be designed. At that point you can shift to larger and larger systems while still being able to "connect the dots" from a GUI app down to NAND gates. Of course you won't know the details of the whole stack (nobody does), but you'll have enough of a big picture overview to be able to dig your way through it when it becomes necessary.

In this case it's computer science paired with a bit of hardware engineering ;)

Same, but I keep reminding me that I grew up with different interests and a different set of skill sets. Web development and the like sounds boring in comparison though.

And maybe these guys would have trouble delivering week after week productive changes in a system used for important things by important people.

After all, all this genius did is open a nintendo switch to eventually run software made by you for free. The contrary of productive work I d say and close to parasitism.

Or so you can tell yourself when you feel down :D

I think that allowing me to run whichever software I choose, on a device I paid lots of money for is, is a great service to society. Essentially the exact opposite of parasitism, especially considering these people are doing it for free.

Yes, but you did buy the device knowing full well its limitations and obviously felt it was worth it in the end.

The trillion dollar question is how to get rid of parasitic corporations (capitalism) ruining our lives for a profit, making it so we need dedicated hackers to do something as "simple" as owning our hardware.

I think voting… more

How so? Has voting ever brought any major social change? If anything, history has shown that when a vote threatens the status quo, an armed coup financed by the industry (and often foreign imperialism) will take place (see: Allende, Sankara, etc).

I think the situation about voting is summarized pretty well by this funny rap clip form the 2016 USA elections: https://yewtu.be/watch?v=D9Qv6S_GtLo

Most big societal change was done through riots, not so much voting.

Boston Tea Party, Union strikes/riots, guillotine.

Oligarchs slow boil the proles until they pop.

No trouble, but man, how boring.

The same, of course, but now I know this: if you want to learn, you will learn. If you spend 10 years keeping the passion, you will become a magician. Most of us still have at least 10 years of life remaining to spend on something exciting.

Take into account that this particular post glosses over a looot of stuff. A more detailed write-up would not make you feel that dumb.

>A more detailed write-up would not make you feel that dumb.

Errr, no. Just because you would have more details and everything would seem easier to understand and it's simple to sit on the couch and be like "yeah that looks simple, I could have done that", because that still glosses over the huge amount of research and the number of trails and errors the author put in not just during this project but throughout his career to get to this point where everything Just Works™ for him so he can make it look simple for everyone else.

Just like with Edison and the light bulb, even if the end result seems very simple in hindsight, the process of getting there is definitely not.

I still fall into that same trap. Oh wow these guys amazing (they probably are). But when you watch a presentation it has this weird effect of compressing what took them weeks/months/years to do into a 45 min presentation, making you feel really bad. I try to remind jr devs the same thing. I am showing you this thing I have been working on. They had no idea I was working on it. But suddenly I have hundreds of lines of code and it all 'just works'. It makes me look like a wizard to them. But I remind them I spent the past 4 months working on that thing, do not feel bad or stupid. It just takes time and failing thousands of times over and over.

This is like studying a technical subject in university... As you said, you wouldn't know how they reached that knowledge, but at least you would understand how the matter itself works.

Maybe, or maybe not. Presumably that "more detailed write-up" would be something like the third footnote: https://yifan.lu/2019/02/22/attacking-hardware-aes-with-dfa/. Despite being much more detailed, I still felt just about as dumb! :)

To loosely paraphase Feynman - the fact the author can explain it so clearly to those not deep in the field demonstrates a real understanding and depth.

(Feynman said something like, if you can't explain it to a layman you don't really understand it.)

on the contrary, it'd make me feel a lot dumber...

You can do a lot with persistence too. I recently tried to reverse engineer some software protection. Ultimately I decided to cut my losses but the 2-3 weeks I spent on it was a very gradual chipping into more and more advanced stuff. Now imagine having years of experience, and chipping away over years. One "wow" moment in a blog article might have taken 3 months of headaches to reach.

Reading the blog post, it sounds like a supergenius' long weekend, so the timeline O(months) is not to be skipped over. 1% genius, 99% perspiration. Or more recently, "an overnight success after trying for 10 years".

I sometimes feel like that and agree with other comment about the oncologist.

To add one cent: most of us have to be "productive" in a sense that requires us to use a LOT of abstractions.

The author went the other way, understanding how things work under the hood in quite a deep level.

It's truly awesome, but you also have to appreciate how abstractions have their place in modern software dev industry.

I've been out of the security game for a while, this almost reads like fiction. Good god this is nuts. I've heard of extracting keys with timing attacks but this is even more impressive!

Also, apparently the Switch perma-pwn got pwned? Sad face...

Also also, I hope other popular cryptoprocessors aren't so vulnerable?

> Also also, I hope other popular cryptoprocessors aren't so vulnerable?

You might be surprised, but also this chip wasn't intended to be used to secure a chain of trust but had to be press ganged into service after being let down by the main bootrom, which was done by a team at NVidia without much experience of doing these things and made a lot of elementary errors. And being used for a games console is painting a big target on your back.

But ultimately a lot of secure chipset areas have been subject to a lot of... learning on the job shall we say. Things are much better than they used to be, but you don't have to go back many years before things get very hairy. People constantly say they want more OS version support for Android, but I would not want to use a five year old processor from Samsung or Qualcomm if I cared about the hardware backed security on my phone.

> but you don't have to go back many years before things get very hairy

For the NV TSEC-equivalent Falcon successor on Ampere, it’s indeed not vulnerable to this attack because that security subsystem was made much more secure.

But that’s an arch released in… 2020.

> People constantly say they want more OS version support for Android, but I would not want to use a five year old processor from Samsung or Qualcomm if I cared about the hardware backed security on my phone.

What I would really like is a modern Android that doesn't brick half the security features by e-fuse when I root it and many apps refuse to run properly afterwards - why the fuck, for example, does the PayPal app refuse fingerprint unlocking after rooting but other apps don't?! All this incentivizes me as the user is to choose an insecure password that I can actually remember.

Its a small miracle paypal doesnt flat out prevent you from running the app when rooted.

I'm fairly certain you're not eFused out, but you would have to give up your root and let your phone powerwash itself if you wanted things to go back to normal.

Samsung Knox fires an e-fuse the moment you flash an alternative boot image.

That's gross, sorry to hear that.

OnePlus does not do this.

> Also, apparently the Switch perma-pwn got pwned? Sad face...

All erista units (the ones with the bootrom flaw) are still pwnable.

6.2.0 was released on November 19, 2018 - first indication of a new hack was posted on twitter Nov 24, 2018 by @elmirorac and atmosphere 0.8.0 was released on Nov 29, 2018. So the fix he talks about in the paste lasted for around 10 days before a new one was generally available.

That's why he says:

> And it would have been perfect if not for the many security flaws in TSEC secure boot.

Heh, I had a little laugh to myself reading this bit:

> (2) its own "secure boot"

As soon as you see the quotes, you know what's coming! It's like Chekhov's gun :)

Some features in NVIDIA chipsets, like changing the operating frequency, needs (hardware checked, I think) signed binary blobs. This prevents the open source nouveau driver from achieving good performance. Does this hack helps in this front?

no, this is about Tegra platform

That doesn't necessarily rule it out. Cryptography is one of those things that once you get it down, you stick with it. It is entirely possible this may give enough insight into Nvidia's SOP with cryptography to extend the PoC proven on Tegra to something like their firmware signing functionality in GM 204 cards. At least, I haven't seen anything that stands out enough to downright disclude the possibility from possibly nudging things down the road, in theory.

There it is. Money shot in one of the references:

>Because this is a (unmitigable!) hardware issue in all Falcons which have SCP, not just TSEC -- we were also able to use the same attack on the Falcon unit used for GPU power management, recovering its (different) signing key as well.

In short, the same methodology, assuming you put a crap ton of time into reading this, and really grokking it, suggests this attack could be applied not just to Tegra, but any secretful Falcon.

Hat is off to the author for executing like that. I feel like I missed out by never playing games, as cracking them seems like the real game behind the game.

From a design perspective, this is why you don't have your entire ecosystem depend on a shared secret stored in secure hardware, even if they're written when the chip is still in the flasher at the fab. You need either to diversify your keys in the flasher, or do an initialization/personalization protocol to update the keys to new unique per-console ones so that a crack like this isn't portable across every other customer device. As a design consideration, it means the customer has to be online to personalize the device to get their unique keys, but that's the trade off.

The beauty of demonstrating this attack is that if you think game consoles with security modules are vulnerable to having ecosystem compromising shared secrets extracted, wait until you see phones.

……or this is why you let people own the hardware they purchase and give up on this ridiculous “secure” boot idea?

some people forget, or don't know about, Atari's first consoles, which had zero "secure" features and the enormous mess that caused. the entire home video game industry in the US virtually died for a while because of all the crap games that were produced. imagine mobile-like shovelware games selling for $60 each for a while, and consumers being unable to tell which games were good and which were bad until you took them home and played them. then, finding out you can't return the crap ones because too many other people were returning those same games, and the retailers couldn't absorb the cost.

then, seeing all console video games in "bargain bins" for $1-$2 each and even then extremely few people were buying them. I recall seeing bins going virtually untouched for months.

parents just stopped buying games for their kids' consoles, nearly completely. the idea of a home video game console was so negative that Nintendo needed to call the NES an Entertainment System in order to get their device into homes. That Nintendo Seal of Quality really meant something, and only Nintendo could manufacture the lockout chip that prevented unauthorized games from running, so that Seal of Quality really had weight and it basically meant "no shovelware games" for it's entire existence.

Entrepreneurs showed Nintendo and Atari what happens when you have no console security: that lots and lots of people will eagerly crush the entire market in exchange for a bit of money. Nintendo has not forgotten this lesson, and they're not likely to.

How does hardware/software security solve this? Is this not just a certification issue? If Atari had officially blessed games and told retailers which games were good, would that not have solved the issue? I guess securing the device to not load unauthorised games is a solution, but isn’t just certifying games and publishing information about the good ones enough? I guess distributing information widely was a lot harder back then.

If the reason for people buying shovelware is that they need to go home and try it to see if it’s bad, wouldn’t a simple blank CD packaged as a game achieve the same thing?

do you really think that game publishers would simply obey orders from a console manufacturer who told them to "only make good games"? game publishers take every shortcut possible when publishing games, including (but not limited to) strongarming the developers, lying via box art and/or wording on packaging, and so on. game publishers optimize for profit, not game quality.

even with the NES CIC chip keeping most unauthorized games from working, Atari successfully duplicated the functionality of the chip and published their own games under the "Tengen" brand. Nintendo was very unhappy about this, and while Nintendo won in court, they learned their lesson and doubled-down on console protections from then on.

Nintendo are unlikely to ever forget the lesson that this court case taught them.

So, with that, even with the presence of technological measures to keep non-nintendo-made carts out of the NES, there were still non-nintendo games running inside NES consoles.

things like this are why Nintendo, Microsoft, and Sony are so dedicated to locking the consoles down and doing everything they can to prevent unauthorized access. Nintendo has been shown multiple times that companies will just do whatever they want unless there are strong lockdown features to the hardware.

others have learned from Nintendo's experience, as well, and that's why we are where we are today. we will not be returning to the early days of video game consoles, where hardware was unprotected.

That’s fair. Nintendo can’t make the game publishers do anything in an open ecosystem. What they can do is maintain a list of good game publishers and encourage people to buy from that list. Again, arguably easier to do now with say the Nintendo website, then back before high availability of internet.

From Nintendo’s perspective, locked hardware makes sense. They risk being locked out of profits from other publishers.

However, I’m not convinced that’s the case from a consumer perspective. The reason you gave in the original comment was that consumers suffered from the open platform. I still disagree with that. Consumers suffered because the platform was open AND there was no trusted source for quality control. You don’t have to take away 1) for consumers to not accidentally get terrible games, you just have to have a trusted source which tells consumers which games are good so that 2) isn’t the case. As soon as consumers know they can buy good games directly from Nintendo (or from blessed retailers), the onus for running crapware is on them.

well if you can find a way to control quality while letting anyone who owns your hardware to do as they please, I would encourage you to share that info, in detail, with the world, because no one else has figured that out, yet.

Nintendo won't even sell development kits for the Switch to just anyone. you have to have a "good enough" game pitch (with no published rules on what is good enough and what is not) and you must commit to actually producing the game before they will even let you SEE the development kit and related items in the developer account store.

of course you can buy dev kits for the Wii U and 3DS, the discontinued systems, but you can't produce software that runs on the retail hardware, even for those.

Have you heard of Steam? PC is open hardware. There is a minimum bar of quality. Steam controls which games you can buy through it. Very profitable too.

Steam theoretically can control quality as much as they want. The bar they choose is entirely arbitrary and completely upto them.

I did not know about that, very interesting!

> If you can get 1-2 bitflips in the last two rounds, you can solve for the key.

What about the bit flips allows the key to be solved for? That is the part of this I don't understand

The security of cryptographic primitives relies on the mixing that happens from the first round to the last round. If you can analyze the input-output relationship of a single round, you can easily derive the round key (and the way the AES key schedule works, if you have any round key you can run it backwards to derive the original key). Bit flips in the middle of the algorithm allow you to do just that; if you have a bit flip in the second to last round, that'll have a specific effect on the output, and that effect will depend on a small part of the round key. Collect enough samples and you can solve for it. It turns the problem of brute forcing a 128 bit key into the problem of brute forcing a few bits at a time, because with only a round or two there is very little diffusion, i.e. every output bit only depends on a few key bits.

I've done the same thing to break "white-box" AES implementations, which are software versions of AES with the algorithm obfuscated and the key baked into it, in the form of flattened per-round-byte lookup tables (this concept is complete snake oil, but a few companies insist on selling it; they claim it's hard or impossible to get the key out, but this method works every time). You can introduce faults by patching the code or using a debugger to change state in the last round or two, and compute the key from the results. I did a targeted attack where I surgically introduced faults by replacing intermediate values with ones from a different input (which works even when the algorithm uses redundant, booby trapped encodings, which is another feature these vendors peddle), but in most cases you can also just literally randomly corrupt execution and use the same script Yifanlu wrote, just like a random hardware glitching attack.

It seems from accomplishments like this from amateurs that state level actors will have compromised any current "secure" or "trusted" computing platform.

I'm not sure why you claim that this is an amateur. Yes, this is likely a project done in their spare time, but there is no reason to think that this person is not a professional security analyst

These appear to be sha256 hashes of the keys, not the keys themselves...?

I assume that's so it doesn't get immediately DMCA'd. They can either distribute the keys separately or other people follow the paper and the keys will become known.

This seems to indicate that this involves the Nintendo Switch, but that it only involves older models where the first layer of security was broken and now a second. Wouldn't the new models have patched the first layer of security by now where this wouldn't result in anything of value?

Yes, HAC-0001-001 (better battery life) and HDH-001 (Lite) use a new ROM revision with this patched out. But there are a lot of original Switches out there.

Why aren't more devices use smartcards for signing/crypto? They are omnipresent, satellite TV receivers had them, phones have them, banking cards ARE them. And yet gaming console manufacturers would rather invent their own measures to combat pwnage/piracy.

> satellite TV receivers had them

And regularly get their card security schemes busted.

The advantage that smartcards currently have is that not many people are looking into their security outside of pay-TV pirates. Phone users don't have a need to hack their own SIM cards, friends of OpenBTS simply use blank SIM cards, bank users don't need to hack their own cards, and card cloners have a hard time getting physical access to the chip on a victim card for long enough to run a software-based attack (since the ATM eats the card and spits it out when done, it is easy enough for a skimmer device to clone the stripe while the card passes, but outright impossible to establish electrical contact with the chip).

What is interesting to hackers is anything where NFC can be exploited, and as a result - at least to my knowledge - there currently is no tag-based authentication that can't be cloned.

> And regularly get their card security schemes busted.

It was wild west, proper ones were never cracked to my knowledge. But it's probably safe to say because of that they are as secure as they are now.


> there currently is no tag-based authentication that can't be cloned.

Smartcards use ISO 14443, not NFC. Those are related standards I think.


MIFARE tags and cards (which you can clone) are not smartcards (which you can't clone)


> It was wild west, proper ones were never cracked to my knowledge.

Some did, some got close. Most card based systems have to replace the cards every five years or so when their intel suggests groups are getting nearer cracking them.

Ultimately the reason is cost. The conditional access vendors who work on those cards spend a lot lot lot of money on it because that's their entire business and they need to work for a long time or you pay out tens of millions in postage and support. Most chipset vendors shrug their shoulders after two years because it's out of support and they don't care, everyone else gets broken too. In this case Nintendo probably got some significant compensation of Nvidia that hurt them, but it was Nvidia's first real go at it and they fluffed it.

> Most card based systems have to replace the cards every five years or so when their intel suggests groups are getting nearer cracking them.

Do they still? Proper smartcards like javacards can be updated by the receiver, it wouldn't affect existing customers if there's a protocol version change or something like that. It's pirates and missed revenue they need to worry about. In any case with internet so widespread those satellite TV networks were on their way out for a long time.

Do ATMs in the US still read the magstripe?

Yes, unfortunately. We're a more than a little bit behind the rest of the civilized world.

Even my debit card which includes a mandatory EMV smartcard provides for an easy downgrade attack if the chip fails to read three times.

A cheaply made smartcard chip should be much easier to breach than the main SoC. Both because it's way simpler, older process silicon… and much more importantly because you have an external bus going to it. Now that the big consoles use semi-custom SoCs, their NUMBER ONE principle is to not trust anything that's off the main silicon die.

See the Xbox security talk: https://www.youtube.com/watch?v=U7VwtOrwceo

Nonsense, practice says otherwise. There might be not much incentive for gaming console hackers other than bragging rights, but incentives for state-level actors are monumental. They either can do it and keeping it quiet or they can't. Or maybe it's too impractical. Show me one article where someone claims to have extracted a key from a proper, modern smartcard.

The TSEC is basically a smart card built in to the SoC.

No, it's more like TPM or fTPM it seems to me. Similar in purpose, not similar in structure. And external TPM are just as vulnerable.

so how does this voltage glitching stuff work exactly? are caps to ground/buffers removed/defeated? is it timing dependent? (sounds like they're sending i2c messages to the power circuitry here?) do people do things like setup precise triggers or hook up function generators to kick the supply voltage around and just wait to get lucky?

More commonly, voltage glitching does rely on removing capacitors etc., and extremely precise timing (nanosecond resolution, or below).

However, the fun part about this particular attack is that it does not require quite the same level of precision. The voltage is dropped incrementally until the CPU starts to make faulty calculations. It just so happens that AES is one of the most complex operations, and thus, the first to start faulting.

Hilariously it appears you can just direct the system to turn down its own supply voltage for you (I2C to the PMIC, as you spotted), and that one of the levels has a bit error rate that's low enough to run programs most of the time but triggers a bit flip in AES often enough to leak the key.

A properly designed crypto subsystem that cared about security would detect the voltage drop and either refuse to operate or would have its own local power regulation circuits such that a system level voltage reduction wouldn't impact it.

I presume the CPU here was VERY cost conscious and so trade-offs were made.

I have no background in gaming but have worked with flawed "security" solutions. Often the business does not care that the engineers explain how flawed some security thing is before release, if there's more money to be made by not fixing it then it won't get fixed. Often doing all of the right things is MUCH too expensive, either in dollars, size, or power.

> I presume the CPU here was VERY cost conscious and so trade-offs were made.

It was also more intended as a media chipset and they managed to flog it to Nintendo, and was an early effort by the team involved.

> would have its own local power regulation circuits such that a system level voltage reduction wouldn't impact it.

How do you know if external voltage is low if you check it against... a voltage derived from it?

This is called brownout detection, and it's a common feature even on low-end microcontrollers. As the other commenter says, it's derivable from a bandgap reference.

In re "properly secured system": it's a cost-sensitive games console for children, while there's a big incentive against piracy ultimately there's a limit to how much you can defend. And this is the second line of defense, the bootloader having been breached earlier.

The whole "games console" context also comes waaaay after the chip was made — it's not a custom for-Nintendo chip! tegra210 was previously used in the Nvidia Shield 2015, then the Google Pixel C (also from 2015) and the Jetson TX1 dev board.

Bandgap reference can't tell anything about body bias voltage, unless it itself built on top of insulator

Could you elaborate? Isn't the substrate usually tied to ground so we can ignore it in modelling? Or are you saying that it could be floated separately to ground in order to attack the chip?

Yes, and normally it is. I think on every moderately modern CMOS IC, body bias is controlled.

> What is body bias?

> Body bias is used to dynamically adjust the threshold voltage (Vt) of a CMOS transistor. While CMOS transistors are usually thought of as having three terminal devices, with terminals for the source, gate, and drain, it’s increasingly common to have a fourth terminal connected to the body (substrate). Because the voltage difference between the source voltage (Vs) and body voltage (Vb) affects the Vt, the body can be thought of as a second gate that helps determine how a transistor turns on and off.


Almost all micros have brown out detection so you can tell if the device undervolts, and typically you can force the device to auto shut down/reset if that condition trips using an onboard programmable fuse. I don't know how much of an attack surface your typical BOD circuit is though.


Isn’t that what bandgap voltages are for?


Yes and after a while it isn't luck. Reportedly state actors and bigcorp even go so far as to decap chips, laser off security features/caps/whatever and do stuff like inspect the memory with a microscope. There is another youtube channel that is even more impressive but Liveoverflow https://www.youtube.com/watch?v=FktI4qSjzaE has some videos defeating AES with device that makes it much easier.

One of my favourite talks is Decapping Chips the Hard Way by Adam Laurie and Zac Franken: https://youtu.be/0Z4aF-qiziM

That was 8 years ago

how long before they will introduce special bad sectors on floppy disks, - sorry, wrong epoch, - special half-faulty components required for proper execution in chips to disable chip simulation/reverse engineering.

This effectively already exists. The Xbox One SOC apparently monitors its own voltage and shuts down / doesn't function if it falls out of spec [1] as part of a totally OTT suite of anti-engineer techniques.

As an aside, I hate all of this. If someone made an open console, I'd buy it in a second. I realise that's probably my computer running linux next to me, but still...

[1] https://www.youtube.com/watch?v=U7VwtOrwceo

Almost all microcontrollers have a feature like that - chiefly because at power-on the voltage rises gradually, and if it started running before there was a high enough voltage for it to run correctly you get a bunch of reliability issues.

> If someone made an open console,

Steam Deck?

What is OTT?

over the top

Yes, there are schemes that deliberately exploit difficult-to-reproduce manufacturing defects, for things like fingerprinting / making hard-to-clone things.

The general field is called PUF (physically uncloneable functions)


I’ll never forget the crack that I downloaded back in the days. It came with a PDF or image of a CD you were supposed to print, so you could drill through the burned CD at exactly the proper position. This was supposed to defeat the hardware dongle the software otherwise required. Can’t remember what software it was and never tried it to find out if this was a prank or actually worked, but I loved that as a kid :D

There is no angle position sensor on the CD, and only one continuous track. This could work if you burned multisession CD, scratched at some specific radius of the first session, followed by coding that information into executable burned in the second session.





This one describes very similar floppy protection - encoding physical disk parameters in the executable per every individual disk. https://scarybeastsecurity.blogspot.com/2020/12/the-cleveres...

That sounds unlikely to work. I had a professor at the university who started his lecture on ECC by saying he always drilled a hole in all his CDs since he knew it would work anyway.

Or perhaps the ECC simply works well enough that there's no perceptible data loss for audio?

i think your leg was being pulled.

that said, one of the first copy protection schemes involved writing specific sectors to a magnetic disk with an invalid checksum. when the program would start, it would verify that reading those sectors would return an error, but if you used a regular disk copy program it would not copy the invalid sectors- either resulting in an aborted copy or a copy that would zero out the bad sectors on the target which would not produce the expected read errors on startup telling the program it had been copied.

Wouldn't the CD explode soon after it got to speed? This happened plenty to discs that were just not made uniformly enough.

The Z80 had deceptively-designed transistors designed to throw off reverse engineers: https://twitter.com/kenshirriff/status/1331425858268012545

Now I'm curious, which channel is even more impressive? LiveOverflow is definitely great

Can't find it, however you might be interested in this article from HN earlier in the year. Written/done by an intern.


If you're interested in voltage glitching and other power analysis attacks you should take a look the excellent open source ChipWhisper.




https://embedded.fm/episodes/286 (podcast interview)

Yes to all of your questions. A lot of it can be automated with e.g. a ChipWhisperer.

The magic to me is that the CPU glitches are caused completely on the software side. With dedicated hardware such as flying probe testers this attack is state-of-the-art afaik. But glitching the CPU only with software, i.e., causing hardware bugs only with software is what really surprised me.

Are these numbers “illegal” to share like the BluRay key?

Like this?

> sha256(csecret_01)=43449338c1bc8ceb1b3232a611f955f9095254f492117a158528589cd16f2930 NVIDIA TSEC code signing key

Hopefully not.

Based on the file contents, these are just the sha256 hashes of the keys, not the keys themselves.

Right, but the actual key itself then?

Remember Digg?

Wow, this was the first I had read about m2m i2c injection hacks to mess with the PMIC. That is a clever trick!

What is a TSEC?


  Well, some clever guy ;-) reminded them that the T210 chip (the main CPU)
  has a proprietary NVIDIA "security processor" called TSEC, which has: [2]

      (1) its own SRAM                 (protected from the rest of the system)
      (2) its own "secure boot"        (protected from the rest of the system)
      (3) bus mastering capabilities
      (4) and.. is able to DMA to ARM7's memory

Tegra Security Co-processor

Why are there valuable keys on the device? I (wrongly) assumed the device would only contain public keys to verify signed code.

They're AES keys, so the keys to decrypt and encrypt are the same. (Symmetric key)

To answer my own question, it's because the games are (symmetrical) encrypted.

Skimming switchbrew it looks like there is public key crypto as well. https://switchbrew.org/wiki/NRR

So this key leak doesn't mean homebrew can be signed for unmodified consoles.

Does this txt seem to end early for anyone else? How does one go from the bit-flipped output to the key?

That's covered in the referenced article, https://yifan.lu/2019/02/22/attacking-hardware-aes-with-dfa/, in the `Extracting keys` section

I love everything about this. It has brought joy to my day.

What is the implications of breaching these keys?

Hardware attacks never cease to amaze me. nice work, thanks for sharing

Seriously, security is an utterly pointless field


How is this remotely relevant to the post or any other comments? Please do not spam.


When $NEW_PRODUCT is released, you must dispose of $OLD_PRODUCT and consume $NEW_PRODUCT.

Yes, all the effort all these mega corps put in jailing us could be put to do something actually positive.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact