
Super Micro says review found no malicious chips in motherboards - teddyfrozevelt
https://www.reuters.com/article/us-supermicro-chips/super-micro-says-review-found-no-malicious-chips-in-motherboards-idUSKBN1OA12R
======
exoesquitur
From a technical perspective I found this story compelling, so I tried out a
simple hack to see if it were "possible".

Using an attiny85 uC, a couple resistors, a cap, and a couple diodes I had
laying around, I was able to wire up a two terminal "device" that pretty much
acts like a 5k pull up resistor on a I2C line.... But when you pass data
through the signal line (SDA) wire it can read and modify it. It is crude and
very limited, but it works (only at lower I2C data rates in this case, but
hey, it's a cheap hack).

A nation state adversary could trivially miniaturize this to the size and form
of an SMT resistor, and use a much more capable uC in the process.

Im not saying that this substantiates the Bloomberg story in any way.

Just saying it's a great (black hat) idea, and it works.

It would surprise me a little if this weren't used in the wild by _somebody_.

~~~
lupire
Hardware hacks are easy for anyone who works in that industry. What's hard is
making software that runs on that hardware do anything useful -- it would need
to communicate with external command&control and know how to read interesting
data or send interesting effectful commands to the mainboard.

Making the main board fail arbitrarily would be easy, but controlling the
board or exfiltrating data is hard.

~~~
PeterisP
I have the entirely opposite opinion - once you have managed to attack the
supply chain and covertly deploy, say, some hardware can write a few hundred
arbitrary bytes to the firmware (which was described as the attack vector by
Bloomberg), then that's essentially game over. Perhaps designing the hardware
hack is easy, but getting the malicious chip on the devices shipping to your
targets and keeping it a secret is not trivial.

"communicate with external command&control and know how to read interesting
data or send interesting effectful commands to the mainboard." is hard only in
the sense that it takes some effort, however, this requires pretty much the
same capabilities and skills as every engineered malware we've encountered, so
you can assume that every serious adversary can do it, not only nation state
adversaries but many serious commercial pentesting companies and cybercrime
teams have demonstrated such capabilities.

I can imagine an attacker that can make the "hard" software required but
doesn't have the capability to insert that modified hardware within a supply
chain - as in, it's not even assumption, for pretty much every intelligence
agency it's _known_ that they can easily do software which "would need to
communicate with external command&control and know how to read interesting
data or send interesting effectful commands to the mainboard" \- even just
counting things that have failed (because we've detected and analyzed and
attributed them), there's clear evidence that they can do it because they've
done it many times.

I literally can't imagine an agency that can pull off the supply chain attack
but doesn't have the capability to write software to control the board and
exfiltrate data.

~~~
llama052
I just can't imagine anyone exfiltrating the data on a corporate level at any
scale without raising alarms. It's just not realistic, once it leaves the
board it's pretty easy to see over a network.

Now specific targeted attacks is more believable, at that point though I'd
think a one off MITM hardware swap would be more likely.

------
neya
Let's say Super Micro is right and there were no malicious hardware at all for
sure. What are the consequences for Bloomberg for this incompetence? I mean,
there needs to be something..

Just because you're a news organization, you can't simply escape with "Oh, my
bad". This had real implications on stock prices of so many companies and
wiped off shareholder value on many of them, including Super Micro.

If Bloomberg's story was false, they shouldn't just walk away like that
because "it's the free press".

~~~
finnthehuman
>What are the consequences for Bloomberg for this incompetence?

You, not the general concept of the reader, but you personally neya. You stop
trusting Bloomberg's reporting. That's the consequence. Their reputation
suffers.

Why do threads like this on HN always have such a desire for retribution?

~~~
bob_theslob646
Their reporters are compensated based off of whether or not they move markets.

Did you know that?

([https://www.businessinsider.com/bloomberg-reporters-
compensa...](https://www.businessinsider.com/bloomberg-reporters-
compensation-2013-12))

~~~
admax88q
That doesn't seem like a bad goal. It incentivizes stories that are important
to their target market, and provides an easy way to measure that.

~~~
atoav
The thing is – if you have such an incentive and you are faced with the choice
between reporting a boring truth or a spiced up lie, you will go for the
later. And that has nothing to do with journalism anymore.

It _could_ work – if the editors are espeically on the hunt for bogus stories.

~~~
occamrazor
This was the opening story on their homepage. There us no way the editors (and
the legal department) did not scrutinize it thoroughly.

~~~
tertius
Just because it passes legal muster doesn't mean it's ethical. And I think
that's one of the things that people are calling for.

------
stupidbird
There has to be more to this story that we don't know. Bloomberg has a lot to
lose by publishing such a harsh claim that's not extremely fact-checked.
Reliable newspapers generally don't throw around anonymous government sources
without doing background checks on these people. I have little doubt that they
got the information from who they say they did.

At this point I wonder if they should stop protecting their sources on this
story and see if that shakes out any truth.

The problem with that is of course that it erodes trust for future sources,
and potentially puts these sources at personal risk. But if not doing so
starts putting the entire publication into question, that might be a risk they
have to take.

~~~
admax88q
> At this point I wonder if they should stop protecting their sources on this
> story and see if that shakes out any truth.

That would be a mistake. Don't just throw your sources to the wolves if you
couldn't prove their story.

They should press their sources for proof, or stronger evidence. If they can't
find any then they should issue a retraction.

~~~
crispyambulance
The "should have" pressed the sources for hard proof, or found examples in the
wild of adulterated servers before publishing the story. That ship has sailed,
of course.

But if what was reported was actually true the best thing that can happen now,
for Bloomberg and the public, is for their sources to step forward voluntarily
with proof.

Admittedly, that will take a lot of courage in today's political climate, but
if it's not all just bullshit, highly principled people may do it.

What I would like to know is what the Bloomberg reporters expected would
happen as a result of this story?? Did they think Supermicro and Apple would
just admit it? Then what?

~~~
lupire
If the sources are willing to step forward now, why wouldn't they before?

Sometimes the reason for reporting evidence you have is to motivate other
people to display evidence they have but don't know was interesting or useful
until they saw your story. That's why so often a single public allegation
leads to an avalanche of similar allegations.

------
assblaster
The question I have is: is it possible that there was such an incredible
threat to national security that even an auditor could be convinced by a
federal agency to give a false report?

If it really didn't happen, how could a reputable news agency get a report so
wrong? What exactly is going on here?

~~~
scarhill
WRT the how could they get it so wrong question, I guess it's time for the
obligatory link to Michael Crichton's essay "Why Speculate?" and his
discussion of the "Murray Gell-Mann Amnesia Effect" [1]

Money quote: "You open the newspaper to an article on some subject you know
well. In Murray's case, physics. In mine, show business. You read the article
and see the journalist has absolutely no understanding of either the facts or
the issues. Often, the article is so wrong it actually presents the story
backward—reversing cause and effect. I call these the "wet streets cause rain"
stories. Paper's full of them.

"In any case, you read with exasperation or amusement the multiple errors in a
story, and then turn the page to national or international affairs, and read
as if the rest of the newspaper was somehow more accurate about Palestine than
the baloney you just read. You turn the page, and forget what you know."

1 - [http://larvatus.com/michael-crichton-why-
speculate/](http://larvatus.com/michael-crichton-why-speculate/)

~~~
Symmetry
There really are some publications where you can read an article on a topic
you're familiar with and they get it right. For instance I have a subscription
to The Economist and sometimes their coverage is a bit shallow. And sometimes
it repeats an expert consensus I disagree with. But most of the time the
coverage is as good as it can be in the number of paragraphs allotted and
sometimes it's downright excellent[1]. You probably have to actually pay money
for high quality reporting.

[1][https://www.economist.com/briefing/2018/12/01/the-
semiconduc...](https://www.economist.com/briefing/2018/12/01/the-
semiconductor-industry-and-the-power-of-globalisation)

~~~
ghaff
And I’m guessing a lot of people here conflate simplifying for a mainstream
audience as getting it wrong because they’ve omitted a lot of details.

Mind you, simplifying with a degree of accuracy is difficult and top writers
like those with the Economist do it better than most. With tech stuff, I find
more poor and incomplete explanations than I do outright errors. Mind you,
back when I provided commentary for a lot of news stories, there were some
reporters I always dreaded calls from because I knew steering them in the
right direction was going to take an hour out of my life.

------
thetricia
So correct me if I'm wrong, but the most sinister part of the story in how
some might assume SuperMicro is a Chinese or a MainlandChinese-founded
company. It came right around the time ZTE, Huawei and others were facing
renewed scrutiny. So you can imagine how easy it is to read the story and just
think "oh, another Chinese company got busted".

~~~
dijit
It's also easy to read this as a smear campaign against Chinese companies.

~~~
thetricia
Well the big difference is it's reasonable to assume a Mainland Chinese
company could get banned or sanctioned in some manner.

Now just to be fair, from what I read, there was a lot of Super Micro drama
going on beforehand which likely magnified the pessimism.

Btw if you search for 'supermicro "is a chinese"' you will find some people
that do think that.

------
jamesholden
Has Bloomberg even replied to the "Uhm, WTF you talking about Willis"
responses from Apple/Amazon? I don't recall seeing one. I wonder what they
will say now..

How can they just make up a story like this and it can slide?

~~~
deadbunny
To be super cynical: They got their ad revenue from the story, why would they
care?

~~~
endorphone
The story was given legs because Bloomberg has generally been a credible
organization. They are tainted if they don't firmly explain their side, and
forever anything they report will be coupled with "they were the ones behind
that debunked SuperMicro thing".

Clickbait and manufactured stories is a dead end tactic, so they certainly do
care, and they certainly have some reasoning. Perhaps they were intentionally
mislead.

~~~
Karunamon
It would be worth mentioning that the particular journalists behind this story
(nevermind Bloomberg as a whole) have been caught pushing nonsense before[1].

On top of that, the story itself has a number of technical implausibilities
that render it difficult to believe.[2]

[1]:
[https://twitter.com/RobertMLee/status/1049617855396933632](https://twitter.com/RobertMLee/status/1049617855396933632)

[2]: [https://www.servethehome.com/investigating-implausible-
bloom...](https://www.servethehome.com/investigating-implausible-bloomberg-
supermicro-stories)

------
ineedasername
Evaluating this is tricky. On the one hand, Bloomberg claims it's a well
sourced article, not a single person's unsubstantiated claim. On the other,
Super Micro claims an audit showed nothing, but then they have an incentive to
be less than honest, or to have performed a very superficial check. And
couldn't an audit simply miss the issue if the malicious functionality were
embedded in an otherwise legitimate chip?

Either way, it seems like Bloomberg really should have pushed for a particular
example, e.g. "look at this on model X boards for confirmation"

~~~
fraudsyndrome
Super Micro didn't perform the audit of themselves (that would be silly), it
was done by Nardello & Co as per the article.

~~~
ineedasername
Yes, but the level of detail requested for the audit would be dictated by
Super Micro. For example, if the request was to audit boards against the
design specs, the audit would never catch something inserted during the design
phase. Nor would it catch malicious functionality inserted as part of a
legitimate chip. It seems like the potential number of attack vectors is
extremely high if the design &/or manufacture process has been subverted.

Alternatively, if there was no attack, it becomes exceedingly difficult to
prove the negative. But that also leaves us with the perplexing situation of
multiple sources-- 17 from different companies and NSA-- deceiving Bloomberg
reporters. Or Bloomberg reporters themselves deceiving everyone. In the later
case the motives are clear. It's a career-making story that can't easily be
disproved. In the former case the motives are less clear: a desire to smear
Super Micro? Who benefits? A desire to stoke anti-China fear? It's all very
strange.

------
eeZah7Ux
The elephant in the room is that detecting hardware backdoors will remain
practically impossible due to the closedness and secrecy of the industry.

Especially for backdoors installed only on few servers.

Yet, open schematics, publicly available hi-resolution pictures and public,
peer-reviewed, automated inspection are potentially possible.

------
lsc
I'd be super interested in the follow up, and not just for monetary reasons[1]

I mean, my impression is that there has been a lot of news trying to stir up
US vs China animosity; I mean, I'm sure that a lot of industrial/state
espionage happens, but the renewed focus just seems a little suspicious to me
at a time when our government is trying to distract us from possible Russian
interference in our election.

Makes me wonder about huawei - is this all just our government trying to take
the heat off their Russian allies? I mean, it wouldn't even need to be made up
in that case, I'm sure that if you looked, you could find plenty of sketchy
things Chinese companies are already doing.

But that was the weird thing about this supermicro thing... it was really
pretty easily disprovable. Like... there are a lot of really smart people in
the field looking for this sort of thing. If it was a lie to begin with, why
pick one that can be shown to be false, when there are so many other
possibilities that can't be proved either way?

Man, I hope I live long enough to read good history books about this era.

[1]I bought a bunch of SMCI when the story first broke; Aside from buying and
using a lot of SuperMicro I simply didn't believe that they would be the
_only_ company effected if the story was true. When the story broke, I though
that someone had probably found something, and that as people tore apart
hardware looking, we would find something from other manufacturers, too. As
nobody has found anything yet? I now think the story is just false. I know
smart people are looking. (software/firmware compromises may not be as
durable, but they are a lot easier to implement.) Either way, my SMCI holdings
are up a few grand at a time when the rest of my portfolio is looking pretty
sad.

~~~
T-A
> the renewed focus just seems a little suspicious to me at a time when our
> government is trying to distract us from possible Russian interference in
> our election

That might be a reasonable suspicion if Bloomberg were a pro-Trump
publication. It is anything but. Its editorials and opinion pages bash the
current administration almost daily, and there have been rumors recently that
Michael Bloomberg is preparing to sell his stake in order to run for president
in 2020 (after deciding not to run in 2016 because that could have split the
opposition to Trump).

~~~
lsc
I think if you dangle government sources, even "anonymous" ones, in front of a
reporter like Jordan Robertson, you are gonna get a story, regardless of the
editorial slant of the paper.

Of course, my crazy conspiracy story doesn't explain Robertson's industry
sources. It could be, as others have said, a misunderstanding (I mean, we all
know IPMI is run through with exploits; no conspiracy there. You just don't
put your IPMI on an untrusted network. ) and it doesn't explain his continuing
to stand with the story even after the giant and poorly controlled security
community has been set to tearing apart motherboards.

Really, that last bit points to... if not incompetence, at least to him not
understanding how this sort of thing works.

------
cm2187
Their stock price increased 30% from the drop after the publication of the
article, but still 20% down from before the article. Though most of that is
probably tech stocks tanking. So they are probably close to where they would
have been without the article.

~~~
asdff
If you were a bad-faith source you would have anticipated this and profited
from the movement in both directions. Short when the article is released, then
take your profit and buy at an artificially low price.

------
bhhaskin
But did they find malicious chips elsewhere? And what is their definition of
motherboard, chips and malicious?

All of these statements seem to be pretty well crafted. It is entirely
possible that they found components that don't belong, but don't consider them
malicious without the underlying payload that gets uploaded.

------
devy
Who's this 3rd party Nardello & Co.? A google search leads to a New York based
law firm[1]? If so, what's their technical capability to conduct this
technical assessment?

[1]: [https://www.nardelloandco.com/en/](https://www.nardelloandco.com/en/)

------
lawnchair_larry
For what it's worth, no security experts believe the Super Micro story.

The reasons have nothing to do with whether or not it would be technically
possible. That is irrelevant. The point is that the alleged events did not
happen.

------
paraditedc
Bloomberg got the ads revenue, and people who read Bloomberg won't stop doing
that just because of this.

The only thing changed is the lower cosine distance between China and hacking
in the English corpus, as well as people's minds.

~~~
bunnycorn
It's much more than that.

They have destructed 100's of billions of dollars of capital.

~~~
user5994461
supermicro was not worth that much.

~~~
bunnycorn
They have dragged Apple into it, and with Apple, the entire NASDAQ.

So, yes, they have destroyed 100s of billions.

------
mtnGoat
lets just say what bloomberg claims, was happening, really happened. this
audit does nothing to disprove that. considering how many they sell and how
many their mentioned clients have. picking a few at random to check is junk
science. considering the adversary only needs one to be operating inside the
datacenter.

~~~
lysp
That's exactly what I was thinking too.

Assuming it did happen, it would have focused on a small batch of servers to a
small batch of clients.

They wouldn't have been installing this hack on every single server they
produced.

So unless they pull every server for every major client and check, simply
checking their warehouse stock proves nothing.

------
pasbesoin
You know what really kills a relationship? Distrust.

Something our intelligence community has been fostering for... Well, what
timeframe do I specify? But the past 15 - 20 years have brought something of a
nadir.

And the corporate world isn't faring much if any better.

Reassurances don't seem very reassuring, these days...

------
stdbrouw
> Let's say Super Micro is right

Yet we still don't know whether this is the case, so it's not even clear
whether "consequences" are in order and whether incompetence is at play.

You can't prosecute someone because "if they had stolen money from me, that
would be theft."

------
nyc_pizzadev
Seems like a lot of people don't know who to believe. Me too. Hasn't anyone
pulled a few Supermicro boards and confirmed that this chip exists? It was
reported that the chip might be inside the wafers, but still, that would make
finding this a clear smoking gun.

------
perseusprime11
I read this as "We've conducted a thorough review and made sure we've covered
all tracks"

------
balthasar
We have investigated ourselves and found nothing wrong.

~~~
catacombs
YUP.

Unless SuperMicro hires an independent, third-party auditor, the ball is still
in their court.

~~~
esmi
First sentance in the linked article.

"Computer hardware maker Super Micro Computer Inc told customers on Tuesday
that an outside investigations firm had found no evidence of any malicious
hardware in its current or older-model motherboards."

Further down

"A person familiar with the analysis told Reuters it had been conducted by
global firm Nardello & Co and that customers could ask for more detail on that
company’s findings."

~~~
justtopost
Does that firm have any technical acumen? Or just a supermicro contract for a
result?

------
kushti
These days most of American media stories about China, Russia etc are probable
fakes. Only mass boycott of Western MSMs can fix the situation with these
terrible weapons of psyops and propaganda.

------
jorblumesea
Is anyone at all surprised that a company investigated itself and found
nothing wrong? It might be true, it might be false, but there's an obvious
conflict of interest here.

~~~
kickopotomus
They did not investigate themselves. The audit was performed by Nardella & Co
as stated in the article.

------
time-domain0
What are the odds that Bloomberg ran this story under pressure/collusion from
Supermicro competitor(s) in Taiwan, US fed govt or other business actors
trying to tarnish China's image? That they would knowingly run such a big
story without commensurate easily verified evidence and reliable sources is
irrationally foolish for such a large news shop.

------
NedIsakoff
"Fake but accurate"?

------
cfv
Are all this people going to bill Bloomberg for the costs of the tests? They
prob should

------
fallingfrog
Not sure I believe this. Would anything bad happen to them if they just lied
about it? Probably not. Unless they stepped on some politician's foot somehow.

~~~
danso
Who is “they”?

------
creeble
Bloomberg reporters clearly need retraining. As I've stated a few times
regarding this story, "photo or it didn't happen". It's a pretty simple
standard, maybe Bloomberg will think about applying it now.

Or maybe someone else will come up with a new conspiracy to punk them.

~~~
catacombs
"Photos or it didn't happen" isn't as easy to obtain as one might think. Best
case is obtaining documents that corroborate this whole thing.

------
cannedslime
So let me get this straight... Super Micro hired a firm, who then asked the
manufacturers, "Hey guys can we please have your design files for your PCBs"
and then they looked at the parts list and said, "Nope, no dedicated backdoor
chips here" ?

They didn't really do a thorough examination of the devices in question, but
asked the manufacturer for the CAD files?

~~~
hau
You should really do thorough examination of the article it's really short.
They did examine hardware both in production and already sold to Apple and
Amazon samples.

~~~
cannedslime
Well what I was wondering was, what did they examine to be exact? Did they
decap chips and reverse engineer them?

The article as you say, is really short. Too short, there isn't really any
information in it, besides "Trend micro said this, take their word for it"...

~~~
dracodoc
Why are you keep using "Trend micro"? Did you realize that's a totally
different company?

~~~
cannedslime
Oh, my bad... Doesn't really change much though.

------
ryanmercer
People worry about malicious chips in instances like this and I mean it's a
valid concern but one method of attack I've always thought would be effective
and extremely dangerous is strategic placing a component in a circuit that, if
it fails, disables the entire device then you simply need a way to activate
it.

Radio.

Design a circuit, or better something that appears to be a capacitor and
functions as a capacitor but has a small internal compartment at say the top
so that it performs at less than what it is rated for but has a small circuit
that over-volts via a joule thief and causes a failure. Have the trigger be a
small receiver that activates at a certain frequency probably in the ELF or
SLF range with just a few bits needed as the activation key.

Put that into the supply chain of whatever industry and when you want to
disrupt to cause economic damage, or even as part of causing a bit of chaos
preceding a military attack, fire up your ELF station and start pumping out
the few bits of data to activate.

ELF will penetrate hundreds of meters of water, it should reach inside most
buildings and even if you only had something like a 5% success rate you'd
disable a LOT of whatever you'd installed them in. If it's networking
hardware, you could likely cripple anything that relies on the internet by
causing considerable distributed failures.

~~~
moftz
Only problem with LF stuff is that you need a pretty big antenna. You would be
better off designing a chip with an RF section that operates in the GHz range
so you can get away with a tiny microstrip antenna or with the antenna inside
the chip.

~~~
ryanmercer
I will admit radio is not my thing, that works too though and then you can
make it a much larger activation key, even just bytes instead of bits would
exponentially increase odds of not having an accidental activation which
_should_ make it not get set off by a router or microwave and you could
probably activate it simply by flying over with a slightly modified cargo or
passenger aircraft that looks business as normal but just belches high-power
bursts of the activation signal.

