
In Which I Savagely Impugn the Honor of IPMI and its Friends - dfc
http://fish2.com/ipmi/
======
thrownaway2424
Pfft, this guy only mentioned the obvious horrible security implications.
Aside from that glaring flaw, there is also the small matter that IPMI
software is all implemented by raging idiots who got fired from BIOS vendors,
and then got fired again from the people who write mobile radio firmware, and
then got fired again from the company that writes the software in your
electric toothbrush. They never work in actual outages, cause more host
outages than they solve, and are in no way a substitute for having humans on
site. I had a rack full of garbage from Penguin Computing that would hang on
reboot with the message that the BMC was not responding. Thanks, that's
helpful. And dozens of horror stories from every vendor ranging from
SuperMicro to HP and IBM.

I encourage you to guess whether Facebook, Google, and Amazon use IMPI to
manage anything. Have a look at the OpenCompute designs for a hint.

~~~
subway
[http://opencompute.org/wp/wp-
content/uploads/2012/05/OpenCom...](http://opencompute.org/wp/wp-
content/uploads/2012/05/OpenComputeProjectOpenHardwareMachinesManagement.pdf)

These designs?

~~~
thrownaway2424
Doh. Just goes to show, you can open up your process, but you can't make
people have judgement and taste.

~~~
dfc
And/or that you should look before you speak.

------
contingencies
Late last year I was building infrastructure with some higher security
requirements and was quite shocked at the IPMI situation - concerned enough to
email Redhat (who obviously support lots and lots of clients on hardware from
lots and lots of vendors) via a personal contact in a global security team
there, with my concerns and some evidence related to our particular hardware
vendor.

After some time to ponder, they got back to me with "your concerns look
legitimate". Discussions followed, but I am not aware of any specific
advisories resulting from this.

I also understand that recently there was an IPMI-related presentation at
Ruxcon Breakpoint in Melbourne, Australia:
<http://www.ruxconbreakpoint.com/speakers/#Igor%20Skochinsky> ... no idea of
the content, though.

Honestly, I think the first thing is for vendors to be a lot more honest: if
someone has root on your host, they can most likely trivially obtain root on
and backdoor the IPMI controller. Once they get root on the IPMI controller,
they can do anything they want on the IPMI LAN. This reality differs greatly
from the 'dedicated' and 'separate' words used in vendor marketing literature
and documentation, and is no doubt a _direct contributor_ to insufficiently
paranoid systems architecture, resulting in real world vulnerabilities in some
pretty important systems.

Why do I say pretty important? Well, outside of remote KVM, IPMI is frequently
used for node fencing in high availability (HA) cluster scenarios, which means
that this technology is likely to co-occur with some fairly paranoid /
24x7x365 systems (air traffic control, stock markets, etc.). Attackers with a
foot-hold can thus use IPMI-based vulnerabilities to (permanently) compromise
such higher-end systems.

What are we as implementers supposed to do? Here are some ideas. (1) Run a
relatively paranoid, receive-only, anomaly-based NIDS on your IPMI segment to
encourage 'after the fact' detection of bad behaviour. (2) Run IPMI on a
dedicated VLAN. (3) Run your IPMI VLAN on dedicated hardware (4) Program your
network infrastructure to ensure that regular IPMI nodes on your average
server cannot communicate with any other peers on the link layer (ie. other
servers) _except_ for the expected source of management operations (5) Make
sure the expected source of management operations isn't running an IPMI
controller (or at least, one from the same vendor on the same subnet) (6)
Don't use IPMI at all

Honestly, IPMI can be very useful in some cases, though it's probably a
massively unspoken vulnerability in some high end systems. As always, don't
trust any single component... security only comes in partly-effective layers
...

------
modeless
Are these IPMI things ever installed on desktops and laptops, or are they
strictly for server hardware?

~~~
vault_
You may be able to find it on high-end workstations sold by server companies
(read, a 4U server turned on its side, with added feet), but by and large it's
a server-only thing. The reason being it's only really necessary to have the
level of remote access offered by these systems if it's difficult to access
the hardware, or you have a very large amount of the hardware (in most cases
where it's used, both of these are true).

~~~
SEJeff
You can also buy discrete bmc pci cards from a few different vendors. I
believe super micro sells them, but I've always loathed any environment that
uses super micro servers as they tend to be low to mediocre quality compared
to the "big 3" tier 1 server vendors.

------
Adaptive
Anecdotal, but the IPMI software I have to use regularly for access to a
specific server is, without doubt, the worst server utility I've ever used.

I love having the BMC there, but you are really at the mercy of vendors to
update their IPMI/BMC firmware. Guess how often that happens.

------
patrickgzill
IPMI is so bad, I don't even use it for my servers when it is available!
Better to just wait and call someone at the datacenter who will take care of
reboots properly.

