
Tesla PowerWall 2 Hack - amai
https://github.com/hackerschoice/thc-tesla-powerwall2-hack
======
nemosaltat
Can’t believe Tesla would ship something with anything resembling a default
password. At first glance, I assumed this would be a clear violation of the
requirements of CA SB-327 (goes into effect Jan 1).

Reread the bill, and it actually says: “The preprogrammed password is unique
to each device manufactured.” If the default is based on the serial number, I
guess it’s “unique” under the letter, but certainly not the spirit.

Link to bill:
[https://leginfo.legislature.ca.gov/faces/billTextClient.xhtm...](https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=201720180SB327)

~~~
sorenjan
I guess you haven't seen this then.

[https://www.reddit.com/r/EnoughMuskSpam/comments/99sbwa/form...](https://www.reddit.com/r/EnoughMuskSpam/comments/99sbwa/former_tesla_programmers_anecdotes_about_problems/)

~~~
ncmncm
This is amazing. It is all completely plausible, and clearly includes only the
most easily explained horrors, with anything that would require explanation
just omitted.

It is worse than I would have been able to invent.

~~~
marvin
IMHO, these stories aren’t unexpected from a tech company that grows 80% year
over year, developing and expanding as fast as technology and market forces
allow. Yes, there are mistakes and unfortunate incidents in the organization
of people and tech priorities here, but this is inevitable in an engineering
org that moves this fast. You really need to measure this up against Tesla’s
achievements:

Growing high double digits each year for more than a decade in a brutally
competitve space, developed 4 hugely successful vehicles, first US auto
company to succeed (knock on wood) or grow to significant size (>700,000 cars
sold) since Ford, production constrained continuously, did this with almost
zero paid marketing, built global networks of dealers, service and chargers,
improved the state of battery tech by a multiple and doubled global li-ion
battery pack output, built 3 car factories, made electric cars economically
viable, _incidentally_ seriously wounded the car dealership model, succeeded
with qualitatively different software approach with OTA updates and tight
software integration, designed cars with hitherto unparalleled performance in
their class, and others that I couldn’t come up with on the spot.

This is only controversial because this is safety-critical tech and it’s
uncomfortable to see the development pains of consumer grade software affect
it. There’s no evidence that the approach have led to more injuries or deaths
than cars do in general, quite the opposite. Tesla cars are safer than
competition, the argument can be made that fast deployment is a net _win_
regarding safety. Even in the absence of any other things they have achieved.

I believe there’s also an element of envy, or misunderstood beliefs that the
best tech approach is taking things slowly. Yes, in isolation. But then market
forces will crush you because someone else was faster, or because you missed
the window of opportunity for making things so dramatically better that you
manage to unbalance the local maximum of the status quo.

[Edit: To everyone who downvotes this: For the sake of enlightening
discussion, could you please express your disagreement in words rather than
the downvote button? Note that I'm talking about GP's linked commentary on
Tesla's engineering mishaps in general, not this specific vulnerability].

~~~
aguyfromnb
> _For the sake of enlightening discussion, could you please express your
> disagreement in words rather than the downvote button?_

I'll give it a try: maybe it's because someone made an off-hand comment about
Tesla shipping a default password, and you chimed in with an apologetic post
that reads a lot like astro-turfing?

> _did this with almost zero paid marketing_

 _Traditional_ marketing. Tesla spends millions on marketing annually, it's
right there in their financials. Just recently a pile of pro-Tesla Twitter bot
accounts were banned. Did you know that Tesla is paying people to post
messages on social media?

~~~
marvin
I am not affiliated with Tesla or paid by them in any way, if you're implying
that. Was also not making apologies for shipping devices with guessable
passwords, that's obviously a serious screwup that needs to be fixed - ideally
at a level of changing the engineering culture if that's what it takes. But
the rest of this thread treats that subject in depth, and I wouldn't
contribute anything material to that discussion beyond piling on, which is a
bad use of time and brain power. Thought I made it quite clear in my comment
that I was specifically adressing the discussion of sub-par software
engineering practices, and how this is a very common consequence of extreme
growth.

I know Tesla has a marketing budget and marketing people, that's obvious
enough, so traditional marketing was indeed what I meant. Maybe I could have
made that more clear.

Wasn't aware that they're paying people to astroturf though; first I hear of
this. Do you have proof of it?

~~~
xedeon
> Wasn't aware that they're paying people to astroturf though; first I hear of
> this. Do you have proof of it?

They are not and he can't/will not be able to provide proof because it's a
false claim.

------
unnouinceput
"The PW can store up to 13.5kW of electric power..." No, you store energy, not
electric power, and energy is Wh, that's why you have your phone battery with
5V and 8000 mAh as units and not 8A.

"How does the grid feel about me oscilating this and who will die first, the
PW or the sub-station?" Definitely the PW. Sub-stations are made to withstand
thousand of households switching all at once their light-bulbs at evening and
during storms nature electric discharge (ie, lightning strikes). Your measly
PW is fully ignored.

"Do not try any of these Grid Codes (really, please do not):

AU ASS4777.2 DE VDE4105 UK GA59 21A, GA83 16A IT CEI-021 NZ NZS47772 US:
IEEE1547 Split Phase 240V 60Hz " I'm a bad boy, I tried them. Nothing
happened.

~~~
nexuist
If you were to try this and somehow cause damage to the sub station, would you
be held liable?

Actually, this brings an interesting question with IoT - if devices damage or
otherwise exploit your utility company’s equipment, is it their fault of
yours? Presumably doing anything bad is against your utility’s ToS, but if you
didn’t do it knowingly...

~~~
mbreese
I would imagine that unless you were a licensed electrician, you would be
liable.

~~~
KMnO4
I’m sure you’d still be liable if you were a licensed electrician.

------
londons_explore
The only bug here is the default password.

After authentication, the fact you can make it charge or dump power into the
grid is by design. If I wanted the grid to suffer, I can do this by plugging
in and unplugging a multi-kilowatt heater every few milliseconds too.
Residential properties have a fuse (usually 60-100Amps), and anything you can
do without blowing that fuse won't damage the grid.

~~~
jacquesm
> and anything you can do without blowing that fuse won't damage the grid

That's what you'd think. But I have actually done damage to the grid and had
that fuse still in one piece. I accidentally connected the -HV line of a neon
light installation to ground. After I could see again I realized the power had
gone out. I called my boss (landline still worked) who lived more than a
kilometer away from the workshop and there too the power had gone out. It
turned out my little accident had put a good chunk of the eastern part of
Amsterdam out of commission and the fuse was just fine.

Fuses react to current, not to power and when you dump a high enough voltage
back into the grid your fuse might not blow before you cause something further
upstream to trigger to protect the rest of what's connected.

~~~
madaxe_again
This is exactly what RCDs are for - with a huge earth leak like that (and
therefore L/N imbalance) it would have tripped within milliseconds, before
cooking the grid.

I am surprised by how infrequently they seem to be used outside of the U.K. -
I’ve just rewired my house in Portugal as it was completely unearthed, and
just had fuses, no RCDs.

~~~
jacquesm
I'm not sure how common these are in an industrial setting nowadays, but back
then _all_ big electrical tooling would leak at least a little bit and if you
hooked them up through ground fault interrupters those would trip all the
time.

A typical lathe, mill, plasmacutter or welder (which that particular workshop
was full of) would be in the 10KW range with peaks far in excess of that
during start-up.

I don't remember if we had the office area (where I was working) on protected
circuitry or not, that would have been an interesting bit of data. The
building still exists, I'm almost tempted to go and have a look.

That HV supply was a pretty beefy one by the way, the crocodile clamp that
made 'first contact' was gone entirely.

------
vermilingua
_This_ is the mythical power-grid attack that people have been talking about
since the concept of cyber-warfare was first dreamt up.

It’s lucky we caught this now, before there are enough PowerWalls to seriously
destabilise the grid if this attack were to occur.

~~~
londons_explore
Rate of Change of Frequency protection (ROCOF) in embedded storage and
generation systems is what will kill the grid in a massive cascading failure.

ROCOF works well to keep things safe when only a small percentage of the grid
demand is met by residential solar/powerwalls.

As soon as any significant proportion is residential solar (and thats already
the case in some countries at some times of day) it acts as a cascading
failure mechanism. As soon as any failure occurs, embedded generation sees a
rapidly decreasing frequency, and rather that increase supply as traditional
generators would be instructed to do to stabilise the grid, ROCOF protection
requires they cease supply, making the issue far worse.

Within a fraction of a second, all embedded generation will disconnect, likely
causing a near total blackout nationwide. Since system frequency that ROCOF
measures is nationwide, failures won't be local to one geographic area.

I suspect these rules were made when people thought "consumers feeding energy
back into the grid will never be more than 0.1% of the total - we'll always
have enough spinning reserve to make up for that". Thats no longer the case,
and unless the ROCOF limits are changed, and _the majority of home solar
/wind/powerwalls get a firmware update_, expect a few very large blackouts.

~~~
londons_explore
I believe the solution to this problem is to ban ROCOF protection, and the
related phase shift protection, and instead instruct a few big energy
producers to transmit a gold code on top of the 50 Hz AC, bandlimited to
48-52Hz and power limited to 0.01% of the system power. Transmitting that code
would be easy (cheap) for anyone who does DC/AC conversion with solid state
electronics, so that's normally solar, wind farms, and long distance undersea
transmission lines.

That gold code could be received and decoded anywhere on the network. If power
islanding occurs, embedded generation will detect the loss of the gold code
(since they are no longer connected to the generator injecting the code), and
cease supply.

The only disadvantage is it introduces a security vulnerability by design:
Anyone could transmit the gold code from their house, effectively disabling
islanding protection in their neighbourhood. _If_ power islanding were to
occur, and _if_ there was sufficient embedded generation to keep a stable
power island, grid hardware could be destroyed through overvoltage,
overheating, and circuits closing without frequency synchronisation. I think
it's a worthwhile tradeoff though - damage will be localized and minimal, and
a very unusual set of circumstances have to happen outside the attackers
control for the attack to do damage.

~~~
amluto
Would a better long term fix be to upgrade grid hardware such that it can
survive an inadvertent largish island?

Sometimes I think that a DC grid would be better. Issues like frequency
synchronization wouldn’t exist.

~~~
Denvercoder9
A DC grid is likely better nowadays, but we're stuck with the historic AC
grid. Most of the problems DC faced back in the war of currents have been
solved by solid state technology.

------
JaRail
This is an amazing lack of security best practices. To me, this screams
outsourced. Given how many people hate Tesla, they need to be taking this
seriously. This truly blows me away. This is "people should be fired" levels
of organizational incompetence. There's no way some of these issues haven't
already been noticed and put in the issue tracker. They're just not taking it
seriously. It reminds me of Boeing to be perfectly honest.

~~~
dogecoinbase
It does not strike me as outsourced, given what we know about software
engineering practices at Tesla:
[https://twitter.com/atomicthumbs/status/1032939617404645376](https://twitter.com/atomicthumbs/status/1032939617404645376)

~~~
xedeon
Yeah, some random Twitter account that was never substantiated. Fact: He never
provided proof he is who he said he was.

------
mdorazio
Did they even try to submit these issues to Tesla? They have a bug bounty
program and have been reasonably good about patching issues in vehicle
software. If not, this is pretty irresponsible disclosure.

~~~
FireBeyond
"Responsible disclosure" is an invention of vendors who want you conforming to
their policies and timelines (and more).

Tesla is also "good" at disabling aspects of people's property (like ethernet
ports, or ability to receive future firmware updates) when they dislike what
people find "wrong" or otherwise in Tesla software.

~~~
kerng
I think this comment highlights a lack of understanding what responsible
disclosure is about. It's there to reach the best possible tradeoffs to
protect consumers and force a quick turnaround with fixes. Just publishing
vulns, which the vendor might not even see or learn about(!), will not help in
getting things improved and puts consumers knowingly at risk at scale.

~~~
FireBeyond
I understand perfectly well what it is. But see my sibling comments, that
reflect an agreement with the concept of "coordinated disclosure", rather than
"responsible disclosure", which gives an implication that I am being
irresponsible if I do not work to the vendor's needs and priorities.

~~~
kerng
Publishing vulnerability details without informing the vendor is
irresponsible.

------
rsync
Is there an alternative to the powerwall that is _functionally_ similar, in
terms of power density and utility, but doesn't have things like a network
stack and other wifi / IoT / "smart" features ?

We are just about to install lithium battery backups in our home and, of
course, the Tesla powerwalls are an obvious choice, but I don't look forward
to having to disable/reenable the network connectivity on them, manually, in
some cat-and-mouse with Tesla for updates or whatever...

In fact, I'll bet that most (all ?) of the updates that Tesla has to send the
powerwalls are for the update system, and accompanying network stack, itself
...

~~~
peteradio
What is the reason you can't store excess on the grid?

~~~
marvin
Punitive pricing, or at best you're at the mercy of the grid operators and
power companies.

------
joeraut
I'm amazed Tesla went ahead and used easily guessable and unchangeable
passwords. It's just such a trivial and obvious issue that it seems strange
Tesla engineers let this happen.

I wonder if this was an off-the-shelf solution they're using here, or was
developed by external contractors, for lack of a better explanation.

------
xoa
I have a couple of PW2s installed, connected via ethernet only (isolated on
its own VLAN, though Tesla is total garbage about basics like "what firewall
rules are needed"), no cellular here either. But the TEG-$(SERIAL){3} network
has always been right there anyway, which is just really lazy design. I happen
to be physically far enough away from anyone else that it's very unlikely to
be a security issue in practice, but it's still unnecessarily polluting WiFi
even beyond that. This has been noticeable for a long time too, it's not hard
to find threads going back a long ways with people asking how to disable the
gateway WiFi, ie., ( [https://teslamotorsclub.com/tmc/threads/disable-gateway-
wifi...](https://teslamotorsclub.com/tmc/threads/disable-gateway-wifi.157212/)
), or people speculating about the obvious vuln/interference implications.
From that thread back in June:

> _It would be nice to find a way to turn off the TBG 's WiFi hotspot. I
> already have too many WiFi hotspots in my area for my taste. TBG's
> broadcasting WiFi is just a exploit waiting to happen._

The whole thing is genuinely perplexing. Dependence on WiFi, while
regrettable, I guess can sometimes make sense from a "user friendly"
perspective for a lot of typical consumer installed gear. But the PowerWalls
are absolutely not consumer installable, nor obviously in any way inherently
wireless. They represent serious electrical infrastructure, and require
professional installation with a lot of run cable. Adding in some shielded cat
5 or whatever along with that is frankly trivial for multi-thousand/ten-
thousand dollar professional projects, even when not dealing with people who
can do a drop themselves.

Minimizing attack area is really trivial security, and here it's got other
bonuses like just being more reliable. The entire IOT space is full of shit of
course, but it seems much stranger in this instance to me than something like
lightbulbs. And why even have passwords at all for access versus using keys?
None of this is supposed to be generally accessible anyway. It's not like this
would cost Tesla anything extra.

Edit to add: HOSTNAME INCLUDES THE FULL SERIAL. I thought I'd take a second
look at this and just pulled up the client info for the PW2 Gateway on my
network, and hostname is 11XXXXX-00-J--S$(SERIAL). So no local physical access
is required, the gateway itself just broadcasts the whole serial, which in
light of this is an, interesting, decision. I can confirm that using the
hostname with an added S at the front (so on mine serial was T[...], I used
the password ST[...]) I was able to connect to the WiFi spot, and in turn to
the management page described. Incredible.

Incidentally DPI is also kind of wacky now that I look, just running a simple
Suricata setup but connections are all over the place (why is YouTube showing
up?). About 4.5 GB down and 9 GB up, down I assume would be updates of some
kind, up monitoring data though that seems like a significant amount for what
should generally be text. I haven't had it that long, must be kind of chatty.

~~~
astronautjones
>Edit to add: HOSTNAME INCLUDES THE FULL SERIAL. I thought I'd take a second
look at this and just pulled up the client info for the PW2 Gateway on my
network, and hostname is 11XXXXX-00-J--S$(SERIAL). So no local physical access
it required, the gateway itself just broadcasts the whole serial, which in
light of this is an, interesting, decision. I can confirm that using the
hostname with an added S at the front (so on mine serial was T[...], I used
the password ST[...]) I was able to connect to the WiFi spot, and in turn to
the management page described. Incredible.

this is staggeringly irresponsible design! thanks for the confirmation

~~~
xoa
I guess in practice it doesn't really matter much anyway because while my
serial doesn't specifically follow the format described in the link it'd still
be utterly trivial to brute force. So that they leak the whole thing onto LAN
anyway just turns it into even more of a joke, but I guess would rarely make a
real difference. Some of the HN crowd have security monitoring on their own
networks that would detect suspicious hammering patterns, have more device
isolation, etc, but I strongly suspect the vast, vast majority of PW installs
are pure ISP default setting CPE environments. Using crummy default password
routers full of their own holes too!

------
morpheuskafka
The password configs are clearly documented:
[https://www.tesla.com/support/energy/powerwall/own/monitorin...](https://www.tesla.com/support/energy/powerwall/own/monitoring-
from-home-network)

~~~
713233eb
> Step 3: Select your home Wi-Fi network and enter your Password.

So an attacker can even get LAN access from across the street.

------
lgats
Does anyone have the FCC ID information for a Tesla PowerWall?

------
whattheactualf
This is a far cry from a 'hack'. Oof.

------
marvin
The criticism of this security vulnerability has been well covered at this
point, so I'd just like to point out that this should be trivially solvable in
a few hours by changing all passwords from HQ.

It'd be quite irresponsible of this researcher if they didn't give notice
beforehand (don't know if they have).

------
propercoil
So many people like to nitpick when it comes to Tesla, It reminds me of the
Apple critics in the early days of the iphone. They assume Tesla should have
the highest standard and be absolutely impeccable with all their products.

Just don't buy it if you don't like it. Let the rest of us enjoy a sustainable
future with insanely safe full self-driving electric cars.

~~~
astronautjones
this is an exploit that allows an attacker to disrupt the electrical grid -
even if it's only to the substation, that effects all of your neighbors that
didn't encourage you to blindly support Tesla's endeavour. it's highly
irresponsible to have this kind of control behind such a trivial login on
something that connects to your power lines, something extremely dangerous.
far from a "nitpick"

~~~
generatorguy
It doesn’t disrupt the grid anymore than turning a heater on and off.

~~~
sangnoir
Can your heater push power _into_ the grid?

~~~
generatorguy
An increase in embedded generation is the same as a decrease in load to the
system. So turning a heater off is akin to “pushing power in to the system”,
unless the embedded generation is larger than the load on a feeder, in which
case the direction of power flow would be reversed, which may or may not cause
problems depending on the protection settings at the substation.

The peak system load and generation on the Western North American grid is
167,000 MW. There is so much inertia there a 10kW power wall isn’t going to do
anything.

