Hacker News new | past | comments | ask | show | jobs | submit login
Another 0-day looms for many Western Digital users (krebsonsecurity.com)
400 points by danso on July 2, 2021 | hide | past | favorite | 226 comments



"they discovered a chain of weaknesses that allows an attacker to remotely update a vulnerable device’s firmware with a malicious backdoor"

Once again, this is why firmware needs a hardware write-enable switch, not a software one.

Cue the arguments that remote updating is needed to fix bugs that allow remote updating. :-/


I want hardware switches for:

1. firmware updating

2. write-enable for disk contents

3. turning the microphone on

4. turning the camera on

In a surprise development, the webcam I just bought comes with a flip-up lens cap. Yay! It's Nexigo, they deserve a shout-out for this. But in the Dept of Half-Assed Features, the lens cap does not disable the microphone, so I still have to unplug it when not in use.


I recently bought USB off/on switches [1] for the external webcam and microphone on my desktop. I think they control the power line and not the data lines, but they do the trick. Reduces port fatigue and USB orientation frustration.

[1] https://www.amazon.com/gp/aw/d/B08M44D79T


funny story, i just spent a day troubleshooting why a microcontroller would not reset after cutting its power lines

turns out it was leeching power from another still-active device through its data pins!

there was not enough power flowing through this way to actually do something, but there was enough to keep the brownout detector from kicking in and resetting the chip


To be fair, the 'correct' way to do this is to use a double-pole switch that actively pulls (whole-device) VCC to ground when off, but that has it's own problems, especially if any of your sensors are capable of generating electricity on their own (piezoelectric microphone, radio reciever, alleged photo'transistor's that can operate photovoltaicly, etc).


The amount of energy generated by the microphone is gonna be crazy small


I mirrored the pinout of an AVR once and spent a few hours debugging why ISP wasn't working (so I unsocketed it for programming) and all the pins were wrong. It worked just fine pulling ground and Vcc from the I/O pins just opposite. These are of course fairly low-power 1.8-5 V devices, so when run on 5 V there is a huge margin for the supply voltage.


You were powering it through the protection diodes.

Some 10-15 years ago someone built dirt simple radio tags this way. Just a microcontroller, with a capacitor and an antenna trace connected to some io pin. I loved that hack.



Incredible hackery.


I bought a super cheapo USB hub with four ports and on-off switches. I didn't think I'd have any use for them, but it turns out they're extremely useful because it turns out I need to unplug/replug things on my desk much more often than I thought.

In related news, I used an old Android phone and DroidCam over USB as a webcam. The picture quality is stellar, much better than any webcam you might find, and it's very simple to stop it by unloading the driver (I know, I know...).


I didn't know those existed. Nice! But it does have a serious flaw - no indication which switch position is "On" or "Off". C'mon, makers!


True, but that flaw can be addressed with a label maker.


Trust me, I have a label maker and use it :-)

It's really, really helpful to figure out which wall wart goes with which device.

Another tip I learned from another. You know those green plastic tabs that keep a bread bag closed? They clip onto a cable nicely, and write on them with a sharpie which device the other end is attached to. That really helps with the rat's nest of wires under my desk. One of them says "cam" on it :-)


My boss took my label maker away. Apparently, I am "not responsible".

Also, with respect to cables, this is really why we need tri-colored braided cables from a reputable dealer (ANKER!?): white, black, gray, blue — that gives 64 possible combinations!


> My boss took my label maker away. Apparently, I am "not responsible".

I'd consider that an honor.


Also cellophane tape with a chunk of index card inside. (More legible on account of high contrast.)


Now I really want a USB blade switch.


> I want hardware switches for

Here you go: https://puri.sm/security/.


Also PinePhone.


pinephone's hardware switches are kind of an afterthought and not particularly accessible, being behind the battery cover in the form of a tiny block of DIP switches better suited to one-time configuration.


Exactly. With Pinephone, you will not be able to switch on your microphone while receiving a phone call, unlike with Librem 5.


Bug or feature is that?


Depends on whether you need a microphone during a phone call.


It's usually the wife calling, and during those calls she's not listening to a word I say anyway.


>3. turning the microphone on

instead we have undocumented microphones for 'future purposes'. Thanks Google


Any chance of a citation on this? I’m interested in reading more.



A few months later they turned it into a feature and added software support to the hidden microphone


My new Lenovo has a built in camera cover which was a nice surprise, but only a software hotkey for the mic.


My workstation came equipped with a removable piece of masking tape I found in the supply closet.


Microphones are ... tricky. I remember seeing a proof of concept of using laptop speakers as a microphone. And more recently, I read about using the mouse to "listen" in on the environment. Apparently the sensors in mice are sensitive enough to detect a lot of vibration. Not good enough to listen in on a conversation, but give it time ...


My Streamdeck is great for turning microphone off at a system level. Coupled with eg OBS Studio and using virtual camera as source, where you can have several scenes available for quick switching in a pinch, you get a much better control for meetings.

I highly recommend them for remote working software engineers; the macros are amazing (eg start camera, lights, open meeting software in one go - then another to shut it all down).

You will still need to double check that all the mappings haven't changed for whatever reason, from time to time. (I'm on Win20/WSL2).


Ah, the flip-up for the camera. Still records your voice and keystrokes, proof-of-concepts have demonstrated it is possible to capture passwords using sounds. Also the driver can contain malware, and anyway the camera itself can be on battery even if unplugged and use 5G now that the billing is per-data and not per-sim, or Sigfox networks, all of this for less than 10€ of component out of a 80€ webcam. I really don’t understand why switches are not proposed by the largest vendors.


>I really don’t understand why switches are not proposed by the largest vendors.

Because paying customers don't understand or care about privacy.

Facebook is still a thing. Let that sink in.


2. write-enable for disk contents

It's funny how such basic things from the past were thrown away. Every floppy disk ever had this.

However, i also believe that if such a thing existed for modern gear, it would only be used by 1% of people, and even then, mostly accidentally, resulting in millions of trouble tickets. So I'm not sure what the compromise is.


I don't buy the argument that if not everyone uses it, nobody should get it.

BTW, I would read TV repair manuals as a kid (yes, weird). There was always the "check to see if it is plugged in". Plugging TVs in made a lot of money for service people.

I see similar things in car manuals for car won't start. "Put gas in it."

Edit: This was back in the days when you could repair a TV with a soldering iron and a screwdriver. Every hardware store had a tube testing machine. I'd have fun by randomly swapping the tubes that fit in the same socket and seeing what effect that would have on the TV's operation.


And of course, blow the dust out of the connector!

https://devblogs.microsoft.com/oldnewthing/20040303-00/?p=40...

I was also the family "TV tube test person" as a kid. I must have been around 6 or 7.

For the young'uns, TV sets used to have tubes and hand-soldered point-to-point circuitry. Just like an ENIAC, a tube TV would always "go on the fritz" as the tubes burned out.

My dad showed me how to pull out all the tubes, and we would put them in a cigar box and go to the little corner grocery, which had a tube tester in front. I would dial up all the settings for each tube and test it, and we would buy replacements for the bad ones. Take them back home and I would plug them in, and the TV worked again! Dad was always generous and made sure I got credit for it.

BTW did you ever get to discharge the high voltage connection to the picture tube with a screwdriver and wire with alligator clips? One clip to chassis ground, the other to the screwdriver, then slip the screwdriver under the rubber insulated connector, and BANG!

https://en.wikipedia.org/wiki/ENIAC

https://en.wikipedia.org/wiki/Vacuum_tube


I'll never forget the time I was driving to pick up my first new car. I was 3 blocks from the dealer when my old car died. Nothing I did could get it started again. Finally a cop pulled up and asked if I was having trouble. I told him, then he asked if I had gas. Of course since I was anticipating a new car, I hadn't been paying attention to the gas level in the old one. Thankfully I was just across the street from a gas station.


  > I don't buy the argument that if not everyone uses it, nobody should get it.
That's not the argument. The argument is that for every N people who use the feature, X*N ( X>>1 ) will accidentally enable the feature and thus require an expensive tech support call.


SD cards had this, but it's up to the driver to respect that. There is nothing in hardware preventing writes, it's just a signal to software saying "Hey, please don't write to me!"


I don’t remember PATA(IDE) disks having Write Enable jumper settings. Apparently some parallel SCSI drives had them but pretty rare for non-removable media at all.


I do remember them. I'm old.


Don't forget hardware switch for wireless networking. Laptops used to come with that...


The problem with the wifi switches was support. You would not believe how common it is for someone to flip those small switches accidentally and not even know its there. Then the support calls come in for wireless issues.

The answer to "Did you check the wifi switch?" is almost always "What wifi switch?".


And it only got worse when that moved to soft switches (e.g. "press Fn-F6 to toggle wifi"). Typically the on-screen feedback would only work in DOS, or would depend on a weird vendor utility, so there might not be any obvious indication when you switched it off.


> 3. turning the microphone on

Most people are surprised that speakers can be used as microphones by "running them in reverse", and so you also need a hardware switch for your speakers to maintain privacy.


While technically true, in most real life situations, this is not possible to exploit. If the speakers have an amplifier in line with them, they will not work in reverse. If the speakers are built in to a laptop, the driver circuitry will not allow them to work in reverse.

Pretty much the only way this might be possible is if you had an audio port that was capable of functioning as both a TRS output and a TRS input (not a TRRS "headset" port), and had a set of headphones plugged into said port, and had a piece of malicious software that was able to reconfigure the port to act as an input.


>and had a piece of malicious software that was able to reconfigure the port to act as an input.

That's actually a feature of many realtek sound drivers. https://www.reaper-x.com/2012/02/13/how-to-remap-retasking-r...


>Pretty much the only way this might be possible is if you had an audio port that was capable of functioning as both a TRS output and a TRS input

Most embedded PC sound cards made in the last few years have this.

(also, you'll need headphones without an amplifier as well!)


When I was a boy I'd hook up a speaker to a phono input, which made a great PA system! An even longer wire attached to the phono input turned any amplifier into an AM radio.

A simple intercom is just two speakers, one on each end, wired together in a loop.


Nice!

In my childhood I took apart a broken WalkMan and discovered if I connected a random ~8" loudspeaker driver in the tape head's place, I could eavesdrop on my siblings and parents from across the house by placing the speaker against the walls or floor, complete with volume control and everything.

It was incredibly sensitive, and infuriating to learn how much everyone was constantly lying and talking behind eachother's backs at that age.


If you're worried about a secret hardware input attached to the speakers, you might as well be worried about a secret extra microphone. And at that point switches won't help at all.

If there's non-secret hardware inputs on the speakers... it's probably easier to just remove that.


You've misunderstood. There's no extra hardware, secret or non-secret. It's possible to run devices in reverse. Take a computer with separate headphone/microphone jacks (not the combined jack), and plug a speaker in to the microphone jack and scream in to the speaker; the speaker hardware works just fine as a (crappy) microphone. Or try the opposite, plug a microphone in to the speaker jack and turn the volume up, you'll hear sound coming out of the microphone.


If the user plugs their speakers into the microphone jack, that is either a deliberate act or a mistake that will be quickly fixed. It's not a threat to the user.

The threat is if the speaker jack has recording hardware. That's why I said "attached to the speakers".

If you're thinking about adding a switch to disable recording via the speaker jack, for safety purposes, you should probably just remove that capability entirely.


The point is that jacks are software configurable on most computers. So a speaker jack is a setting change away from being a microphone jack.


And I'm saying while that's true on many computers, the reasonable solution to that isn't a switch.


Then what is it? To trust the software? Because then the arguments just gone full circle.


It's to change the hardware. But while you're changing the hardware, you don't need a switch. It never needs to be on! Make it permanently off.


No, everyone understood just fine. The point is that speakers behind an amplifier can’t be used as a microphone with just a software change. And if you’re worried about malicious hardware that would allow that, then you might as well be worried about an extra hidden microphone.


Why can't they be? What's the physical mechanism behind it not working? Yelling into a speaker attached to an amplifier definitely produces electrical changes on the output transistors, changing the amount they're biased, etc. Does that produce no measurable input if the speaker is connected to a software-switchable input/output port? It doesn't have to be a large effect to be useful.

Although at that point I think I'm more worried about the microphonic properties of ceramic capacitors in the signal path.


This may be a horribly naive question, but do computers have the circuitry/sensors required to treat speakers as an input device?


Sure. Try setting a pair of headphones as a mic and then talking into them.


If I understand what you're suggesting properly, that's really not what I meant. I've wired up speakers and used them as microphones as a kid. What I meant is, do you really need to cut the cord to your speakers in say a laptop to stop the possibility of them being used as speakers? Aren't your speakers output only as far as the computer is concerned?


I've seen audio devices where it's the same socket and you choose in software whether it's an output or input.

I think it was realtek built-in on a hp SFF PC.


Shouldn’t it be possible to disallow this on the level of the sound card?


A "firmware update" hardware switch is challenging to implement. A "read only" switch means you have to separate your firmware and your configuration into two separate storage devices.

Hardware switches are easier for microphones and cameras, because you literally cut the power for a device.


> A "firmware update" hardware switch is challenging to implement.

No, it’s not. The actual low-level chip on the flash has a separate pin that must be connected to ground to enable writes.


Often, the same chip is used for more than just firmware. For instance, for UEFI firmware AFAIK it's common to have the UEFI variables stored on the same flash chip; not being able to write to these variables will break more than just firmware update.


Yeah, it’s a bit more nuanced in practice. Most chips now have the ability to specify ranges that are locked or unlocked which then have different requirements for what it takes to write to them, and treat the /W line differently depending on that configuration. But they’re also 20c parts, so using two chips isn’t crazy (many use multiple either as backup or for the different components anyway).


I don’t know how PC hardware does it, but microcontrollers typically have separate “flash” (large, less write cycles, requires complex rituals to write, executable) for programs and “EEPROM” (small, more write cycles, requires little if any preparation to write, often non-executable) for configuration and (very lightweight) logging. Prohibiting writes to the former but not the latter shouldn’t be particularly difficult, although I’ve yet to see a chip that would actually do it.


Isn't it actually the other way around? EEPROM (electronically-erasable programmable read-only memory) is limited in write cycles, requires a lot of preparation to write, and is usually executed directly on the processor it's connected to. In comparison to EEPROM, flash memory is larger, can endure more write cycles and has a less complex ritual to write. Flash memory is used to store anything, even user data.

Then there's NVRAM (non-volatile random access memory), which is usually smaller than EEPROM but has infinite write cycles, requires very little preparation to write (since it's just RAM), and is often used to store configuration data, not code.


Yes, it doesn’t mesh well with historical usage of these terms, but that’s what (e.g.) AVR and STM8 manuals call it, so that’s what I’m used to. From what I understand “flash” (program memory) is NAND flash with ~ 10⁴ write cycles and larger erase blocks, connected to the instruction bus, and “EEPROM” is NOR flash with ~ 10⁵ write cycles and smaller erase blocks (sometimes single bytes), connected to the data bus or even accessed indirectly.

> Then there's NVRAM

I was under the impression that NVRAM is less of a specific technology and more of a desideratum. As for implementations of it aside from ages-old battery-backed RAM, there’s FRAM used in some TI microcontrollers and (Wikipedia tells me) some other stuff, but it’s all patented to hell and back so we’re unlikely to see any of it in general use (although TI microcontrollers are admittedly lovely).


This is actually why bios chips fail so often in personal computers.


Is this true? I have repaired a lot of computers since I were young and non of them had hardware BIOS related problems.


I’ve had three laptops fail in the last five years due to a flash rom chip getting corrupted after saving changes in the BIOS one time too many. Enterprise warranty covered an HP and a Dell service rep to come on site and swap out the motherboard+cpu+gpu combo twice, I swapped out the rom myself the third time.

I’ve also had to do it on a desktop Gigabyte motherboard circa 2009 after a successful BIOS update left the flash rom unstable.


I'll tell you what!

You make an embedded Linux device with a read only partition based on a hardware switch. You figure out all the bugs that are caused by software not being able to write temporary files to disk. You figure out how to do configuration management on a separate system with something more complicated than a ten line YAML file.

Want to change your password? That's /etc/shadow -- did you some how rig that up to be writeable, while the rest of /etc was not? Also, since I presume your management decided to not let the users have root, because of course they did... You'll need to resort to software tricks to make sure the user can't change the root password.

Oh, and remember. No software read only tricks. Hardware switch.

Please let me know when you finish, I'll help audit your system.

Last edit: To all the reply guys, yes. I know it's possible. My statement is it isn't easy, and there are many challenges. (Especially compared with the simplicity of a power cut switch to a webcam.)

I can make you a microcontroller with a firmware update switch that blinks a light. By the time you scale that up to a full fledged embedded Linux system with a board designed in house, with weird hardware that is keeping you back on Linux 3.16 because nobody knows how to port your drivers, with cryptographically signed updates, fault tolerant firmware slots, and a nasty stack of software developed by web devs that can't fathom why they can't write to disk, that has to interoperate with legacy hardware and systems, that has a management bureaucracy that can't understand why it's taking so long to implement the new media server plugin, and devices in the field aren't getting automatic updates...

No. No it's not easy. Part way through, management will kill the project, you'll end up with a switch that's read in software, and eventually wind up on the front of HN as someone who did security wrong.

But by all means, take your "easy" idea to WD and tell them you'll have it working on their devices by Q1 2022.


I’ve already done it, and it’s not that hard. Others have done more. It’s the best way to avoid SD card or flash write wear. My production devices default to read-only mode and must have a dip switch toggled before any changes persist beyond the shadow ram-resident overlay that resets at power cycle.

(Aside: As for my idea of a configuration system, I’ve developed entire [incremental!] build systems that take a kernel source tree and configuration files and generate fully boot-ready images with drivers, packages, and even GUI support down to specifying the themes and customizing panel layouts, and more via a fully declarative syntax. The images have been booted on commodity hardware not under our control spanning some twenty-plus years of technology on more than a 100k machines. This is HN: not everyone is merely an armchair expert in whatever the topic of discussion is for today. It can be beneficial to assume expertise is out there and seek it rather than deny things are possible.)


Have you ever used a Linux LiveCD? Or booted a machine off the network? The latter is a very common way to operate servers.

In both cases you can write to the filesystem just fine. The writes just stay in RAM and don't get committed to disk.

There are cons to this approach, but you've listed none that apply in the real world


I've built a BusyBox image that TFTPs over to do the initial firmware flash, all ramdisk based. I've got physically write protected ICs on my boards. I almost rigged up my board to do write once NOR flash for U-boot. I know read only systems can be built, and everything else can be tmpfs. (And infact, I've built them.)

People seem to be thinking I'm saying this is impossible. I'm not, I never did. I'm sorry I'm frustrated, but it's difficult to respond to things you didn't say.

I'm saying, compared to a power cut switch for a webcam (which, I seem to remember even Apple screwed up accidently), a write protect switch is more challenging.

A power cut switch is mostly challenging mechanically. How do I get the dang thing on the case? But otherwise, that's the only consideration.

For a truly hardware based write protect switch that disables write capabilities at the silicon level, you have to adapt your image, your software, your hardware, and many of your procedures for the bring up process.

Is that challenging? For some people in this thread, I suppose not. But compared with a power cut? Orders of magnitude more challenging. Especially when you are bringing this to a massive codebase that hasn't had this as a design consideration.


You can use an overlay filesystem to do this or do like CoreOS or ChromeOS and have a read only root with necessary symlinks to a writable directory. Systemd also has helpers for this.


Dude, this is basic overlay filesystem stuff / just look at every live image ever.


I thought that was a solved problem w/ "unionfs" or such. Lots of Linux-based devices boot from read-only media.


Challenging, but almost always possible.

Most flash chips have a write-enable line that you can put a switch on. Usually have to cut a trace but often can avoid soldering right to the legs by following traces.

Was a common thing to do to receivers (“Integrated Receiver Decoders”) back in the paytv days. Thankfully they had firmware on a parallel eeprom and config stuff on a smaller serial eeprom (that could handle 1m writes instead of 1k writes). Receivers could have a lot of wires especially after they implemented some lock-detection that had to be countered with some 74ac logic that could disrupt the 2nd step of starting a write job.

Should be doable for something like a router or cable modem, but maybe not on something like these WD drives. Like a mod chip without having to worry about the vendor trying to counter you.

Of course you’re still screwed if something is only non-persistent but at least any issues are resolved with a simple reboot.


The firmware and configuration are already split into several devices. These machines have Arm Cortex A9 and similar processors that go through several stages to start up:

First, some internal boot ROM, likely with fuses burned in for the particular IO configuration, reads a bootloader (likely Das U-boot) from external flash memory. That first-stage bootloader initializes the parallel/SPI NAND/NOR flash interface and DRAM controllers, and then launches the second-stage bootloader. The second-stage bootloader uses those memory controllers to read the firmware image out of memory into RAM, then executes it.

If you want to update the firmware - more precisely, to change the location or signature of the image that should be loaded by the second-stage bootloader - it would be trivial to add a check for a GPIO switch to allow or deny changes.


Challenging? I think you mean fun.


I'm genuinely curious -- is there any empirical evidence to show that's the most effective approach?

Because then the firmware can never auto-update, but needs to be manually and explicitly done -- flick the switch, apply the update, flick again.

And clearly a significant proportion of people (probably a very large majority if we're being honest) will simply never update firmware.

So which is the bigger threat: unpatched firmware, or firmware auto-update vulnerabilities?

The answer doesn't seem intuitively obvious at all to me. But there must be stats available -- frequencies and severities of vulnerability categories, and how often people update firmware on non-auto-updating devices. So it doesn't seem terribly hard to compute an answer?


Split the difference. Automatically update devices until they are end-of-lifed, then send a last update that blows a fuse allowing automatic updates. Anything after that point requires a write switch to be flipped.

Actually, preferentially they should be taken offline and the consumer should have to opt-in to leaving them connected to the Internet, but that's a whole separate issue.


I don't know, that doesn't seem like splitting the difference -- it seems like it might be the worst of both worlds.

Before EOL auto-update is a vulnerability, and after EOL security patches might still be made available for the absolute worst vulnerabilities, but now wouldn't get to practically anyone.


That might add an additional 1$ to the BOM. In quantity of 10 million, that's a lot of extra money for a feature that maybe 100 people will use.

Not arguing against the idea, just saying that the economics will never work in favor of this.


That's why I've suggested that every time you buy a disk drive, when posting a review, take off 1 star for no write-enable switch.


I suggest everyone who has a use for such a switch do that. But I think they already do that.


Add $1 to BOM, add $2 to price, problem solved.


As someone that works with OEMs in the semiconductor business... I wish this were true.


"Cue", not "Queue".


Fixed.


And then not a single user ever updates their firmware which is probably even worse than auto updates. The real answer is crappy OEM hardware should not be exposed to the internet. Put it behind a gateway by a decent vendor like Apple or Google who will make sure it stays secure.


> We strongly encourage moving to the My Cloud OS5 firmware,” the statement reads. “If your device is not eligible for upgrade to My Cloud OS 5, we recommend that you upgrade to one of our other My Cloud offerings that support My Cloud OS 5.

Not sure how this isn’t illegal. You sell something so defective that it destroys the thing it’s designed to protect and you refuse to fix it, and rather use it as a chance to force customers to buy new devices that are likely just as bad


>Not sure how this isn’t illegal.

Many people believe that regulations on companies stifles innovation, so this is what we get. Apparently, it's your own fault if you bought a defective product.


Quantity of regulation is not the issue. It's the quality of the existing tens of thousands of pages.


While no regulation is bad, regulation is often bad or worse. Hard to know where to point a finger.


People screwed by this are surely going to buy more WD. /s


Sadly they probably will. At least the ones who didn't have their shit totally deleted.

So many "average" users just want consistency and will go with WD again because they don't need to relearn as much (even if the relearning is minimal it's still a mental barrier for anyone who does not feel totally technically competent).

I think of my parents who, despite being very smart people, are frustrated by tech because it doesn't come easy to them. Any extra step isn't beneficial, it's stressful.


If they're smart, they'll buy one with just a USB port.

If they're even smarter, they'll buy one with just a USB port and then rescue the 3.5-inch drive from its plastic prison.


Depending on how long ago the products were purchased new from the store, here you could claim that the device did not last its reasonable expected lifespan under the consumer protection laws.

You can also make a claim that the product contains a flaw that must be fixed, or the sale should either be retroactively discounted, or even cancelled. I have managed to cancel a sale on a product after its warranty expired due to a software issue that the manufacturer claimed was a feature, but which the consumer protection agency ruled was against reasonable consumer expectations that if it was a feature, it should have been clearly laid out for the consumer.


Because we are only just seeing the results of a new wave of tech. You didn't have to worry about your hdd not getting firmware updates and being hacked before so there is no law about it.


Continued security updates being somewhat important for hardware you bought has been a topic since the Internet became ubiquitous. So perhaps for two decades or so by now? That's plenty of time to upgrade regulations.


How many times do people need to be burned by closed-source, cloud boxes before they learn to stop buying them?

Western Digital deserves their fair share of blame here as always but honestly the pattern of failure and consequences here is pretty well established by now.

Rolling your own remote access solution(SSH/VPN+ strict FW rules) that can be used in conjunction with your own DIY raspberry pi network share(SMB+external drive USB or docked HDD) service is just really well documented in so many articles and is very maintenance free once you cronjob the updates.

It is time to own your digital destiny people. The stakes have always been high enough to justify the time and effort. Just do it!


> How many times do people need to be burned by closed-source, cloud boxes before they learn to stop buying them?

> Rolling your own remote access solution(SSH/VPN+ strict FW rules) that can be used in conjunction with your own DIY raspberry pi network share(SMB+external drive USB or docked HDD)

These target completely different audiences.


You can also just buy a regular hard drive that isn't connected to the cloud. But where's the fun in that, I guess?


You could also just pay a few $ a month for cloud storage and be much safer than any home made or self hosted setup.


Then it's no longer just your data. Someone else now also has a copy. How do you know they don't leak it or provide it to someone? There's value in hhavingyour data only local with some off-site arrangement.


Backblaze's personal backup has a feature to use your own private key to encrypt your backup data before transmission.


I trust Google and Apple to secure my data more than I trust myself.


I trust Google to randomly lock me out because their stupid AI determined that I'm a suspicious geek instead of a normal person. It's happened before, it will happen again.

Very secure but not in my hands. No thanks.


If the government wants my data, they can just raid me and take my home server. I trust that google can secure it from random hackers better than I can.


Arq Backup will encrypt your data (supports a bunch of different backends Google, AWS, etc, including your own)


3-2-1...

You also need the first two, really.


Could anyone recommend a specific foss stack + guide for setting this up for somebody who has no idea how to set it up? I’m most concerned about misconfiguring something, which is sort of what this Wd exploit is - somebody misconfigured an account to not have a password in this case. I can only assume they forgot to do that step, or didn’t know how to avoid doing so

What software do you use to push your files from your windows/Linux machines? How do you test your backups most easily? How do you test you aren’t leaving your device exposed?


This worked for me:

https://www.howtogeek.com/139433/how-to-turn-a-raspberry-pi-...

I don't remember if all the instructions worked precisely without a few tweaks, as the Raspberry Pi software has changed a bit since this was written. But at the very least it's worth just perusing the article to see if this is something you'd like to tackle.

I have a Raspberry Pi 4 with a (Western Digital, yeah I know) USB3 hard drive, that is a file server for my family's home network. I have not set up automatic backups, but do it manually by SSH'ing into the RPi periodically. The Pi 4 doesn't seem to like powering two drives at once, so I plug the drives into a powered USB3 hub.

There may be better ways of doing this, but of course mental inertia has set in, since it works and has been trouble free.


A truenas mini is the fastest way there.

Reading zfs and truenas documentation then building your own is the second fastest.


Looks like $700 USD entry price? Might be worth it but seems overkill for a lot of people. I will read those docs however to see about building my own, thanks for the tip


Or if you got your old desktop computer, that'll do too. I'm on my third iteration of retired-desktop-pc NAS, didn't buy anything except a couple of 10gbe nics on ebay.


Do you actually saturate the 10gbits via spinning disks or is it more for the reduced latency?


Well ZFS is fairly good at caching, so while I might not be able to saturate the 10gbit/s from the disks directly all the time, it's still a noticeable jump up from "merely" 1gbit/s.

So depending on which disks is hit, I can get 300-500MB/s for uncached data.

However when copying to the NAS, it can saturate as long as there's room in the RAM cache.

In sum it was a quite worthwhile jump in performance given the investment of about $30 or so, even if it's "only" 3x in some cases.


Used workstations (hpe proliant, dell poweredge tower, etc.) on ebay plus 4x 4 TB hard drives clocks in around $700 too. Couple it with something like B2 or S3 replication and your data is safe and secure.

It ain't cheap, but you're buying reliability and privacy.


It doesn't buy locational redundancy, though; with that setup a fire is sure to take your drives with it unless you get an expensive fireproof NAS. 1tb via Google One or even Google Cloud is sure to be at least 5 times cheaper a year than getting 1tb hard drives in 2 extra continents.


How does "B2 or S3 replication" not solve this?

Using google guarantees your files are not private.


I'm not saying it doesn't, i'm saying it's cheaper. S3 or Google Cloud storage is going to be so much more expensive in this scenario. B2 is the same as Google One at $10/mo for 2TB, which doesn't include data re-downloading ($20 to redownload the full 2tb). The only benefit you get with B2 is that you only pay for what you use.

And I would only recommend consumer cloud storage in an encrypted fashion - cryptomator or rclone are great.


>I'm not saying it doesn't, i'm saying it's cheaper.

So I read this wrong?

>It doesn't buy locational redundancy, though; with that setup a fire is sure to take your drives with it unless you get an expensive fireproof NAS.

Also, B2's pricing is competitive with google and dropbox for 2 TB and under (within 50%). I haven't priced their larger tiers, but I'd be surprised if it wasn't also competitive.

I'd rather have my files sit on an encrypted volume that is easily accessible to me than try to live around integrating obscure higher level encryption schemes. It's a larger attack surface and takes integration with other software off the table.


I think your over-estimating how little most people think/care about their storage drives.


Which is why google drive or the ms version is the real solution for most people. Zero effort, low cost, automatically backed up, and has a huge security team keeping your data safe.


It looks low cost, until you realize you want to store your data for the rest of your life. 10 years, 100$ / year. And after 10 years, you either have nothing, or an old synology (or similar) + enough saved up for a new one. Same reason I stopped netflix at al. Imagine you're 75, retired, you either have a massive collection of films (digital) or .. nothing.


Self hosting doesn't last forever and doesn't have zero ongoing costs. At one point I was running my own nextcloud server at home and realized that I am paying more in electricity costs to keep it running than google drive costs. After uploading all of my personal data (no tv shows, etc) I only have about 50GB of data which costs me almost nothing to store and it means I don't have to worry about backups, hardware failures, hacking, etc.


Until your malware is getting copied to your (insert cloud provider)


No one capable of doing those things would have even bought one of these WD devices in the first place...


I seriously considered building my own NAS based around FreeNAS (something I'm perfectly capable of doing), but then decided to go for a commercially available, low-end NAS for two reasons:

  1. my tendency to scope creep on the hardware requirements meant that I was looking at a BOM that was about 3x the cost of the commercial NAS.
  2. it seemed likely that I'd spend a lot of time engineering my NAS and fighting compatibility issues with e.g. Time Machine. The commercial NAS had all the features I wanted out of the box.
Ultimately, I bought a low-end Synology NAS and have been pretty happy with it. I haven't been affected, and my device is still supported 7 years later, but my story could easily have turned out like these WD customers.

Some of us don't want to spend our free time maintaining a NAS.


> Some of us don't want to spend our free time maintaining a NAS.

The issue is that, for me anyway, it's often easier/faster to just set up something myself. Most of the time it's a "configure once"-thing and then it "just works" with just the occasional updates.

And if something does tend to go awry it's usually easy to diagnose and fix. If something goes wrong with one of those NAS black boxes it tends to be much more complicated. Or if I want to add $feature_x this tends to be fairly easy as well.

Of course, this vastly depends on your skill and what you use it for: I don't have a mac so I never tried Time machine. My point is just that for some of us at least, "building their own" is actually done for the same reasons: I want to spend as little time on this as possible.

Synology are pretty neat machines last I checked them out though, we used to sell quite a few of them (over 10 years ago). I stopped using my CentOS "NAS" when I moved a few years ago, but if I were ever to be interested in buying one I'd probably consider it as an option.


No way mom and pop are going to know how to do this. Even semi tech literate people will struggle.


I've seen plenty of non-tech people struggle with more advanced NAS solutions like Synology as well.


Heck, I've been using Linux for a decade and this is something I still aspire to do one day.


The typical buyer of this type of product has no idea what "closed source" means. They went to Harvey Norman and asked the 17 year old store assistant what they should buy to keep their important photos and documents safe.


> How many times do people need to be burned by closed-source, cloud boxes before they learn to stop buying them?

Probably when their thermostat turns off during a heatwave.


Do you have one article that you particularly like?


Products that rely on third party servers to function should be required to carry an expiration date that guarantees service and security patches up to that date.


They still support this device and there is an OS update that closes the vulnerability.

They are providing data recovery services to customers of the older devices. Would have been nice if they warned those customers about the vulnerability when they found out about it, even if the fix was to buy another $X00 product.


there is an OS update that closes the vulnerability

Not for all devices: The article indicates that some may not be compatible with OS 5 and that WD says those customers should buy a new one.


and its a full rewrite of the OS that is missing some functionality used by users.


who cares about missing functionality when compared with deletion of your data?

surely deletion of data is worse alternative to losing the ability to theme the web UI, or whatever.

this is why Microsoft has so many updates so often for Windows 10. security issues which require no intervention from the victim are VERY REAL, and when left alone, users will not update. this has been proven time and time again. A user can take no action and still be vulnerable today when they were not vulnerable yesterday. this WD instance is yet another example of users not knowing what is best for themselves; not knowing to update their devices, or to take their devices off of the internet.

there are secure, free, easy-to-setup ways to access files over the internet on a NAS which does not have internet access...

WD will hopefully force users to update in the future for internet connected devices, and for devices that go out of support, and can no longer receive updates, WD should take them off the internet as a final action, to protect the consumer.

THIS EXACT SITUATION is why updates should be forced on users.

nothing shoots itself in the foot as often or as thoroughly as a user that doesn't know what they're doing, believing they know what they're doing.


WD shouldn't be removing features that users rely on. Bundling feature loss with security updates is just bad practice.

If faced with losing functionality critical to the reason someone purchased a device vs. vague release notes that mention security updates, the average consumer in many cases is going to weigh the intangible risk of security problems pretty small against the guaranteed loss of required features.


> the average consumer in many cases is going to weigh the intangible risk of security problems pretty small against the guaranteed loss of required features.

what are those lost features? do any of those lost features include the unintentional loss of data or the inability to access said data? if not, if the user can maintain access to the stuff on the NAS after a security update, they should update, because there is no NAS security update that takes away your ability to access your data.

I really do wonder what these missing features are because there is zero likelihood that the ability to access the storage device itself is one of the lost features.


This exact situation is why users don't update.


> This exact situation is why users don't update.

users not updating CAUSED this situation.

the actual blame lies on the attackers, of course, and users who do not take security updates make this type of attack possible.


Vendors who make customers choose between features and security aren't helping. Removing stuff from the product and saying it's still "supported" is a little sketchy.


No, that's even worse. It's called planned obsolescence.

Instead, along the same lines as right to repair, such products should be required to release the firmware source code.


Planned obsolescence seems better than unplanned obsolescence? If your uniformed consumer were informed of the lifetime of the product they buy, they might not be conned into buying things your informed consumer is aware of and avoids.

That's good because, as of current, when the population of uninformed consumers drive market forces, they often push out the options informed consumers would choose or at the very least, create trends towards the uninformed bias purchases that force informed consumers to start choosing the same options as well or drive up prices for the products informed consumers often buy due to lessened demand.

You know, when a bunch of people decide we'll let businesses stop producing devices we can repair or put out rent-seeking price structures and the rest of people are forced to use those options, all because of large scale manipulation of consumer perception. Then we end up with markets filled with garbage with fluffy profit margins for their owners... Then again, cigarettes still have a large market somehow, so maybe we're out of luck either way.


>Instead, along the same lines as right to repair, such products should be required to release the firmware source code.

Releasing the source code doesn't necessarily mean that people would be legally permitted to modify or even utilize that source code.


It also doesn't mean anyone will actually support it. I have plenty of devices which you can flash your own OS on but eventually the hobby community gets bored and no one is left supporting it.

You also have the problem of getting every user of the product to flash some custom OS on their hard drive when they likely don't know how or don't know why they should care.


I feel like this is implied or should be. Even if not explicit, you should know you can't get unlimited software updated into eternity which is what I believe you're saying omission of an expiration date means.


Sure, I know that, but the average consumer doesn't. When an average consumer buys a thermostat for ten times what she paid for her last one she expects it to last twenty years like her last one did. When a ten year old buys a videogame he expects it to be playable until he throws it away or sells it. "People shouldn't be so naive" is not the correct response to the precipitous decline in the quality of consumer goods.


While I don't disagree with your example I would point out that caveat emptor is a core principle of commerce for quite some time. In times of rapid change/technology, the consumer doesn't know what they're buying. Before long, it will be common knowledge that "smart" everything has a cost that was not obvious to average Joe at purchase. But, Joe was also so impressed with the shiny object he didn't think about the consequences of things. Another point, in Texas, energy companies have been adjusting people's thermostats remotely when the grid is struggling. While I never thought of that specific thing, I have been anti-connecting-things-to-wifi because I know once you open it up, it's just a matter of time before someone else is mucking around with your devices. I'd totally expect it to be some script kiddies doing it for lolz instead of the electric company, but at that point, it doesn't really matter - you're not solely in control of your device.


So, for those who don't know: Your MyCloud will spam you six times a day about firmware updates if you didn't update. They replaced the vulnerable OS back in March. So people probably should know/have done this by now. It's hardly a zero-day at this point: It was fixed months ago.

Second, and I feel like this should be obvious: People should not be exposing their NAS appliance directly to the Internet! Stop doing it. Just don't. If you do, you deserve what you get, because you intentionally went into your consumer-grade firewall and poked a hole in it.


Except for the folks that can't update and WD won't provide a patch, telling them to buy a new device.


> They replaced the vulnerable OS back in March.

When a full rewrite that removed functionality, so some users aren't going to bother to update, and as far as I'm concerned, thats on WD, not the users.


It's still not a zero day: Fixed software has been available for months. And it's free.


For some hardware. For everybody else, WD says to buy new hardware.


I don't think you understand that they use dark patterns that default to the incorrect, dangerous, behavior.

Western Digital is not free of sin.


Are you sure it's always intentional? There might be some UPnP thing going on here.


Possible, though we should shame any routers that still allow uPnP. The horror. And MyCloud does default to local only communication.


Every time I read replies or comments from WD the less i want to buy anything from them again. Very disappointing as every few years i buy 1 drive that backsup all my previous backups plus the new stuff. So i guess all my drives are unsupported. Not buying any cloud solution, NAS ever. A company not taking care of its customers is either not worth investing or are about to go belly up anytime.


Best spinning hard drives though, especially after they acquired HGST.


The manufacturing of HGST got sold to Toshiba -- that's where you want to look for quality drives now. HGST is just another WD brand at this point.


Is there a better alternative? Or are the competitors just a day away from their own disaster event?


Good point, i did go for cheap and redundant with WD. Also i like to have spare power suplies, meaning 2 or 3 drives Within same line, will use the same. Did that with maxtor 15 years ago, those drives still work... but are 120GB. Not sure which is the brand to buy now. But it looks like is time for a change.


Nextcloud on a linux system is your best bet. And its a huge pain. You essentially have to learn devops and regularly check in to see everything is running correct. And then one mistake and your data is at risk. You also have to manage backups somehow which you want to be not on the same machine.


The real lesson is they aren't a software company.


> The researchers said Western Digital never responded to their reports.

> The communication that came our way confirmed the research team involved planned to release details of the vulnerability and asked us to contact them with any questions,” Western Digital said. “We didn’t have any questions so we didn’t respond.”

Lol. Is this entire company, from the developers to the people in charge of comms, complete idiots?

I guess this is what you get when you think software is nothing but a cost center then gut + outsource it.


Having worked there -- mostly. I engaged in multiple arguments with leadership whom wanted to measure engineer productivity based on lines of code added to version control.


My parents actually use this Western Digital MyCloud as a local backup because of concerns about data being exported out to cloud servers a la Apple, Microsoft, Google, etc. Are there any recommendations for good local backup solutions for middle aged people not great with tech?

Edit: Needs auto-backups, so it has to be more than a USB or old computer.


Synology NAS. Synology does a pretty good job updating their software, and it's a core part of their business.

With WD it's like they just wanted to bolt on some NAS features on the cheap and the result was the current mess.


Synology is a totally different price range though. You'd pay the same for the empty NAS as you'd pay for the WD with the drive included :)

But then again it's clear that you get what you pay for.


A two bay disk-less filled with ~2tb drives is only slightly more.


I've waffled between 2-bay and 4-bay for personal use. Currently on 2-bay, but semi-regretting - I feel like 4-bay would offer more notably more flexibility with drive expansion options. (Assuming single parity drive and SHR-1 in either case.)


Synology. QNAP is good too but Synology is probably the easiest to use and they have very strong and long-standing software support.

Edit: QNAP has had some security issues too. I’ve had Synology gear for close to a decade, interspersed with DIY servers and homelab stuff and really, really like it. If I were getting my parents a NAS/backup system, that’s what I would get.


I've thought about that exact scenario - getting my parents a NAS/backup system - and I'd go for Synology if my parents were within driving distance, but they're across the country, and I don't feel like it's quite at the point where it would be free of maintenance to the degree I'd want.

(Especially considering the covid situation, where I haven't been able to see my parents in a couple years now due to quarantine requirements.)


I own a Synology, but I'm still not opening it up to the internet. I use a Wireguard VPN on a RPI to access it.

It's a minor inconvenience, but I can sleep sound at night knowing my NAS isn't being wiped by a zero day.


Many home routers include an option to plug USB storage into it. From there you can just mount it on the computer and use the OS's built-in backup software: they all have some, automation included.

Edit: Seagate doesn't seem to make the option I mentioned for them. Removed.

A simple external USB drive will work though: Windows 10 has built-in automated backup capabilities. Actually it's been possible since at least XP.


Home routers aren't exactly known for getting regular firmware updates or being super-secure either.


Internal devices are typically isolation from the world though, unless you have NAT punch a hole through it. It doesn't matter (as much) if the router has security holes if a hacker can't get to it from outside the network. I don't recall hearing of any hacks that have pushed through the ISP's modems, down to the router, and into local devices without ports punched through NAT to the outside world.


An Intel NUC style computer with openmediavault, or other easy to use open source NAS solution is what I would use in that case.

If you don't care about the small size of a NUC, an old office PC with a couple hard drives should do well.


Sadly keeping a general purpouse server OS consistently secure and patched up is not realistic for "middle aged people not great with tech". I wonder if there are good affordable ways to outsource this...


Agreed. GP reads like the famous Dropbox comment. That just isn't realistic for people that aren't very tech literate.


If only the OS was simple to use and updated itself...

I have a gigabit connection and am disgrunted that I can't self-host most services I need without turning it into a 2nd job


I run Debian on a small file server in my parent's house. Granted I had to set it up for them, but after configuring unattended upgrades, I only needed to work on it to upgrade from Debian 8 to 10 because it was getting close to being EOL. I keep SSH open to the internet in case I need to troubleshoot something. Their computers automatically run weekly incremental backups and it's transparent to them.

It's not the best solution, but it works reasonably well with little maintenance on my part. On Windows you can set a smb drive to mount automatically at boot and it'll behave like a normal drive. So it was easy to explain to them that you can access that folder from both machines simultaneously.

I agree that this is not a good solution for someone that has to set it up themselves. In that case I'd recommend something like a Synology unit.


I'm surprised people recommend OMV. It's very "Web 1.0" with its user interface IMO.

I use it myself heavily but that's because you can install it on top of regular debian. So you get a NAS that you can customize to the wazoo. Which I do, it runs a lot of custom scripts. I basically use OMV only as an easy GUI for adding shares, changing out drives etc. I could do it all by hand and perhaps next time I will.

However I wouldn't choose to run it if I didn't have that requirement. There's much more modern options out there.

What made you choose it yourself?


A USB stick works well, or a USB drive if more space is needed.


The go-to standard for quite awhile. Unfortunately, it doesn't come with the convenience of auto-backups and runs the risk of being lost along with all the memories and data it contains.


Make a copy of your parents' backup and keep it yourself.

I have a friend whose grandparents took tons of film of him growing up. Then their house burned down, all lost. Give a backup to an offsite family member.


Those cloud service companies have better security than your parents' house. Especially if you use them with a backup tool that encrypts with a key only you have.


The video is pretty interesting, it looks like the nobody account was not meant as a backdoor, but the secret api is just doing authentication, without authorization. Couple that with having the api using the Linux auth and you have a problem. ...I wonder if it's doing pam or just reading the shadow file direcly, doesn't really matter

When I first read there was a backdoor account I thought it would be one that was on purpose. At an old job about 15 years ago we used network equipment that had a vendor backdoor built in. Only reason we knew it existed was one of our engineers had recorded a remote session with the vendor's support team. The account gave you full admin access and didn't even show up as another logged in user. It was disturbing to say the least.


Network equipment vendor name?


Carrier Access It was the Adit 600


A more precise (perhaps less clickbaity) title would mention "Western Digital MyCloud users". I thought of their HDDs first, not some of the related products they push (which one shouldn't expect to be secure anyway: being network-connected [black?]boxes, aiming non-tech-savvy users, by a company not specializing on that).

To rant a bit more about the title, I find it rather awkward (as a non-native speaker though) when an adjective that is commonly used with a noun becomes used as a noun, and instead of that noun: as "runtime error" in some contexts is replaced with just "runtime", or "0-day vulnerability" is commonly replaced with "0-day" (even when it's not that anymore). This practice seems to just create more confusion.


So, I've "owned" a PR4100 for 3 or 4 years. I wanted it because it supposedly supported hardware transcoding for Plex. Sadly the transcoding was limited to 5mbps h264-- the signal looked BAD. It was like watching confetti. Later the capability was removed altogether.

Which is why I haven't been affected by these 0days as of late-- the damned thing is useless and therefore turned off.


Why not return it when it didn't work well for Plex?


I happened to get it just before I moved, I just didn't have time to deal with it in the window-- and the other fellow is correct. I had assumed that it would be possible to tweak it to get better quality.


Odds are by the time he was done mucking around with all possible plex settings, the return period has passed


The main thing that struck me about this, is that they only supported their NAS for 5 years? It's a NAS, wouldn't the expectation be that people are running this for 10-15 years?


I've never had any problems with external WD drives on mac. However, my Seagate 4Tb is almost unusable. It corrupts my final cut file every time I'm editing off of it. It will randomly disconnect, such that its still mounted under /Volumes/ but its not actually there. Not sure if its overheating. So wanted to go back to WD, but not sure now.


IMO if they have a point in time where they decide they will no longer provide security updates, they should adjust the MTBF calculation, setting the maximum possible lifetime to be the EOL on the software.


That sounds ecologically disastrous. Although I remember when I once purchased a TV when they were still tubes, there was an additional €9 recycling fee at the time of purchase.


Not if they extend the software EOL to match the life of the mechanicals.


This is why I just let the NSA do my backup management for me.


Easy to recover the backup using a FOIA request too!


I have a friend who is considering a refurb PC with openmediavault as a replacement for one of these. She isn't using the WD remote access tools, so it's not a security issue with the product, but more like an old-OS issue.

I'm not sure if she plans to shuck the drive for use in the new system, and am wondering if shucking is pretty easy or not...

Does anybody have experience with OMV on this kind of setup? It made me curious.


Not all drives are "shuckable". But this is pretty common, you see people posting shuckable drives on Reddit often when there's a good sale.


On the other side I would like to thank synology for 10 years of updates that always worked. This is the way it should be and why I recommend them.


We may be using different Synology products. Updates frequently break filesharing or drive sharing or encryption for me.


It's not just WD NAS that are facing issues like this. I have a QNAP, so I follow news on that and they've been getting hit repeatedly with ransomware cryptolockers recently as well.

It's nearly always UPNP that's causing the device to be exposed unknowingly to the internet and then a some software bug that allows the exploit.


"The people pay for the newest version, and the newest version fixes the vulnerability." -- Mitch McConnell


WD has been cutting engineering corners for years. It’s finally catching up.


There's no indication EasyStore is affected, right? Assuming not using any WD Backup software.


The share price seems to hold up pretty well despite these revelations.


when this whole unfortunate thing is done, will there be a lot of cheap WD nas drives on ebay?


IOUT

Internet of Unsuitable Things.


Who still uses this crap?!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: